Discover ofirpress/sandwich_transformer Open Source project

Stars
55
Rank 537,381 (Top 11 %)
Language
Python
License
Other
Created over 4 years ago
Updated almost 4 years ago

ofirpress

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

This repository contains the code for running the character-level Sandwich Transformers from our ACL 2020 paper on Improving Transformer Models by Reordering their Sublayers.

attention_with_linear_biases

Code for the ALiBi method for transformer language models (ICLR 2022)

Python

493

YouMayNotNeedAttention

Code for the Eager Translation Model from the paper You May Not Need Attention

Python

292

self-ask

Code and data for "Measuring and Narrowing the Compositionality Gap in Language Models"

Jupyter Notebook

290

shortformer

Code for the Shortformer model, from the ACL 2021 paper by Ofir Press, Noah A. Smith and Mike Lewis.

Python

146

UsingTheOutputEmbedding

Code for the EACL paper "Using the Output Embedding to Improve Language Models" by Ofir Press and Lior Wolf

Lua

0plot

Use 0plot to automatically build matplotlib plots using ChatGPT.

JavaScript

PartialShuffle

Python

tstl_t5_bias

This is our implementation of the T5 bias for fairseq.

Python

ofirpress/sandwich_transformer

ofirpress

Reviews

Repository Details

More Repositories