ofirpress/self-ask

Stars
290
Rank 142,981 (Top 3 %)
Language
Jupyter Notebook
License
MIT License
Created about 2 years ago
Updated 11 months ago

ofirpress/self-ask

ofirpress

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Code and data for "Measuring and Narrowing the Compositionality Gap in Language Models"

self-ask

This repo contains the code and data for our "Measuring and Narrowing the Compositionality Gap in Language Models" paper.

You can read more about self-ask prompting here (including a 2 minute demo video).

Or read more about our Compositional Celebrities dataset, and the Compositionality Gap here (also includes a 2 minute video.

attention_with_linear_biases

Code for the ALiBi method for transformer language models (ICLR 2022)

YouMayNotNeedAttention

Code for the Eager Translation Model from the paper You May Not Need Attention

shortformer

Code for the Shortformer model, from the ACL 2021 paper by Ofir Press, Noah A. Smith and Mike Lewis.

sandwich_transformer

This repository contains the code for running the character-level Sandwich Transformers from our ACL 2020 paper on Improving Transformer Models by Reordering their Sublayers.

UsingTheOutputEmbedding

Code for the EACL paper "Using the Output Embedding to Improve Language Models" by Ofir Press and Lior Wolf

0plot

Use 0plot to automatically build matplotlib plots using ChatGPT.

PartialShuffle

tstl_t5_bias

This is our implementation of the T5 bias for fairseq.