• Stars
    star
    535
  • Rank 82,940 (Top 2 %)
  • Language
    Assembly
  • License
    Apache License 2.0
  • Created over 1 year ago
  • Updated 6 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Repo for "Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture"

Monarch Mixer

M2 logo

Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture
Daniel Y. Fu, Simran Arora*, Jessica Grogan*, Isys Johnson*, Sabri Eyuboglu*, Armin W. Thomas*, Benjamin F. Spector, Michael Poli, Atri Rudra, and Christopher Ré.
arXiv | M2-BERT blog post

M2-BERT checkpoints:

Updates:

  • October 21: M2-BERT-large checkpoints are now up on HuggingFace (260M, 341M). The 260M model matches BERT-large in GLUE fine-tuning with 24% fewer parameters, and the 341M model outperforms BERT-large.
  • October 18: M2 paper is now up on arXiv, and will be presented at NeurIPS as an oral!

Transformers have taken the world by a storm! The architecture is composed of two core operations: Attention for mixing information across the input sequence and MLPs for mixing information across the model dimension. Each operator scales quadratically -- the complexity of Attention is quadratic in sequence length and the complexity of an MLP is quadratic in model dimension. Ideally, we can have alternatives that scale more efficiently, while preserving Transformer-level quality. Towards this goal, we've been developing Monarch Mixer (M2), a framework for training models that are sub-quadratic in both sequence length and model dimension.

M2 diagram

Our basic idea is to replace the major elements of a Transformer with Monarch matrices — which are a class of structured matrices that generalize the FFT and are sub-quadratic, hardware-efficient, and expressive. In Monarch Mixer, we use layers built up from Monarch matrices to do both mixing across the sequence (replacing the Attention operation) and mixing across the model dimension (replacing the dense MLP). This repo includes code and models for training Monarch Mixer architectures!

Current Contents

October 21: M2-BERT-large checkpoints are now up on HuggingFace, and the paper is on arXiv!

July 24: We are excited to release Monarch Mixer BERT (M2-BERT), which has 25% fewer parameters/FLOPs than BERT, and matches in average quality on the GLUE benchmark. The BERT folder includes code for pretraining and finetuning BERT baselines and M2-BERT. We also release pretrained checkpoints at 128 sequence length for an 80M parameter BERT, which matches the average GLUE benchmark score of the BERT-base-uncased 110M parameter model, and a parameter matched M2-BERT model.

Citation

If you use this codebase, or otherwise found our work valuable, you can cite us as follows:

@inproceedings{fu2023monarch,
  title={Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture},
  author={Fu, Daniel Y and Arora, Simran and Grogan, Jessica and Johnson, Isys and Eyuboglu, Sabri and Thomas, Armin W and Spector, Benjamin and Poli, Michael and Rudra, Atri and R{\'e}, Christopher},
  booktitle={Advances in Neural Information Processing Systems},
  year={2023}
}

You can also cite our previous work that this repository builds on:

@article{poli2023hyena,
  title={Hyena Hierarchy: Towards Larger Convolutional Language Models},
  author={Poli, Michael and Massaroli, Stefano and Nguyen, Eric and Fu, Daniel Y and Dao, Tri and Baccus, Stephen and Bengio, Yoshua and Ermon, Stefano and R{\'e}, Christopher},
  journal={arXiv preprint arXiv:2302.10866},
  year={2023}
}
@article{fu2023simple,
  title={Simple Hardware-Efficient Long Convolutions for Sequence Modeling},
  author={Fu, Daniel Y. and Epstein, Elliot L. and Nguyen, Eric and Thomas, Armin W. and Zhang, Michael and Dao, Tri and Rudra, Atri and R{\'e}, Christopher},
  journal={International Conference on Machine Learning},
  year={2023}
}
@inproceedings{fu2023hungry,
  title={Hungry {H}ungry {H}ippos: Towards Language Modeling with State Space Models},
  author={Fu, Daniel Y. and Dao, Tri and Saab, Khaled K. and Thomas, Armin W.
  and Rudra, Atri and R{\'e}, Christopher},
  booktitle={International Conference on Learning Representations},
  year={2023}
}

More Repositories

1

flash-attention

Fast and memory-efficient exact attention
Python
3,673
star
2

deepdive

DeepDive
Shell
1,957
star
3

ThunderKittens

Tile primitives for speedy kernels
Cuda
1,555
star
4

state-spaces

Sequence Modeling with Structured State Spaces
Jupyter Notebook
1,372
star
5

data-centric-ai

Resources for Data Centric AI
TeX
1,099
star
6

safari

Convolutions for Sequence Modeling
Assembly
867
star
7

meerkat

Creative interactive views of any dataset.
Python
826
star
8

hgcn

Hyperbolic Graph Convolutional Networks in PyTorch.
Python
597
star
9

hyena-dna

Official implementation for HyenaDNA, a long-range genomic foundation model built with Hyena
Assembly
585
star
10

ama_prompting

Ask Me Anything language model prompting
Python
538
star
11

H3

Language Modeling with the H3 State Space Model
Assembly
513
star
12

evaporate

This repo contains data and code for the paper "Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes"
Python
479
star
13

manifest

Prompt programming with FMs.
Python
440
star
14

pdftotree

🌲 A tool for converting PDF into hOCR with text, tables, and figures being recognized and preserved.
Python
431
star
15

metal

Snorkel MeTaL: A framework for training models with multi-task weak supervision
Python
423
star
16

fonduer

A knowledge base construction engine for richly formatted data
Python
408
star
17

aisys-building-blocks

Building blocks for foundation models.
377
star
18

hyperbolics

Hyperbolic Embeddings
Python
372
star
19

legalbench

An open science effort to benchmark legal reasoning in foundation models
Python
341
star
20

flyingsquid

More interactive weak supervision with FlyingSquid
Python
313
star
21

flash-fft-conv

FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores
C++
276
star
22

KGEmb

Hyperbolic Knowledge Graph embeddings.
Python
249
star
23

bootleg

Self-Supervision for Named Entity Disambiguation at the Tail
Python
213
star
24

based

Code for exploring Based models from "Simple linear attention language models balance the recall-throughput tradeoff"
Python
209
star
25

HypHC

Hyperbolic Hierarchical Clustering.
Python
192
star
26

fly

Python
191
star
27

TART

TART: A plug-and-play Transformer module for task-agnostic reasoning
Python
190
star
28

tanda

Learning to Compose Domain-Specific Transformations for Data Augmentation
Python
171
star
29

hippo-code

Python
166
star
30

butterfly

Butterfly matrix multiplication in PyTorch
Python
164
star
31

spacetime

Code for SpaceTime 🌌⏱️. Proposed in Effectively Modeling Time Series with Simple Discrete State Spaces, ICLR 2023.
Python
163
star
32

zoology

Understand and test language model architectures on synthetic tasks.
Python
160
star
33

lolcats

Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"
Python
154
star
34

babble

A system for generating training labels via natural language explanations
Python
146
star
35

EmptyHeaded

Your worst case is our best case.
C++
138
star
36

domino

Python
134
star
37

blocking-tutorial

C++
132
star
38

mindbender

Tools for iterative knowledge base development with DeepDive
CoffeeScript
117
star
39

reef

Automatically labeling training data
Jupyter Notebook
106
star
40

fm_data_tasks

Foundation Models for Data Tasks
Python
100
star
41

fonduer-tutorials

A collection of simple tutorials for using Fonduer
Jupyter Notebook
100
star
42

eclair-agents

Automating enterprise workflows with multimodal agents
Jupyter Notebook
92
star
43

TreeStructure

Table Extraction Tool
Jupyter Notebook
90
star
44

CaffeConTroll

C++
76
star
45

epoxy

Interactive Model Iteration with Weak Supervision and Pre-Trained Embeddings
Python
76
star
46

HoroPCA

Hyperbolic PCA via Horospherical Projections
Python
68
star
47

structured-nets

Structured matrices for compressing neural networks
Python
66
star
48

hidden-stratification

Combating hidden stratification with GEORGE
Jupyter Notebook
62
star
49

numbskull

Numba-based version of DimmWitted Gibbs sampler
Python
46
star
50

prefix-linear-attention

Python
44
star
51

model-patching

Model Patching: Closing the Subgroup Performance Gap with Data Augmentation
Python
42
star
52

skill-it

Skill-It! A Data-Driven Skills Framework for Understanding and Training Language Models
Jupyter Notebook
41
star
53

cs145-notebooks-2016

Public materials for the Fall 2016 offering of CS145
Jupyter Notebook
35
star
54

mandoline

(ICML 2021) Mandoline: Model Evaluation under Distribution Shift
Python
31
star
55

mongoose

A Learnable LSH Framework for Efficient NN Training
Python
30
star
56

thanos-code

Code release for the paper Perfectly Balanced: Improving Transfer and Robustness of Supervised Contrastive Learning
Python
28
star
57

ukb-cardiac-mri

Weakly Supervised MRI Series Classification for the UK Biobank
Python
25
star
58

tuffy

Tuffy, a Markov Logic Network solver
Java
24
star
59

snorkel-superglue

Applying Snorkel to SuperGLUE
Jupyter Notebook
23
star
60

correct-n-contrast

Official code repository for Correct-N-Contrast
Python
21
star
61

ludwig-benchmarking-toolkit

Ludwig benchmark
Python
19
star
62

smallfry

Python
19
star
63

tabi

Code release for Type-Aware Bi-Encoders for Open-Domain Entity Retrieval
Python
19
star
64

lp_rffs

Low precision random Fourier features for kernel approximation
Python
19
star
65

ddlog

Compiler for writing DeepDive applications in a Datalog-like language — ⚠️🚧🛑 REPO MOVED TO DEEPDIVE 👇🏿
Scala
19
star
66

wonderbread

WONDERBREAD benchmark + dataset for BPM tasks
Jupyter Notebook
19
star
67

augmentation_code

Reproducible code for Augmentation paper
Python
18
star
68

sampler

DimmWitted Gibbs Sampler in C++ — ⚠️🚧🛑 REPO MOVED TO DEEPDIVE 👉🏿
C++
17
star
69

random_embedding

Python
16
star
70

snorkel-biocorpus

Python
16
star
71

ddbiolib

DeepDive Biomedical Tools
Python
15
star
72

bazaar

JavaScript
14
star
73

Omnivore

Omnivore Optimizer and Distributed CcT
C++
13
star
74

anchor-stability

A study of the downstream instability of word embeddings
Jupyter Notebook
12
star
75

medical-ned-integration

Cross-domain data integration for named entity disambiguation in biomedical text
Python
11
star
76

dd-genomics

The Genomics DeepDive project
Python
11
star
77

embroid

Embroid: Unsupervised Prediction Smoothing Can Improve Few-Shot Classification
Jupyter Notebook
11
star
78

torchhalp

Python
10
star
79

dimmwitted

C++
10
star
80

Accelerated-PCA

Accelerated Stochastic Power Iteration with Momentum
Jupyter Notebook
9
star
81

liger

Liger: Fusing Weak Supervision and Model Embeddings
Python
9
star
82

cross-modal-ws-demo

HTML
9
star
83

hyperE

HTML
8
star
84

treedlib

Jupyter Notebook
8
star
85

ivy-tutorial

An Introductory Tutorial for Ivy
Jupyter Notebook
7
star
86

observational

Observational Supervision for Medical Image Classification using Gaze Data
Jupyter Notebook
7
star
87

chinstrap

C++
6
star
88

quadrature-features

Code to generate kernel features using Gaussian quadrature
Python
6
star
89

icij-maude

Weakly supervised classification of adverse event reports from the FDA's MAUDE database.
Python
6
star
90

librarian

DeepDive Librarian for managing all data sets we publish and receive
Python
3
star
91

halp

Python
3
star
92

bert-pretraining

Python
2
star
93

d3m-model-search

D3M Model Search Component
Python
2
star
94

elementary

Data services and APIs
Python
1
star
95

dependency_model

Structure learning code from [ICML'19 paper](https://arxiv.org/abs/1903.05844)
Python
1
star