google-research/FLAN

Stars
1,460
Rank 32,218 (Top 0.7 %)
Language
Python
License
Apache License 2.0
Created over 3 years ago
Updated 3 months ago

google-research/FLAN

google-research

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

The FLAN Instruction Tuning Repository

Original Flan (2021) | The Flan Collection (2022) | Flan 2021 Citation | License

This repository contains code to generate instruction tuning dataset collections. The first is the original Flan 2021, documented in Finetuned Language Models are Zero-Shot Learners, and the second is the expanded version, called the Flan Collection, described in The Flan Collection: Designing Data and Methods for Effective Instruction Tuning and used to produce Flan-T5 and Flan-PaLM.

Flan 2021

To generate the Flan 2021 data as Seqio mixtures, first install the relevant requirements.txt then use mixtures.py.

Flan 2021 Citation

Please cite the following if you found Flan 2021 useful in your research.

@inproceedings{weifinetuned,
  title={Finetuned Language Models are Zero-Shot Learners},
  author={Wei, Jason and Bosma, Maarten and Zhao, Vincent and Guu, Kelvin and Yu, Adams Wei and Lester, Brian and Du, Nan and Dai, Andrew M and Le, Quoc V},
  booktitle={International Conference on Learning Representations}
}

License

The code in this repository is licensed according to the LICENSE file.

Contact Us

To contact us feel free to create an Issue in this repository, or email the respective authors that contributed to this code base: Jason Wei for the Flan 2021 paper, Le Hou for the Scaling Flan paper, and Shayne Longpre for the Flan Collection.

bert

TensorFlow code and pre-trained models for BERT

google-research

Google Research

Jupyter Notebook

tuning_playbook

A playbook for systematically maximizing the performance of deep learning models.

vision_transformer

Jupyter Notebook

text-to-text-transfer-transformer

Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"

arxiv-latex-cleaner

arXiv LaTeX Cleaner: Easily clean the LaTeX code of your paper to submit to arXiv

simclr

SimCLRv2 - Big Self-Supervised Models are Strong Semi-Supervised Learners

Jupyter Notebook

multinerf

A Code Release for Mip-NeRF 360, Ref-NeRF, and RawNeRF

timesfm

TimesFM (Time Series Foundation Model) is a pretrained time-series foundation model developed by Google Research for time-series forecasting.

scenic

Scenic: A Jax Library for Computer Vision Research and Beyond

football

Check out the new game server:

albert

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

frame-interpolation

FILM: Frame Interpolation for Large Motion, In ECCV 2022.

t5x

electra

ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

kubric

A data generation pipeline for creating semi-realistic synthetic multi-object videos with rich annotations such as instance segmentation masks, depth maps, and optical flow.

Jupyter Notebook

big_vision

Official codebase used to develop Vision Transformer, SigLIP, MLP-Mixer, LiT and more.

Jupyter Notebook

uda

Unsupervised Data Augmentation (UDA)

language

Shared repository for open-sourced projects from the Google AI Language team.

pegasus

dex-lang

Research language for array processing in the Haskell/ML family

torchsde

Differentiable SDE solvers with GPU support and efficient sensitivity analysis.

parti

big_transfer

Official repository for the "Big Transfer (BiT): General Visual Representation Learning" paper.

robotics_transformer

disentanglement_lib

disentanglement_lib is an open-source library for research on learning disentangled representations.

multilingual-t5

circuit_training

tapas

End-to-end neural table-text understanding models.

planet

Learning Latent Dynamics for Planning from Pixels

mixmatch

deduplicate-text-datasets

fixmatch

A simple method to perform semi-supervised learning with limited data.

morph-net

Fast & Simple Resource-Constrained Learning of Deep Network Structure

maxim

[CVPR 2022 Oral] Official repository for "MAXIM: Multi-Axis MLP for Image Processing". SOTA for denoising, deblurring, deraining, dehazing, and enhancement.

deeplab2

DeepLab2 is a TensorFlow library for deep labeling, aiming to provide a unified and state-of-the-art TensorFlow codebase for dense pixel labeling tasks.

batch-ppo

Efficient Batched Reinforcement Learning in TensorFlow

augmix

AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty

magvit

Official JAX implementation of MAGVIT: Masked Generative Video Transformer

pix2seq

Pix2Seq codebase: multi-tasks with generative modeling (autoregressive and diffusion)

Jupyter Notebook

seed_rl

SEED RL: Scalable and Efficient Deep-RL with Accelerated Central Inference. Implements IMPALA and R2D2 algorithms in TF2 with SEED's architecture.

meta-dataset

A dataset of datasets for learning to learn from few examples

Jupyter Notebook

noisystudent

Code for Noisy Student Training. https://arxiv.org/abs/1911.04252

rliable

[NeurIPS'21 Outstanding Paper] Library for reliable evaluation on RL and ML benchmarks, even with only a handful of seeds.

Jupyter Notebook

recsim

A Configurable Recommender Systems Simulation Platform

jax3d

long-range-arena

Long Range Arena for Benchmarking Efficient Transformers

lottery-ticket-hypothesis

A reimplementation of "The Lottery Ticket Hypothesis" (Frankle and Carbin) on MNIST.

federated

A collection of Google research projects related to Federated Learning and Federated Analytics.

bleurt

BLEURT is a metric for Natural Language Generation based on transfer learning.

prompt-tuning

Original Implementation of Prompt Tuning from Lester, et al, 2021

nasbench

NASBench: A Neural Architecture Search Dataset and Benchmark

neuralgcm

Hybrid ML + physics model of the Earth's atmosphere

xtreme

XTREME is a benchmark for the evaluation of the cross-lingual generalization ability of pre-trained multilingual models that covers 40 typologically diverse languages and includes nine tasks.

lasertagger

sound-separation

pix2struct

vmoe

Jupyter Notebook

dreamer

Dream to Control: Learning Behaviors by Latent Imagination

robopianist

[CoRL '23] Dexterous piano playing with deep reinforcement learning.

omniglue

Code release for CVPR'24 submission 'OmniGlue'

fast-soft-sort

Fast Differentiable Sorting and Ranking

ravens

Train robotic agents to learn pick and place with deep learning for vision-based manipulation in PyBullet. Transporter Nets, CoRL 2020.

sam

batch_rl

Offline Reinforcement Learning (aka Batch Reinforcement Learning) on Atari 2600 games

bigbird

Transformers for Longer Sequences

tensor2robot

Distributed machine learning infrastructure for large-scale robotics research

byt5

adapter-bert

mint

Multi-modal Content Creation Model Training Infrastructure including the FACT model (AI Choreographer) implementation.

leaf-audio

LEAF is a learnable alternative to audio features such as mel-filterbanks, that can be initialized as an approximation of mel-filterbanks, and then be trained for the task at hand, while using a very small number of parameters.

robustness_metrics

Jupyter Notebook

maxvit

[ECCV 2022] Official repository for "MaxViT: Multi-Axis Vision Transformer". SOTA foundation models for classification, detection, segmentation, image quality, and generative modeling...

Jupyter Notebook

receptive_field

Compute receptive fields of your favorite convnets

maskgit

Official Jax Implementation of MaskGIT

Jupyter Notebook

weatherbench2

A benchmark for the next generation of data-driven global weather models.

l2p

Learning to Prompt (L2P) for Continual Learning @ CVPR22 and DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning @ ECCV22

distilling-step-by-step

ssl_detection

Semi-supervised learning for object detection

nerf-from-image

Shape, Pose, and Appearance from a Single Image via Bootstrapped Radiance Field Inversion

computation-thru-dynamics

Understanding computation in artificial and biological recurrent networks through the lens of dynamical systems.

Jupyter Notebook

tf-slim

realworldrl_suite

Real-World RL Benchmark Suite

python-graphs

A static analysis library for computing graph representations of Python programs suitable for use with graph neural networks.

rigl

End-to-end training of sparse deep neural networks with little-to-no performance loss.

task_adaptation

self-organising-systems

Jupyter Notebook

ibc

Official implementation of Implicit Behavioral Cloning, as described in our CoRL 2021 paper, see more at https://implicitbc.github.io/

tensorflow_constrained_optimization

syn-rep-learn

Learning from synthetic data - code and models

arco-era5

Recipes for reproducing Analysis-Ready & Cloud Optimized (ARCO) ERA5 datasets.

vdm

Jupyter Notebook

rlds

Jupyter Notebook

exoplanet-ml

Machine learning models and utilities for exoplanet science.

retvec

RETVec is an efficient, multilingual, and adversarially-robust text vectorizer.

Jupyter Notebook

sparf

This is the official code release for SPARF: Neural Radiance Fields from Sparse and Noisy Poses [CVPR 2023-Highlight]

tensorflow-coder

lm-extraction-benchmark

language-table

Suite of human-collected datasets and a multi-task continuous control benchmark for open vocabulary visuolinguomotor learning.

Jupyter Notebook

falken

Falken provides developers with a service that allows them to train AI that can play their games