huggingface/awesome-papers

Stars
1,996
Rank 23,210 (Top 0.5 %)
Language
Created over 4 years ago
Updated about 4 years ago

huggingface/awesome-papers

huggingface

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Papers & presentation materials from Hugging Face's internal science day

Awesome NLP Paper Discussions

The Hugging Face team believes that we can reach our goals in NLP by building powerful open source tools and by conducting impactful research. Our team has begun holding regular internal discussions about awesome papers and research areas in NLP. In the spirit of open science, we've decided to share these discussion materials with the community.

Note: These science day discussions are held offline with no physical presentation or discussion to provide. However, some presentation materials do include limited comments from our team or summaries of internal discussions.

See planned future discussions below.

August 12, 2020

Paper: Pre-training via Paraphrasing
Authors: Mike Lewis, Marjan Ghazvininejad, Gargi Ghosh, Armen Aghajanyan, Sida Wang, Luke Zettlemoyer
Presenter: Sam Shleifer
Presentation: Forum Summary
Community Discussion

June 23, 2020

Paper: Weight Poisoning Attacks on Pre-trained Models
Authors: Keita Kurita, Paul Michel, Graham Neubig
Presenter: Joe Davison
Presentation: Colab notebook/post
Community Discussion

June 18, 2020

Paper: Linformer: Self-Attention with Linear Complexity
Authors: Sinong Wang, Belinda Li, Madian Khabsa, Han Fang, Hao Ma
Presenter: Teven Le Scao
Presentation: Tutorial Blog Post
Community Discussion

June 9, 2020

Paper: Evaluating NLP Models via Contrast Sets
Authors: Matt Gardner, Yoav Artzi, Victoria Basmova, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hanna Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A. Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, Ally Zhang, Ben Zhou
Presenter: Victor Sanh
Presentation: Slides

May 18, 2020

Paper: Movement Pruning: Adaptive Sparsity by Fine-Tuning
Authors: Victor Sanh, Thomas Wolf, Alexander M. Rush
Presenter: Victor Sanh
Presentation: Slideshare

May 5, 2020

Paper: Von Mises-Fisher Loss for Training Sequence to Sequence Models with Continuous Outputs
Authors: Sachin Kumar, Yulia Tsvetkov
Presenter: Victor Sanh
Presentation: Colab notebook

April 22, 2020

Topic: Transfer Learning in Natural Language Processing (NLP): Open questions, current trends, limits, and future directions
Presenter: Thomas Wolf
Presentation: Video

April 7, 2020

Topic: Overview of recent work on: Indexing and Retrieval for Open Domain Question Answering
Presenter: Yacine Jernite
Presentation: Slides

March 24, 2020

Paper: Scaling Laws for Neural Language Models
Authors: Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, Dario Amodei
Presenter: Teven Le Scao
Presentation: Google doc paper tutorial

March 17, 2020

Paper: Representation Learning with Contrastive Predictive Coding
Authors: Aaron van den Oord, Yazhe Li, Oriol Vinyals
Presenter Patrick von Platen
Presentation: Slides

March 10, 2020

Paper: Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
Authors: R. Thomas McCoy, Ellie Pavlick, Tal Linzen
Presenter: Victor Sanh
Presentation: Slides

March 3, 2020

Paper: REALM: Retrieval-Augmented Language Model Pre-Training
Authors: Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, Ming-Wei Chang
Presenter: Joe Davison
Presentation: Write-up

February 25, 2020

Paper: Adaptively Sparse Transformers
Authors: Gonçalo M. Correia, Vlad Niculae, André F.T. Martins
Presenter: Sasha Rush
Presentation: Colab notebook

Planned Discussions

No planned discussions for the moment, check back soon.

transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

pytorch-image-models

PyTorch image models, scripts, pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNet-V3/V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more

diffusers

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX.

datasets

🤗 The largest hub of ready-to-use datasets for ML models with fast, easy-to-use and efficient data manipulation tools

peft

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

candle

Minimalist ML framework for Rust

trl

Train transformer language models with reinforcement learning.

text-generation-inference

Large Language Model Text Generation Inference

tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

accelerate

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

chat-ui

Open source codebase powering the HuggingChat app

lerobot

🤗 LeRobot: Making AI for Robotics more accessible with end-to-end learning

alignment-handbook

Robust recipes to align language models with human and AI preferences

parler-tts

Inference and training library for high-quality TTS models.

autotrain-advanced

🤗 AutoTrain Advanced

deep-rl-class

This repo contains the syllabus of the Hugging Face Deep Reinforcement Learning Course.

diffusion-models-class

Materials for the Hugging Face Diffusion Models Course

Jupyter Notebook

notebooks

Notebooks using the Hugging Face libraries 🤗

Jupyter Notebook

distil-whisper

Distilled variant of Whisper for speech recognition. 6x faster, 50% smaller, within 1% word error rate.

neuralcoref

✨Fast Coreference Resolution in spaCy with Neural Networks

safetensors

Simple, safe way to store and distribute tensors

text-embeddings-inference

A blazing fast inference solution for text embeddings models

knockknock

🚪✊Knock Knock: Get notified when your training ends with only two additional lines of code

speech-to-speech

Speech To Speech: an effort for an open-sourced and modular GPT4-o

swift-coreml-diffusers

Swift app demonstrating Core ML Stable Diffusion

optimum

🚀 Accelerate training and inference of 🤗 Transformers and 🤗 Diffusers with easy to use hardware optimization tools

blog

Public repo for HF blog posts

Jupyter Notebook

setfit

Efficient few-shot learning with Sentence Transformers

Jupyter Notebook

course

The Hugging Face course on Transformers

datatrove

Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.

evaluate

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.

cookbook

Open-source AI cookbook

Jupyter Notebook

transfer-learning-conv-ai

🦄 State-of-the-Art Conversational AI with Transfer Learning

swift-coreml-transformers

Swift Core ML 3 implementations of GPT-2, DistilGPT-2, BERT, and DistilBERT for Question answering. Other Transformers coming soon!

pytorch-openai-transformer-lm

🐥A PyTorch implementation of OpenAI's finetuned transformer language model with a script to import the weights pre-trained by OpenAI

huggingface.js

Utilities to use the Hugging Face Hub API

Mongoku

🔥The Web-scale GUI for MongoDB

huggingface_hub

All the open source things related to the Hugging Face Hub.

gsplat.js

JavaScript Gaussian Splatting library.

llm-vscode

LLM powered development for VSCode

hmtl

🌊HMTL: Hierarchical Multi-Task Learning - A State-of-the-Art neural network model for several NLP tasks based on PyTorch and AllenNLP

nanotron

Minimalistic large language model 3D-parallelism training

pytorch-pretrained-BigGAN

🦋A PyTorch implementation of BigGAN with pretrained weights and conversion scripts.

optimum-nvidia

torchMoji

😇A pyTorch implementation of the DeepMoji model: state-of-the-art deep learning model for analyzing sentiment, emotion, sarcasm etc

awesome-huggingface

🤗 A list of wonderful open-source projects & applications integrated with Hugging Face libraries.

optimum-quanto

A pytorch quantization backend for optimum

llm.nvim

LLM powered development for Neovim

naacl_transfer_learning_tutorial

Repository of code for the tutorial on Transfer Learning in NLP held at NAACL 2019 in Minneapolis, MN, USA

dataset-viewer

Backend that powers the dataset viewer on Hugging Face dataset pages through a public API.

swift-transformers

Swift Package to implement a transformers-like API in Swift

exporters

Export Hugging Face models to Core ML and TensorFlow Lite

llm-ls

LSP server leveraging LLMs for code completion (and more?)

ratchet

A cross-platform browser ML framework.

transformers-bloom-inference

Fast Inference Solutions for BLOOM

lighteval

LightEval is a lightweight LLM evaluation suite that Hugging Face has been using internally with the recently released LLM data processing library datatrove and LLM training library nanotron.

pytorch_block_sparse

Fast Block Sparse Matrices for Pytorch

node-question-answering

Fast and production-ready question answering in Node.js

large_language_model_training_playbook

An open collection of implementation tips, tricks and resources for training large language models

swift-chat

Mac app to demonstrate swift-transformers

llm_training_handbook

An open collection of methodologies to help with successful training of large language models.

text-clustering

Easily embed, cluster and semantically label text datasets

cosmopedia

optimum-intel

🤗 Optimum Intel: Accelerate inference with Intel optimization tools

Jupyter Notebook

controlnet_aux

community-events

Place where folks can contribute to 🤗 community events

Jupyter Notebook

tflite-android-transformers

DistilBERT / GPT-2 for on-device inference thanks to TensorFlow Lite with Android demo apps

nn_pruning

Prune a model while finetuning or training.

Jupyter Notebook

speechbox

100-times-faster-nlp

🚀100 Times Faster Natural Language Processing in Python - iPython notebook

education-toolkit

Educational materials for universities

Jupyter Notebook

transformers.js-examples

A collection of 🤗 Transformers.js demos and example applications

open-muse

Open reproduction of MUSE for fast text2image generation.

local-gemma

Gemma 2 optimized for your local machine.

unity-api

audio-transformers-course

The Hugging Face Course on Transformers for Audio

datablations

Scaling Data-Constrained Language Models

Jupyter Notebook

hf_transfer

dataspeech

huggingface-llama-recipes

Jupyter Notebook

optimum-benchmark

🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Optimum's hardware optimizations & quantization schemes.

diarizers

hub-docs

Docs of the Hugging Face Hub

llm-swarm

Manage scalable open LLM inference endpoints in Slurm clusters

sam2-studio

optimum-neuron

Easy, fast and very cheap training and inference on AWS Trainium and Inferentia chips.

Jupyter Notebook

data-is-better-together

Let's build better datasets, together!

Jupyter Notebook

instruction-tuned-sd

Code for instruction-tuning Stable Diffusion.

simulate

🎢 Creating and sharing simulation environments for embodied and synthetic data research

OBELICS

Code used for the creation of OBELICS, an open, massive and curated collection of interleaved image-text web documents, containing 141M documents, 115B text tokens and 353M images.

diffusion-fast

Faster generation with text-to-image diffusion models.

olm-datasets

Pipeline for pulling and processing online language model pretraining data from the web

api-inference-community

jat

General multi-task deep RL Agent

workshops

Materials for workshops on the Hugging Face ecosystem

Jupyter Notebook

coreml-examples

Swift Core ML Examples

Jupyter Notebook

optimum-habana

Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)

chug

Minimal sharded dataset loaders, decoders, and utils for multi-modal document, image, and text datasets.

sharp-transformers

A Unity plugin for using Transformers models in Unity.

hf-hub

Rust client for the huggingface hub aiming for minimal subset of features over `huggingface-hub` python package