LAION-AI/Open-Instruction-Generalist

Stars
195
Rank 199,374 (Top 4 %)
Language
Python
License
Apache License 2.0
Created almost 2 years ago
Updated over 1 year ago

LAION-AI/Open-Instruction-Generalist

LAION-AI

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Open Instruction Generalist is an assistant trained on massive synthetic instructions to perform many millions of tasks

Open-Instruction-Generalist Dataset

Open Instruction Generalist (OIG) Dataset is intended to train assistants that are part of the LAION-AI's family of assistants. OIG Assistants will be trained on the OIG dataset, a massive synthetic instructions with the goal of performing many diverse types of tasks.

We will have several versions of the OIG Assistant dataset ranging from a small (less than 1M) high quality synthetic dataset, to a massive synthetic instruction dataset. The research goal of OIG Assistant is to create high performing bots by using simple finetuning instead of RLHF.

We will create ever larger instruction datasets with the goal to generate eventually 1T medium quality tokens of instructions. The receipe for training is to do additional pretrain on some subset of the larger instruction sets, followed by a finetune on OIG-small or some other high quality small dataset.

OIG-small-chip2 (200K) - Done and released. See small_instruction_set sub-directory.
OIG-40M - Done - Done and released. See 40M sub-directory
OIG-100M - In progress, to be released expected April 30, 2023

OIG-Moderation

We have also created a small subset of safety data to tag instructions for moderation. This dataset was created by volunteers and also curated and augmented from public datasets (see https://huggingface.co/datasets/ontocord/OIG-moderation)

OIG_safety_v0.1.jsonl (66K)
OIG_safety_v0.2.jsonl (134K)

Related Projects

Check out LAION's Open Assistant Project. We aim to build a chatbot based on RLHF and human feedback data.
Check out our friends Together.xyz's OpenChatKit. They trained a bot based on OIG!

Models

The community has trained several models based on a subset of the OIG datasets including:

Rallio67/joi2_(20,12,7)B_instruct_alpha
Rallio67/chip2_(20,12,7)B_instruct_alpha
Rallio67/joi_(20,12,7)B_instruct_alpha
Rallio67/chip_(20,12,7)B_instruct_alpha
togethercomputer/GPT-NeoXT-Chat-Base-20B

Safety models

SummerSigh/T5-Base-Rule-Of-Thumb
SummerSigh/Safety-Policy
SummerSigh/BART-Base-Rule-Of-Thumb
shahules786/prosocial-classifier
shahules786/Safetybot-mt5-base
shahules786/Safetybot-T5-base
togethercomputer/GPT-JT-Moderation-6B

Available on huggingface.co.

Open-Assistant

OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

audio-dataset

Audio Dataset for training CLAP and other models

CLIP_benchmark

CLIP-like model evaluation

Jupyter Notebook

dalle2-laion

Pretrained Dalle2 from laion

CLAP

Contrastive Language-Audio Pretraining

natural_voice_assistant

laion-3d

Collect large 3d dataset and build models

phenaki

A phenaki reproduction using pytorch.

aesthetic-predictor

A linear estimator on top of clip to predict the aesthetic quality of pictures

Jupyter Notebook

ldm-finetune

Home of `erlich` and `ongo`. Finetune latent-diffusion/glid-3-xl text2image on your own data.

scaling-laws-openclip

Reproducible scaling laws for contrastive language-image learning (https://arxiv.org/abs/2212.07143)

Jupyter Notebook

CLIP-based-NSFW-Detector

laion-datasets

Description and pointers of laion datasets

laion-dreams

Aim for the moon. If you miss, you may hit a star.

laion.ai

AIW

Alice in Wonderland code base for experiments and raw experiments data

LAION-5B-WatermarkDetection

video-clip

Let's make a video clip

Open-GIA

O-GIA is an umbrella for research, infrastructure and projects ecosystem that should provide open source, reproducible datasets, models, applications & safety tools for Open Generalist Interactive Agents (O-GIA). O-GIA systems will act in collaboration with human or autonomously, supporting various kind of validated decision making and assistance.

General-GPT

Jupyter Notebook

Discord-Scrapers

Implementation of a discord channel scraper to generate datasets.

Text-to-speech

Big-Interleaved-Dataset

Big-Interleaved-Dataset

riverbed

Tools for content datamining and NLP at scale

OCR-ensemble

Jupyter Notebook

Conditional-Pretraining-of-Large-Language-Models

interesting-text-datasets

blade2blade

Adversarial Training and SFT for Bot Safety Models

temporal-embedding-aggregation

Aggregating embeddings over time

deep-image-diffusion-prior

Inverts CLIP text embeds to image embeds and visualizes with deep-image-prior.

Jupyter Notebook

watermark-detection

A repository containing datasets and tools to train a watermark classifier.

medical

This repository will be a summary and outlook on all our open, medical, AI advancements.

Jupyter Notebook

Anh

Anh - LAION's multilingual assistant datasets and models

laion50BU

Un-*** 50 billions multimodality dataset

conditioned-prior

(wip) Use LAION-AI's CLIP "conditoned prior" to generate CLIP image embeds from CLIP text embeds.

LAION-SAFETY

An open toolbox for NSFW & toxicity detection

Jupyter Notebook

opendream

Frontend (and soon also midleware and backend) for a new, opensource image generation platform.

laion5B-paper

Building the laion5B paper

laion-dedup

notebooks

A collection of generative and training notebooks getting mirrored to google colab.

Jupyter Notebook

laionide

This repository contains training code and checkpoitns for finetuning glide.

super-resolution

This is the LAION repository for creating open super-resolution models with the help of LAION-5B subsets.

dataset-spec

Describe the format of image/text datasets

LAION-PEOPLE

This project provides a data set with bounding boxes, body poses, 3D face meshes & captions of people from our LAION-2.2B. Additionally it provides clusters based on the poses and face meshes and pose-related captions based on these cluster assignments.

image-deduplication-testset

project-menu

Projects at LAION

laion-ai.github.io

laion github website

dataset-usage

This repository is a summary of all systems and scientific papers that use LAION datasets.

repository-overview

This repository will give a quick overview of all projects and repositories from LAION.

LionizeR

Experiments with Summarization, Long Context and Retrieval

KAISER

Knowledge Acquisition and Interlinking via Semantic Embeddings and Reasoning

lucidrains-projects

A summary of all lucidrains repositores and links to training / research approaches by LAION or other communities.

Jupyter Notebook

decentralized-learning

A basic setup for decentralized-learning that can be used for training future DALLE/CLIP/CLAP models.

diffusion-prior

DALL-E2 diffusion prior

GIF

General / Global Inference Framework

website

This is the development repository of the LAION-AI website.

safety-pipeline

A collection of safety classifiers and models to process image and texts.

NeoGen

laion5b-subsets

Creating subsets from laion5b via embeddings search

Jupyter Notebook

human_artifacts

A repo containing images for artifact annotation.

public-relations

All media / publicity on LAION and related stuff!

public-domain-images

A collection of public domain images donated for ML training.

math_problems-step-by-step_solutions

Here we provide and collect many functions to generate math problem and step by step solutions for LLM training

language-models

dataset-inference

The new repository for the genral inference pipeline.

introduction-resources

Recommended intro resources

balanced-laion5b

This repository shall help finding a good distribution for huge datasets like LAION-5B for more efficient training.

hand-inference

A model to run hand inference on a cluster.

Jupyter Notebook

BUD-E_V1.0

BUD-E (Buddy) is an open-source voice assistant framework that facilitates seamless interaction with AI models and APIs, enabling the creation and integration of diverse skills for educational and research applications.

laion5b-bias

This repository is a collection of found biases in the LAION-5B dataset.

dataset-tasks

datasets that should be downloaded & converted to our standard training formart.

LAION-AUDIO

This repository contains prompts & best practices to annotate audio clips with a very high degree of details using Audio-Language-Models

AIW_webpage

Alice in Wonderland project and initiative webpage