Laion dreams

Overarching goal: Enable the open source community to openly build datasets, papers, models and tools in order to let AGI benefit humankind even faster.

Intro

Laion was initiated with the Laion5B project that successfully produced a 5B (image, text) pairs dataset by processing commoncrawl and filtering with clip. That method proved that it’s cheap to collect large scale dataset from the web using models like clip that give the similarity between items from 2 modalities.

Many models have been trained on laion400m proving the value of this method, with in particular openclip that reproduced the same results that the initial openai clip.

Let’s reproduce that method to more modalities!

Overall rationale

These projects and directions are projects that we would like to promote and help. We do not claim ownership as an organization on these projects. The people that build the projects own these projects.

Directions

Methods

• Open source: releasing everything openly
  • Code: on github with an open license
  • Model: freely distributed models
  • Dataset: freely distributed datasets
• Open development: development is done in public on github and discord, everyone is encouraged to participate, whatever their nationality, age and diploma

Axis of work:

• Open tools
  • Dataset collection
  • Dataset preparation
  • Distributed inference
  • Distributed training
  • Evaluation
• Datasets
  • Open distribution
  • Papers
• Models
  • Open training
  • Open distribution

Scientific domains

• All modalities dataset building
  • Text image
  • Text audio
  • Text video
  • Text 3d
• Contrastive and generative
  • Contrastive
    • Text image
    • Text audio
    • Text video
  • Generative
    • Text to image
    • Image to text

Projects

These projects are collaborations between many people. If you want to know who, check the links and ask in discord. We are open to new collaborators!

Dataset

Model

Tools

Papers