• Stars
    star
    1,638
  • Rank 27,464 (Top 0.6 %)
  • Language
    Jupyter Notebook
  • License
    Apache License 2.0
  • Created over 2 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Generate images from texts. In Russian

ruDALL-E

Generate images from texts

Apache license Downloads Coverage Status pipeline pre-commit.ci status

pip install rudalle==1.1.3

🤗 HF Models:

ruDALL-E Malevich (XL)
ruDALL-E Emojich (XL) (readme here)
ruDALL-E Surrealist (XL)
ruDALL-E Kandinsky (XXL) (soon)

Minimal Example:

Open In Colab Kaggle Hugging Face Spaces

Example usage ruDALL-E Malevich (XL) with 3.5GB vRAM! Open In Colab

Finetuning example Open In Colab

generation by ruDALLE:

import ruclip
from rudalle.pipelines import generate_images, show, super_resolution, cherry_pick_by_ruclip
from rudalle import get_rudalle_model, get_tokenizer, get_vae, get_realesrgan
from rudalle.utils import seed_everything

# prepare models:
device = 'cuda'
dalle = get_rudalle_model('Malevich', pretrained=True, fp16=True, device=device)
tokenizer = get_tokenizer()
vae = get_vae(dwt=True).to(device)

# pipeline utils:
realesrgan = get_realesrgan('x2', device=device)
clip, processor = ruclip.load('ruclip-vit-base-patch32-384', device=device)
clip_predictor = ruclip.Predictor(clip, processor, device, bs=8)
text = 'радуга на фоне ночного города'

seed_everything(42)
pil_images = []
scores = []
for top_k, top_p, images_num in [
    (2048, 0.995, 24),
]:
    _pil_images, _scores = generate_images(text, tokenizer, dalle, vae, top_k=top_k, images_num=images_num, bs=8, top_p=top_p)
    pil_images += _pil_images
    scores += _scores

show(pil_images, 6)

auto cherry-pick by ruCLIP:

top_images, clip_scores = cherry_pick_by_ruclip(pil_images, text, clip_predictor, count=6)
show(top_images, 3)

super resolution:

sr_images = super_resolution(top_images, realesrgan)
show(sr_images, 3)

text, seed = 'красивая тян из аниме', 6955

Image Prompt

see jupyters/ruDALLE-image-prompts-A100.ipynb

text, seed = 'Храм Василия Блаженного', 42
skyes = [red_sky, sunny_sky, cloudy_sky, night_sky]

VideoDALL-E | ruCogVideo by @cene555

Video generation example Open In Colab Finetuning example Open In Colab

Aspect ratio images -->NEW<--

Kandinsky 12B

Request access: Here

роботы акварелью в стиле ван гога

FID = 15.4 (COCO Valid)

🚀 Contributors 🚀

Supported by

Social Media

More Repositories

1

Kandinsky-2

Kandinsky 2 — multilingual text2image latent diffusion model
Jupyter Notebook
2,699
star
2

ru-gpts

Russian GPT3 models.
Python
2,045
star
3

ghost

A new one shot face swap approach for image and video domains
Python
1,030
star
4

ner-bert

BERT-NER (nert-bert) with google bert https://github.com/google-research.
Jupyter Notebook
403
star
5

ru-dolph

RUDOLPH: One Hyper-Tasking Transformer can be creative as DALL-E and GPT-3 and smart as CLIP
Jupyter Notebook
242
star
6

Real-ESRGAN

PyTorch implementation of Real-ESRGAN model
Python
201
star
7

mgpt

Multilingual Generative Pretrained Model
Jupyter Notebook
194
star
8

KandinskyVideo

KandinskyVideo — multilingual end-to-end text2video latent diffusion model
Python
140
star
9

ru-clip

CLIP implementation for Russian language
Jupyter Notebook
126
star
10

ruGPT3_demos

121
star
11

sage

SAGE: Spelling correction, corruption and evaluation for multiple languages
Jupyter Notebook
101
star
12

deforum-kandinsky

Kandinsky x Deforum — generating short animations
Python
100
star
13

digital_peter_aij2020

Materials of the AI Journey 2020 competition dedicated to the recognition of Peter the Great's manuscripts, https://ai-journey.ru/contest/task01
Jupyter Notebook
66
star
14

music-composer

Python
62
star
15

ru-prompts

Python
54
star
16

fusion_brain_aij2021

Creating multimodal multitask models
Jupyter Notebook
47
star
17

model-zoo

NLP model zoo for Russian
44
star
18

gigachat

Библиотека для доступа к GigaChat
Python
43
star
19

OCR-model

An easy-to-run OCR model pipeline based on CRNN and CTC loss
Python
42
star
20

augmentex

Augmentex — a library for augmenting texts with errors
Python
40
star
21

StackMix-OCR

Jupyter Notebook
37
star
22

MoVQGAN

MoVQGAN - model for the image encoding and reconstruction
Jupyter Notebook
35
star
23

MERA

MERA (Multimodal Evaluation for Russian-language Architectures) is a new open benchmark for the Russian language for evaluating fundamental models.
Jupyter Notebook
31
star
24

tuned-vq-gan

Jupyter Notebook
28
star
25

ReadingPipeline

Text reading pipeline that combines segmentation and OCR-models.
Python
23
star
26

htr_datasets

Repository containing our datasets for HTR (handwritten text recognition) task.
Jupyter Notebook
23
star
27

fbc3_aij2023

Jupyter Notebook
20
star
28

mineral-recognition

Python
19
star
29

DigiTeller

18
star
30

fbc2_aij2022

FusionBrain Challenge 2.0: creating multimodal multitask model
Python
16
star
31

combined_solution_aij2019

AI Journey 2019: Combined Solution
Python
15
star
32

railway_infrastructure_detection_aij2021

AI Journey Contest 2021: AITrain
Python
13
star
33

no_fire_with_ai_aij2021

AI Journey Contest 2021: NoFireWithAI
Jupyter Notebook
13
star
34

SEGM-model

An easy-to-run semantic segmentation model based on Unet
Python
11
star
35

ControlledNST

An implementation of Neural Style Transfer in PyTorch.
Jupyter Notebook
8
star
36

kandinsky3-diffusers

Python
5
star
37

mchs-wildfire

Соревнование по классификации лесных пожаров
Jupyter Notebook
4
star
38

no_flood_with_ai_aij2020

Материалы соревнования AI Journey 2020, посвященного прогнозированию паводков на реке Амур, https://ai-journey.ru/contest/task02
Jupyter Notebook
4
star
39

paper_persi_chat

PaperPersiChat: Scientific Paper Discussion Chatbot using Transformers and Discourse Flow Management
Jupyter Notebook
1
star
40

Zoom_In_Video_Kandinsky

Framework for creating Zoom in / Zoom out video based on inpainting Kandinsky
Jupyter Notebook
1
star