• Stars
    star
    328
  • Rank 128,352 (Top 3 %)
  • Language
    Python
  • License
    Apache License 2.0
  • Created about 2 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Learn to serve Stable Diffusion models on cloud infrastructure at scale. This Lightning App shows load-balancing, orchestrating, pre-provisioning, dynamic batching, GPU-inference, micro-services working together via the Lightning Apps framework.


Use AI to inspire your art


Muse

Open source, stable-diffusion production server to show how to deploy diffusion models in a real production environment with: load-balancing, gpu-inference, performance-testing, micro-services orchestration and more. All handled easily with the Lightning Apps framework.

The app is live here.

Full tutorial on how to build this app.

image

Model

Muse uses the opensource Stable Diffusion model made available by stability AI. We apply a few fancy tricks to make the inference super fast.

Here's a small snippet showing our model server

    @torch.inference_mode()
    def predict(self, dreams: List[Data], entry_time: int):
        # handle timeout
        if time.time() - entry_time > INFERENCE_REQUEST_TIMEOUT:
            raise TimeoutException()

        # sets up the inference settings
        height = width = IMAGE_SIZE
        num_inference_steps = 50 if dreams[0].high_quality else 25

        prompts = [dream.prompt for dream in dreams]

        # GPU inference
        if torch.cuda.is_available():
            with autocast("cuda"):
                torch.cuda.empty_cache()
                pil_results = self._model(
                    prompts,
                    height=height,
                    width=width,
                    num_inference_steps=num_inference_steps,
                )
            # apply filter
            nsfw_content = self._safety_checker(pil_results)
            for i, nsfw in enumerate(nsfw_content):
                if nsfw:
                    pil_results[i] = Image.open("assets/nsfw-warning.png")
        else:
            time.sleep(3)
            pil_results = [Image.fromarray(np.random.randint(0, 255, (height, width, 3), dtype="uint8"))] * len(prompts)

        # return the model
        results = []
        for image in pil_results:
            buffered = BytesIO()
            image.save(buffered, format="PNG")
            img_str = base64.b64encode(buffered.getvalue()).decode("utf-8")
            # make sure pil_results is a single item array or it'll rewrite image
            results.append({"image": f"data:image/png;base64,{img_str}"})

        return results

Run your own

To run this app locally, follow these steps:

conda create --name muse_app python=3.9 --yes
conda activate muse_app

git clone https://github.com/Lightning-AI/stable-diffusion-deploy.git
cd stable-diffusion-deploy

bash dev_install.sh

## To run the app locally
python -m lightning run app app.py

## To run the app on the cloud to share it with your peers and users
python -m lightning run app app.py --cloud

You can configure Muse to select customize number of workers, batch size or select the stable diffusion version of your choice. Learn more.

About this Lightning App

Muse is a blueprint for building diffusion-based production systems with Lightning AI. This app shows you how to:

  • Host a multi-tenant Frontend & Backend application architecture
  • Full React.js UI
  • Micro-services orchestration
  • Cloud infrastructure pre-provisioning
  • Serves a gpu-powered diffusion model via a REST API
  • Dynamic GPU batching for inference requests
  • Load balancer that autoscales infrastructure with load-changes
  • Load Testing Lightning Component using Locust
  • Environment variables to parametrize execution environment

Architecture Diagram of Muse -

arch diagram

Slackbot

How to integrate Muse to a Slack Workspace

You can integrate this app in your Slack Workspace and send images in Slack channels.

This app uses the Slack Command Bot Component to interact with Slack commands.

You can also check out this video tutorial:

How to create the Slack Command Bot

Step 1: Go to https://api.slack.com and create an app.

Step 2: Copy the following tokens and secrets from the Slack API settings by going to https://api.slack.com/apps. These tokens have to be passed either as argument or environment variable to SlackCommandBot class.

Required Token name and environment variables:
  • Client ID (SLACK_CLIENT_ID)
  • Client Secret (CLIENT_SECRET)
  • Signing Secret (SIGNING_SECRET)
  • Bot User OAuth Token (BOT_TOKEN)
  • App-Level Token (SLACK_TOKEN)

Step 3:

Implement the SlackCommandBot.handle_command(...) method the way you want to interact with the commands. The return value will be shown only to you.

Step 4: (optional)

If you want your slack app to be distributable to public then you need to implement SlackCommandBot.save_new_workspace(...) which should save team_id and its corresponding bot_token into a database.

During the handle_command(...) method you will need to fetch bot_token based on the received team_id.

More Repositories

1

lightning-flash

Your PyTorch AI Factory - Flash enables you to easily configure and run complex AI recipes for over 15 tasks across 7 data domains
Python
1,733
star
2

lightning-bolts

Toolbox of models, callbacks, and datasets for AI/ML researchers.
Python
1,547
star
3

lightning-transformers

Flexible components pairing 🤗 Transformers with ⚡ Pytorch Lightning
Python
591
star
4

Echo

Production-ready audio and video transcription app that can run on your laptop or in the cloud.
TypeScript
67
star
5

lightning-GPT

Train and run GPTs with Lightning
Python
63
star
6

lightning-Covid19

Classification for covid-19 chest X-ray images using Lightning
Python
56
star
7

paper-AAVAE

Python
46
star
8

Research-template

Quickest way to share everything about your research within a single app
Jupyter Notebook
37
star
9

lightning-diffusion_component

Python
25
star
10

DiffusionWithAutoscaler

DiffusionWithAutoscaler
Python
23
star
11

Pose-app

A Lightning app for animal pose estimation.
Python
18
star
12

Training-Studio_app

Lightning HPO & Training Studio App
Python
17
star
13

Research-poster

Quickest way to share everything about your research within a single app
Python
16
star
14

Triton-Server_component

Triton Server Component for lightning.ai
Python
14
star
15

lightning-quick-start

Python
12
star
16

InVideo-search_app

Python
12
star
17

CVPR22-MAE_research-poster

Jupyter Notebook
11
star
18

Flashy_app

Perform Image/Text Classification with ⚡ Flash built using ⚡ AI
Python
10
star
19

open-bio-ml-workshop

Python
9
star
20

Serve_component

Python
9
star
21

HackerNews_app

Jupyter Notebook
8
star
22

lightning-gpt3

Python
7
star
23

dataumbrella22-intro-pytorch

Code for Data Umbrella Talk "Intro to PyTorch and LightningLite
Jupyter Notebook
6
star
24

API-Access-UI_component

TypeScript
6
star
25

Slack-Command-Bot-component

With this component you can create a Slack bot and enable interactivity with the Slash Commands.
Python
5
star
26

Jupyter_component

Jupyter notebook component
Python
5
star
27

Text-Prediction_component

Python
4
star
28

Text-Classification-component

Python
4
star
29

lightning-template-react

TypeScript
4
star
30

AnimeGAN-v2_research-poster

Jupyter Notebook
4
star
31

Redis_component

Redis component for lightning
Python
4
star
32

MarkDown-poster_component

This component lets you make posters from Markdown files.
Python
4
star
33

CodeRunner_app

A sample app which uses Monaco Editor integration, and runs the script on frame from webcam
Python
4
star
34

Finetune-miniLM

Python
4
star
35

Hate-Speech-Detection_app

Python
4
star
36

lightning-Bagua

Lightning Training strategy for Bagua
Python
3
star
37

DeepChecks_app

Deepchecks App with Lightning
Python
3
star
38

lightning-LLMs

Python
3
star
39

Collaborate_app

JavaScript
3
star
40

TLDR-component

Components for training large language models
Python
3
star
41

Telegram-Messenger_component

Python
3
star
42

JupyterLite_component

Jupyter Lite instance runs completely in your browser, powered by Pyodide
Python
3
star
43

FaceNet_research-poster

Jupyter Notebook
3
star
44

Gradio-template

Python
3
star
45

ODSC

tutorial for converting PyTorch to Lightning
Jupyter Notebook
3
star
46

Dalle-mini-poster

🥑 Dalle Mini Poster App | Generate Images from Text Prompt ⚡️
Jupyter Notebook
2
star
47

ICML22-BLIP_research-poster

Python
2
star
48

YOLOv5_research-poster

This app is a research poster demo of YoloV5 Object Detection model by Ultralytics. It showcases a notebook, a blog, and a model demo where you can upload photos to get a bounding box visualized output image.
Jupyter Notebook
2
star
49

video-streaming_component

Python
2
star
50

BigQuery_component

Python
2
star
51

grid-tutorials

Python
2
star
52

ICML22-OFA_research-poster

Jupyter Notebook
2
star
53

Finetune-miniLM-gallery

Python
1
star
54

lightning-aws

Python
1
star
55

Flash-Serve_component

LAI component for FlashServe
Python
1
star
56

AWS-s3_component

Lightning component to interact with s3.
Python
1
star
57

lightning-Galvatron

Python
1
star
58

Sample_component

Python
1
star
59

Youtube_component

Used to interface with youtube
Python
1
star
60

tgr

Python
1
star