• Stars
    star
    353
  • Rank 117,484 (Top 3 %)
  • Language
    Python
  • Created over 1 year ago
  • Updated about 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Create masks out of depthmaps in img2img

depthmap2mask

Made as a script for the AUTOMATIC1111/stable-diffusion-webui repository.

💥 Installation 💥

Copy the url of that repository into the extension tab :

image

OR copy that repository in your extension folder :

image

You might need to restart the whole UI. Maybe twice.

if you are on colab you can also add this line in a code block before starting the UI/after the installation cells :

!git clone https://github.com/Extraltodeus/depthmap2mask.git /content/stable-diffusion-webui/extensions/depthmap2mask

The look

image

What does this extension do?

It creates masks for img2img based on a depth estimation made by MiDaS.

smallerone5050

Where to find it after installing it?

Go to your img2img tab then select it from the custom scripts list at the bottom.

Explanations of the different UI elements

  • Contrasts cut level

image

This slider is purely optional. The depthmap is in levels of gray. Each pixel has a value in between 0 and 255 depending if they are black (0) or white (255). That threshold slider will cut to black every pixel below the selected value and scale from black to white what is above its value.

Or in a more human language, it will give more depth to your depthmaps while removing a lot of information.

Example before/after with the slider's value around 220 and using the MiDaS-Large model:

00073--1 0- sampler -85-8 1-ac07d41f-2022112517485300074--1 0- sampler -85-8 1-ac07d41f-20221125174934

Using the MiDaS small model will give you similar if not more interesting results.

smallerone5050

So that's more of an extra-extra option or a way to make sure that your backgrounds are untouched by using a low value (like 50).

  • Match input size/Net width/Net height

image

Match input size (On by default) will make the depth analysis at the same size as the original image. Better not to touch it unless you are having performance issues.

The sliders below will be the resolution of the analysis if Match input size is turned off.

You can also just use these functionalities to test out different results.

  • Misc options

image

  • Override options :

    These two options simply overrides the inpainting Masked content method and mask blur. I added these because using "original" for Masked content and Mask Blur at 0 just works better. This saves you the clics needed to switch to the intpaint tab/reupload the image to that tab and select the right options.

  • MiDaS models :

    I'll let you try what suits your needs the most.

  • Turn the depthmap into absolute black/white

image

This option will cut out the background of an image into pure black and make the foreground pure white. Like a clean cut.

Alpha Cropping

You can also save a version of the input image which has had all the masked content replaced with transparent pixels. This is useful when extracting the subject from the background, so that it can be used in designs.

Image

Simply check the "Save alpha mask" option before generating.

Tips

  • Avoid using Euler a or you might get really bad results. Usually DDIM works best.

Credits/Citation

Thanks to thygate for letting me blatantly copy-paste some of his functions for the depth analysis integration in the webui.

This repository runs with MiDaS.

@ARTICLE {Ranftl2022,
    author  = "Ren\'{e} Ranftl and Katrin Lasinger and David Hafner and Konrad Schindler and Vladlen Koltun",
    title   = "Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer",
    journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence",
    year    = "2022",
    volume  = "44",
    number  = "3"
}
@article{Ranftl2021,
	author    = {Ren\'{e} Ranftl and Alexey Bochkovskiy and Vladlen Koltun},
	title     = {Vision Transformers for Dense Prediction},
	journal   = {ICCV},
	year      = {2021},
}

Bug reporting

  • Please check if similar issues exist before creating a new one.
  • Make sure to do a "git pull" from your webui folder in order to have your webui up to date
  • Provide as many details as possible when creating a new issue.

Examples using different MiDaS models and denoising strength

00056-589874964- sampler -32-7-ac07d41f-2022112517401700064-1584461722- sampler -32-7-ac07d41f-2022112517432800100-717650490- sampler -84-8 1-ac07d41f-20221125175700

I forgot my settings but in the end it's all pretty easy to guess what you need.

More Repositories

1

multi-subject-render

Generate multiple complex subjects all at once!
Python
364
star
2

ComfyUI-AutomaticCFG

If your image was a pizza and the CFG the temperature of your oven: this is a thermostat that ensures it is always cooked like you want. Also adds a 30% speed increase. For ComfyUI / StableDiffusion
Python
189
star
3

test_my_prompt

This script is to test your prompts with the AUTOMATIC1111 webui
Python
173
star
4

Vector_Sculptor_ComfyUI

Gather similar vectors within the CLIP weights and use them to redirect the original weights
Python
69
star
5

advanced-loopback-for-sd-webui

Python
68
star
6

sigmas_tools_and_the_golden_scheduler

A few nodes to mix sigmas and a custom scheduler that uses phi
Python
38
star
7

LoadLoraWithTags

Save/Load trigger words for loras from a json and auto fetch them on civitai if they are missing. Optional prompt input to auto append them (togglable). Actual alphabetical order and print trigger words to terminal. Also bypass toggle to disable without aiming the sliders at 0.
Python
30
star
8

reddit-video-downloader

A simple python app with a GUI to download reddit videos
Python
27
star
9

3D_wireframe_engine

A 3D wireframe engine written in python by chat GPT
Python
23
star
10

noise_latent_perlinpinpin

This allows to create latent spaces filled with perlin-based noise that can actually be used by the samplers.
Python
18
star
11

Conditioning-token-experiments-for-ComfyUI

A few experimental nodes about the conditioning and the next closest tokens. For ComfyUI.
Python
13
star
12

ChatGPT_prompt_machine

A simple script that turns chatGPT into a Prompt Generator for Text-to-Image Model.
Python
8
star
13

CustomComfyUINodes

my custom nodes for the ComfyUI. This is more or less a temp/sharing repository at least for now.
Python
6
star
14

CryptHue

A script to modifiy a philipps hue light color depending on the price variation of a crypto currency
Python
4
star
15

temp

temporary repo for random things
Python
3
star
16

Game-of-life

another remake of the game of life in python. Uses multi threading and has a few bonus features.
Python
3
star
17

VolumeAssistant

A simple program with a GUI to have a smooth audio output
C#
3
star
18

3DGravity-simulator

A simple unity project that I've made for fun. Not a flex :)
C#
3
star
19

PyAmbiHue

ambiant light for philipps hue
Python
3
star
20

koreader-autorotate

A simple plugin for Koreader to automatically rotate the page to landscape if it's a double page (only tested with CBZ files on a pocketbook)
Lua
3
star
21

Cutwallpaper

A simple script to cut wallpapers in 3 or 4 with a simple right click
Shell
2
star
22

RosaceKeyboard

This is a radial keyboard for controllers (tested with a xbox360 controller)
Python
2
star
23

problem-solver

problem_solver
Python
1
star
24

small_decipher_for_arg

Python
1
star
25

Pytris

A tetris I made some time ago. It has all the functionnalities of a modern Tetris :)
Python
1
star