LooseControl: Lifting ControlNet for Generalized Depth Conditioning
This is the official repository for our paper:
LooseControl: Lifting ControlNet for Generalized Depth Conditioning
Shariq Farooq Bhat, Niloy J. Mitra, Peter Wonka
[Project Page] [Paper] [Demo 🤗] [Weights (3D Box Control)]
Usage
git clone https://github.com/shariqfarooq123/LooseControl && cd LooseControlStart the UI:
gradio app.pyor use via python API:
from loosecontrol import LooseControlNet
lcn = LooseControlNet("shariqfarooq/loose-control-3dbox")
boxy_depth = ...
prompt = "A photo of a snowman in a desert"
negative_prompt = "blurry, text, caption, lowquality,lowresolution, low res, grainy, ugly"
gen_image_1 = lcn(prompt, negative_prompt=negative_prompt, control_image=boxy_depth)Style preserving edits:
# Fix the 'style' and edit
# Edit 'boxy_depth' -> 'boxy_depth_edited'
lcn.set_cf_attention()
gen_image_edited = lcn.edit(boxy_depth, boxy_depth_edited, prompt, negative_prompt=negative_prompt)Credits
The Cross Frame attention is adapted from Text2Video-Zero
Citation
@misc{bhat2023loosecontrol,
title={LooseControl: Lifting ControlNet for Generalized Depth Conditioning},
author={Shariq Farooq Bhat and Niloy J. Mitra and Peter Wonka},
year={2023},
eprint={2312.03079},
archivePrefix={arXiv},
primaryClass={cs.CV}
}