Prompt Fusion
Prompt Fusion is an auto1111 webui extension that adds more possibilities to the native prompt syntax. Among other additions, it allows to interpolate between the embeddings of different prompts, continuously:
# linear prompt interpolation
[night light:magical forest: 5, 15]
# catmull-rom curve prompt interpolation
[night light:magical forest:norvegian territory: 5, 15, 25:catmull]
# linear attention interpolation
(fire extinguisher: 1.0, 2.0)
# prompt-editing-aware attention interpolation
[(fire extinguisher: 1.0, 2.0)::5]
# define functions and variables to simplify repeating patterns and use a consistent structure
$prompt($style, $quality, $character, $background) = (
A detailed picture in the style of $style,
$quality,
$character lying back,
$background in the background
:1)
The prompt interpolation feature is similar to Prompt Travel, which allows to create videos of images generated by navigating the latent space iteratively. Unlike Prompt Travel however, instead of generating multiple images, Prompt Fusion allows you to travel during the sampling process of a single image. Also, instead of interpolating the latent space, it uses the embedding space to determine intermediate embeddings.
Prompt interpolation is also similar to Prompt Blending. The main difference is that this extension calculates a new embedding for every step, as opposed to calculating it once and using that same one embedding for all the steps.
The attention interpolation feature is similar to Shift Attention, which allows to generate multiple images with slight variations in the attention given to certain parts of the prompt. Unlike Shift Attention, instead of generating multiple images, Prompt Fusion allows to shift the attention of certain parts of a prompt during the sampling process of a single image.
Examples
1. Influencing the beginning of the sampling process
Interpolate linearly (by default) from lion (step 0) to bird (step 8) to girl (step 11), and stay at girl for the rest of the sampling steps:
[lion:bird:girl: , 7, 10]
2. Influencing the middle of the sampling process
Interpolate using a bezier curve from fireball monster (step 0) to dragon monster (step 12, because 30 steps * 0.4 = step 12), while using seawater monster as an intermediate control point to steer the curve away during interpolation and to get creative results:
[fireball:seawater:dragon: , .1, .4:bezier] monster
Features
- Prompt interpolation using a curve function
- Attention interpolation aware of contextual prompt editing
- Complete backwards compatibility with prompt syntax from https://github.com/AUTOMATIC1111/stable-diffusion-webui
- Prompt variables and functions
Webui supported releases
The following webui releases are officially supported:
v1.0.0-premaster(there may be a slight lag for issues arising during quick a1111 webui updates)
Installation
- Visit the Extensions tab of Automatic's WebUI.
- Visit the Available subtab.
- Look for Prompt Fusion.
- Press the Install button.
- Wait for the webui to finish downloading the extension.
- Visit the Installed subtab.
- click on Apply and restart UI.
Alternatively, instead of steps 6 and 7, you can restart the webui completely.
Usage
- Check the wiki pages for the extension documentation.