GAN Inversion: A Survey

TPAMI, 2022
Weihao Xia · Yulun Zhang · Yujiu Yang · Jing-Hao Xue · Bolei Zhou · Ming-Hsuan Yang

This repo is a collection of resources on GAN inversion, as a supplement for our survey. If you find any work missing or have any suggestions (papers, implementations and other resources), feel free to pull requests. You could manually edit items or use the script to produce them in the markdown format.

@article{xia2022gan,
    author  = {Xia, Weihao and Zhang, Yulun and Yang, Yujiu and Xue, Jing-Hao and Zhou, Bolei and Yang, Ming-Hsuan},
    title   = {GAN Inversion: A Survey},
    journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
    year={2022}
  }

inverted pretrained model

2D GANs

Scaling up GANs for Text-to-Image Synthesis.
Minguk Kang, Jun-Yan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, Taesung Park.
CVPR 2023 (Highlight). [PDF] [Project]

StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis.
Axel Sauer, Tero Karras, Samuli Laine, Andreas Geiger, Timo Aila.
ICML 2023. [Project] [PDF] [Github]

StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets.
Axel Sauer, Katja Schwarz, Andreas Geiger.
SIGGRAPH 2022. [PDF] [Project] [Github]

Self-Distilled StyleGAN: Towards Generation from Internet Photos.
Ron Mokady, Michal Yarom, Omer Tov, Oran Lang, Daniel Cohen-Or, Tali Dekel, Michal Irani, Inbar Mosseri.
SIGGRAPH 2022. [PDF] [Project] [Github]

Ensembling Off-the-shelf Models for GAN Training.
Nupur Kumari, Richard Zhang, Eli Shechtman, Jun-Yan Zhu
CVPR 2022. [PDF] [Project] [Github]

StyleGAN3: Alias-Free Generative Adversarial Networks.
Tero Karras, Miika Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, Timo Aila.
NeurIPS 2021. [PDF] [Project] [Github] [Rosinality]

StyleGAN2-Ada: Training Generative Adversarial Networks with Limited Data.
Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, Timo Aila.
NeurIPS 2020. [PDF] [Github] [Steam StyleGAN2-ADA]

StyleGAN2: Analyzing and Improving the Image Quality of StyleGAN.
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, Timo Aila.
CVPR 2020. [PDF] [PyTorch] [Offical TF] [Unoffical Tensorflow 2.0]

StyleGAN: A Style-Based Generator Architecture for Generative Adversarial Networks.
Tero Karras, Samuli Laine, Timo Aila.
CVPR 2019. [PDF] [Offical TF]

ProGAN: Progressive Growing of GANs for Improved Quality, Stability, and Variation.
Tero Karras, Timo Aila, Samuli Laine, Jaakko Lehtinen.
ICLR 2018. [PDF] [Offical TF]

3D-aware GANs

Please check our 3D-aware image synthesis survey, paper list, and project for more details.

EG3D: Efficient Geometry-aware 3D Generative Adversarial Networks.
Eric R. Chan, Connor Z. Lin, Matthew A. Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas Guibas, Jonathan Tremblay, Sameh Khamis, Tero Karras, Gordon Wetzstein.
CVPR 2022. [PDF] [Project] [Github]

StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis.
Jiatao Gu, Lingjie Liu, Peng Wang, Christian Theobalt.
ICLR 2022. [PDF] [Project]

StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation.
Roy Or-El, Xuan Luo, Mengyi Shan, Eli Shechtman, Jeong Joon Park, Ira Kemelmacher-Shlizerman.
CVPR 2022. [PDF] [Project] [Github]

pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis.
Eric R. Chan, Marco Monteiro, Petr Kellnhofer, Jiajun Wu, Gordon Wetzstein.
CVPR 2021. [PDF] [Project] [Github]

inversion method

The section primarily encompasses general-purpose 2D or 3D inversion techniques, whereas the methods presented in the following section cater to particular applications.

3D GAN inverson

High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimization.
Jiaxin Xie, Hao Ouyang, Jingtan Piao, Chenyang Lei, Qifeng Chen.
CVPR 2023. [PDF] [Project] [Github]

Learning Detailed Radiance Manifolds for High-Fidelity and 3D-Consistent Portrait Synthesis from Monocular Image.
Yu Deng, Baoyuan Wang, Heung-Yeung Shum.
CVPR 2023. [PDF] [Project]

E3DGE: Self-Supervised Geometry-Aware Encoder for Style-based 3D GAN Inversion.
Yushi Lan, Xuyi Meng, Shuai Yang, Chen Change Loy, Bo Dai.
CVPR 2023. [PDF] [Project] [Github]

3D GAN Inversion for Controllable Portrait Image Animation.
Connor Z. Lin, David B. Lindell, Eric R. Chan, Gordon Wetzstein.
ECCV 2022 Workshop on Learn3DG. [PDF] [Project]

3D GAN Inversion with Pose Optimization.
Jaehoon Ko, Kyusun Cho, Daewon Choi, Kwangrok Ryoo, Seungryong Kim.
WACV 2023. [PDF] [Project] [Github]

Pix2NeRF: Unsupervised Conditional π-GAN for Single Image to Neural Radiance Fields Translation.
Shengqu Cai, Anton Obukhov, Dengxin Dai, Luc Van Gool.
CVPR 2022. [PDF]

Make Encoder Great Again in 3D GAN Inversion through Geometry and Occlusion-Aware Encoding.
Ziyang Yuan, Yiming Zhu, Yu Li, Hongyu Liu, Chun Yuan.
arxiv 2023. [PDF] [Project]

TriPlaneNet: An Encoder for EG3D Inversion.
Ananta R. Bhattarai, Matthias Nießner, Artem Sevastopolsky.
arxiv 2023. [PDF] [Project]

LatentSwap3D: Semantic Edits on 3D Image GANs.
Enis Simsar, Alessio Tonioni, Evin Pınar Örnek, Federico Tombari.
arxiv 2022. [PDF]

Learning a 3D-Aware Encoder for Style-based Generative Radiance Field.
Yu-Jhe Li, Tao Xu, Bichen Wu, Ningyuan Zheng, Xiaoliang Dai, Albert Pumarola, Peizhao Zhang, Peter Vajda, Kris M. Kitani.
arxiv 2022. [PDF]

2D GAN inversion

ReGANIE: Rectifying GAN Inversion Errors for Accurate Real Image Editing.
Bingchuan Li, Tianxiang Ma, Peng Zhang, Miao Hua, Wei Liu, Qian He, Zili Yi.
AAAI 2023. [PDF]

Intra-Source Style Augmentation for Improved Domain Generalization (Masked Noise Encoder).
Yumeng Li, Dan Zhang, Margret Keuper, Anna Khoreva.
WACV 2023. [PDF] [Github]

Pivotal Tuning for Latent-based Editing of Real Images.
Daniel Roich, Ron Mokady, Amit H. Bermano, Daniel Cohen-Or.
TOG 2022. [PDF] [Github]

E2Style: Improve the Efficiency and Effectiveness of StyleGAN Inversion.
Tianyi Wei, Dongdong Chen, Wenbo Zhou, Jing Liao, Weiming Zhang, Lu Yuan, Gang Hua, Nenghai Yu.
TIP 2022. [PDF] [Project] [Github]

High-fidelity GAN Inversion with Padding Space.
Qingyan Bai, Yinghao Xu, Jiapeng Zhu, Weihao Xia, Yujiu Yang, Yujun Shen.
ECCV 2022. [PDF] [Project] [Github]

Editing Out-of-Domain GAN Inversion via Differential Activations.
Haorui Song, Yong Du, Tianyi Xiang, Junyu Dong, Jing Qin, Shengfeng He.
ECCV 2022. [PDF] [Github]

IntereStyle: Encoding an Interest Region for Robust StyleGAN Inversion.
Seungjun Moon, GyeongMoon Park.
ECCV 2022. [PDF]

Chunkmogrify: Real image inversion via Segments.
David Futschik, Michal Lukáč, Eli Shechtman, Daniel Sýkora.
ECCV 2022. [PDF] [Github]

A Style-Based GAN Encoder for High Fidelity Reconstruction of Images and Videos.
Xu Yao, Alasdair Newson, Yann Gousseau, Pierre Hellier.
ECCV 2022. [PDF] [Github]

Cycle Encoding of a StyleGAN Encoder for Improved Reconstruction and Editability.
Xudong Mao, Liujuan Cao, Aurele Tohokantche Gnanha, Zhenguo Yang, Qing Li, Rongrong Ji.
ACM MM 2022. [PDF] [Github]

Spatially-Adaptive Multilayer Selection for GAN Inversion and Editing.
Gaurav Parmar, Yijun Li, Jingwan Lu, Richard Zhang, Jun-Yan Zhu, Krishna Kumar Singh.
CVPR 2022. [PDF] [Project] [Github]

Style Transformer for Image Inversion and Editing.
Xueqi Hu, Qiusheng Huang, Zhengyi Shi, Siyuan Li, Changxin Gao, Li Sun, Qingli Li.
CVPR 2022. [PDF] [Github]

High-Fidelity GAN Inversion for Image Attribute Editing.
Tengfei Wang, Yong Zhang, Yanbo Fan, Jue Wang, Qifeng Chen.
CVPR 2022. [PDF] [Project] [Github]

HyperInverter: Improving StyleGAN Inversion via Hypernetwork.
Tan M. Dinh, Anh Tuan Tran, Rang Nguyen, Binh-Son Hua.
CVPR 2022. [PDF] [Project]

HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing.
Yuval Alaluf, Omer Tov, Ron Mokady, Rinon Gal, Amit H. Bermano.
CVPR 2022. [PDF] [Project] [Github]

Overparameterization Improves StyleGAN Inversion.
Yohan Poirier-Ginter, Alexandre Lessard, Ryan Smith, Jean-François Lalonde.
CVPR 2022 Workshop on AI for Content Creation. [PDF] [Github]

StyleAlign: Analysis and Applications of Aligned StyleGAN Models.
Zongze Wu, Yotam Nitzan, Eli Shechtman, Dani Lischinski.
ICLR 2022. [PDF]

Semantic Unfolding of StyleGAN Latent Space.
Mustafa Shukor, Xu Yao, Bharath Bushan Damodaran, Pierre Hellier.
ICIP 2022. [PDF]

What Decreases Editing Capability? Domain-Specific Hybrid Refinement for Improved GAN Inversion.
Pu Cao, Lu Yang, Dongxu Liu, Shan Li, Yao Zhang, Qing Song.
arxiv 2022. [PDF] [Github]

Photo-Realistic Out-of-domain GAN Inversion Via Invertibility Decomposition.
Xin Yang, Xiaogang Xu, Yingcong Chen.
arxiv 2022. [PDF]

WaGI: Wavelet-based GAN Inversion for Preserving High-frequency Image Details.
Seung-Jun Moon, Chaewon Kim, Gyeong-Moon Park.
arxiv 2022. [PDF]

LSAP: Rethinking Inversion Fidelity, Perception and Editability in GAN Latent Space.
Cao Pu, Lu Yang, Dongxv Liu, Zhiwei Liu, Wenguan Wang, Shan Li, Qing Song.
arxiv 2022. [PDF] [Github]

Video2StyleGAN: Encoding Video in Latent Space for Manipulation.
Jiyang Yu, Jingen Liu, Jing Huang, Wei Zhang, Tao Mei.
arxiv 2022. [PDF]

Expanding The Latent Space of StyleGAN for Real Face Editing.
Yin Yu, Ghasedi Kamran, Wu HsiangTao, Yang Jiaolong, Tong Xi, Fu Yun.
arxiv 2022. [PDF]

Expressive Talking Head Video Encoding in StyleGAN2 Latent-Space.
Trevine Oorloff, Yaser Yacoob.
arxiv 2022. [PDF] [Project] [Github] [Data]

StyleGAN of All Trades: Image Manipulation with Only Pretrained StyleGAN.
Min Jin Chong, Hsin-Ying Lee, David Forsyth.
arxiv 2021. [PDF] [Github]

Semantic and Geometric Unfolding of StyleGAN Latent Space.
Mustafa Shukor, Xu Yao, Bharath Bhushan Damodaran, Pierre Hellier.
arxiv 2021. [PDF]

Transforming The Latent Space of StyleGAN for Real Face Editing.
Heyi Li, Jinlong Liu, Xinyu Zhang, Yunzhi Bai, Huayan Wang, Klaus Mueller.
arxiv 2021. [PDF] [Github]

Improved StyleGAN Embedding: Where Are The Good Latents?.
Peihao Zhu, Rameen Abdal, Yipeng Qin, John Femiani, Peter Wonka.
arxiv 2020. [PDF] [Github]

Improving Inversion and Generation Diversity in StyleGAN Using A Gaussianized Latent Space.
Jonas Wulff, Antonio Torralba.
arxiv 2020. [PDF]

GAN-Control: Explicitly Controllable GANs.
Alon Shoshan, Nadav Bhonker, Igor Kviatkovsky, Gerard Medioni.
ICCV 2021. [PDF]

From Continuity to Editability: Inverting GANs with Consecutive Images.
Yangyang Xu, Yong Du, Wenpeng Xiao, Xuemiao Xu and Shengfeng He.
ICCV 2021. [PDF] [Github]

Explaining in Style: Training a GAN to explain a classifier in StyleSpace.
Oran Lang, Yossi Gandelsman, Michal Yarom, Yoav Wald, Gal Elidan, Avinatan Hassidim, William T. Freeman, Phillip Isola, Amir Globerson, Michal Irani, Inbar Mosseri.
ICCV 2021. [PDF] [Project]

BDInvert: GAN Inversion for Out-of-Range Images with Geometric Transformations.
Kyoungkook Kang, Seongtae Kim, Sunghyun Cho.
ICCV 2021. [PDF] [Project]

ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement.
Yuval Alaluf, Or Patashnik, Daniel Cohen-Or.
ICCV 2021. [PDF] [Project] [Github]

LatentCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions.
Oğuz Kaan Yüksel, Enis Simsar, Ezgi Gülperi Er, Pinar Yanardag.
ICCV 2021. [PDF] [Github]

Lifting 2D StyleGAN for 3D-Aware Face Generation.
Yichun Shi, Divyansh Aggarwal, Anil K. Jain.
CVPR 2021. [PDF]

Ensembling with Deep Generative Views.
Lucy Chai, Jun-Yan Zhu, Eli Shechtman, Phillip Isola, Richard Zhang.
CVPR 2021. [PDF] [Github] [Project]

Navigating the GAN Parameter Space for Semantic Image Editing.
Anton Cherepkov, Andrey Voynov, Artem Babenko.
CVPR 2021. [PDF] [Github]

StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation.
Zongze Wu, Dani Lischinski, Eli Shechtman.
CVPR 2021 (oral). [PDF] [Github]

Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation.
Elad Richardson, Yuval Alaluf, Or Patashnik, Yotam Nitzan, Yaniv Azar, Stav Shapiro, Daniel Cohen-Or.
CVPR 2021. [PDF] [Github] [Project]

GHFeat: Generative Hierarchical Features from Synthesizing Images.
Yinghao Xu, Yujun Shen, Jiapeng Zhu, Ceyuan Yang, Bolei Zhou.
CVPR 2021. [PDF] [Github] [Project]

Hijack-GAN: Unintended-Use of Pretrained, Black-Box GANs.
Hui-Po Wang, Ning Yu, Mario Fritz.
CVPR 2021. [PDF]

Prior Image-Constrained Reconstruction using Style-Based Generative Models.
Varun A Kelkar, Mark Anastasio.
ICML 2021. [PDF]

Intermediate Layer Optimization for Inverse Problems using Deep Generative Models.
Giannis Daras, Joseph Dean, Ajil Jalal, Alexandros G. Dimakis.
ICML 2021. [PDF] [Github]

Using Latent Space Regression to Analyze and Leverage Compositionality in GANs.
Lucy Chai, Jonas Wulff, Phillip Isola.
ICLR 2021. [PDF] [Github] [Project] [Colab]

Enjoy Your Editing: Controllable GANs for Image Editing via Latent Space Navigation.
Peiye Zhuang, Oluwasanmi Koyejo, Alexander G. Schwing.
ICLR 2021. [PDF]

Disentangled Face Attribute Editing via Instance-Aware Latent Space Search.
Yuxuan Han, Jiaolong Yang, Ying Fu.
IJCAI 2021. [PDF] [Github]

High Fidelity GAN Inversion via Prior Multi-Subspace Feature Composition.
Guanyue Li, Qianfen Jiao, Sheng Qian, Si Wu, Hau-San Wong.
AAAI 2021. [PDF]

e4e: Designing an Encoder for StyleGAN Image Manipulation.
Omer Tov, Yuval Alaluf, Yotam Nitzan, Or Patashnik, Daniel Cohen-Or.
TOG 2021. [PDF] [Github]

StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows.
Rameen Abdal, Peihao Zhu, Niloy Mitra, Peter Wonka.
TOG 2021. [PDF] [Github]

Mask-Guided Discovery of Semantic Manifolds in Generative Models.
Mengyu Yang, David Rokeby, Xavier Snelgrove.
NeurIPS 2020 Workshop on Machine Learning for Creativity and Design. [PDF] [Github]

Face Identity Disentanglement via Latent Space Mapping.
Yotam Nitzan, Amit Bermano, Yangyan Li, Daniel Cohen-Or.
SIGGRAPH ASIA 2020. [PDF] [Project] [Github]

PIE: Portrait Image Embedding for Semantic Control.
A. Tewari, M. Elgharib, M. BR, F. Bernard, H-P. Seidel, P. P‌érez, M. Zollhöfer, C.Theobalt.
TOG 2020. [PDF] [Project]

Understanding the Role of Individual Units in a Deep Neural Network.
David Bau, Jun-Yan Zhu, Hendrik Strobelt, Agata Lapedriza, Bolei Zhou, Antonio Torralba.
National Academy of Sciences 2020. [PDF] [Github] [Project]

Face Identity Disentanglement via Latent Space Mapping.
Yotam Nitzan, Amit Bermano, Yangyan Li, Daniel Cohen-Or.
TOG 2020. [PDF] [Github]

Transforming and Projecting Images into Class-conditional Generative Networks.
Minyoung Huh, Richard Zhang, Jun-Yan Zhu, Sylvain Paris, Aaron Hertzmann.
ECCV 2020. [PDF] [Github] [Project]

MimicGAN: Robust Projection onto Image Manifolds with Corruption Mimicking.
Rushil Anirudh, Jayaraman J. Thiagarajan, Bhavya Kailkhura, Timo Bremer.
IJCV 2020. [PDF]

Rewriting a Deep Generative Model.
David Bau, Steven Liu, Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba.
ECCV 2020. [PDF] [Github]

StyleGAN2 Distillation for Feed-forward Image Manipulation.
Yuri Viazovetskyi, Vladimir Ivashkin, Evgeny Kashin.
ECCV 2020. [PDF] [Github]

In-Domain GAN Inversion for Real Image Editing.
Jiapeng Zhu, Yujun Shen, Deli Zhao, Bolei Zhou.
ECCV 2020. [PDF] [Project] [Github]

Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation.
Xingang Pan, Xiaohang Zhan, Bo Dai, Dahua Lin, Chen Change Loy, Ping Luo.
ECCV 2020. [PDF] [Github]

Your Local GAN: Designing Two Dimensional Local Attention Mechanisms for Generative Models.
Giannis Daras, Augustus Odena, Han Zhang, Alexandros G. Dimakis.
CVPR 2020. [PDF]

A Disentangling Invertible Interpretation Network for Explaining Latent Representations.
Patrick Esser, Robin Rombach, Björn Ommer.
CVPR 2020. [PDF] [Project] [Github]

Editing in Style: Uncovering the Local Semantics of GANs.
Edo Collins, Raja Bala, Bob Price, Sabine Süsstrunk.
CVPR 2020. [PDF] [Github]

Image Processing Using Multi-Code GAN Prior.
Jinjin Gu, Yujun Shen, Bolei Zhou.
CVPR 2020. [PDF] [Project] [Github]

Image2StyleGAN++: How to Edit the Embedded Images?
Rameen Abdal, Yipeng Qin, Peter Wonka.
CVPR 2020. [PDF]

Semantic Photo Manipulation with a Generative Image Prior.
David Bau, Hendrik Strobelt, William Peebles, Jonas, Bolei Zhou, Jun-Yan Zhu, Antonio Torralba.
TOG 2019. [PDF]

Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?
Rameen Abdal, Yipeng Qin, Peter Wonka.
ICCV 2019. [PDF] [Github]

GAN-based Projector for Faster Recovery with Convergence Guarantees in Linear Inverse Problems.
Ankit Raj, Yuqi Li, Yoram Bresler.
ICCV 2019. [PDF]

Inverting Layers of a Large Generator.
David Bau, Jun-Yan Zhu, Jonas Wulff, William Peebles, Hendrik Strobelt, Bolei Zhou, Antonio Torralba.
ICCV 2019. [PDF]

Detecting Overfitting in Deep Generators via Latent Recovery.
Ryan Webster, Julien Rabin, Loic Simon, Frederic Jurie.
CVPR 2019. [PDF][Colab]

Inverting The Generator Of A Generative Adversarial Network (II).
Antonia Creswell, Anil A Bharath.
TNNLS 2018. [PDF] [Github]

Invertibility of Convolutional Generative Networks from Partial Measurements.
Fangchang Ma, Ulas Ayaz, Sertac Karaman.
NeurIPS 2018. [PDF] [Github]

Metrics for Deep Generative Models.
Nutan Chen, Alexej Klushyn, Richard Kurle, Xueyan Jiang, Justin Bayer, Patrick van der Smagt.
AISTATS 2018. [PDF]

Towards Understanding the Invertibility of Convolutional Neural Networks.
Anna C. Gilbert, Yi Zhang, Kibok Lee, Yuting Zhang, Honglak Lee.
IJCAI 2017. [PDF]

One Network to Solve Them All - Solving Linear Inverse Problems using Deep Projection Models.
J. H. Rick Chang, Chun-Liang Li, Barnabas Poczos, B. V. K. Vijaya Kumar, Aswin C. Sankaranarayanan.
ICCV 2017. [PDF]

Precise Recovery of Latent Vectors from Generative Adversarial Networks.
Zachary C. Lipton, Subarna Tripathi.
ICLR 2017 workshop. [PDF] [Github]

Inverting The Generator Of A Generative Adversarial Network.
Antonia Creswell, Anil Anthony Bharath.
NeurIPS 2016 Workshop. [PDF]

Generative Visual Manipulation on the Natural Image Manifold.
Jun-Yan Zhu, Philipp Krähenbühl, Eli Shechtman, Alexei A. Efros.
ECCV 2016. [PDF]

🔝

diffusion inversion and manipulation

diffusion inversion

NULL-text Inversion for Editing Real Images using Guided Diffusion Models.
Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, Daniel Cohen-Or.
CVPR 2023. [PDF] [Project] [Github]

EDICT: Exact Diffusion Inversion via Coupled Transformations.
Bram Wallace, Akash Gokul, Nikhil Naik.
CVPR 2023. [PDF] [Github]

An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion.
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik, Daniel Cohen-Or.
ICLR 2023 (Oral). [PDF] [Project] [Github]

Prompt-to-Prompt Image Editing with Cross Attention Control.
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, Daniel Cohen-Or.
ICLR 2023. [PDF] [Project] [Github]

An Edit Friendly DDPM Noise Space: Inversion and Manipulations.
Inbar Huberman-Spiegelglas, Vladimir Kulikov, Tomer Michaeli.
arXiv 2023. [PDF] [Project] [Github]

Discovering Interpretable Directions in the Semantic Latent Space of Diffusion Models.
René Haas, Inbar Huberman-Spiegelglas, Rotem Mulayoff, and Tomer Michaeli.
arXiv 2023. [PDF]

A Neural Space-Time Representation for Text-to-Image Personalization.
Yuval Alaluf, Elad Richardson, Gal Metzer, Daniel Cohen-Or.
arXiv 2023. [PDF] [Project]

semantic manipulation in diffusion latent spaces

An Edit Friendly DDPM Noise Space: Inversion and Manipulations.
Inbar Huberman-Spiegelglas, Vladimir Kulikov, Tomer Michaeli.
arXiv 2023. [PDF]

Discovering Interpretable Directions in The Semantic Latent Space of Diffusion Models.
René Haas, Inbar Huberman-Spiegelglas, Rotem Mulayoff, Tomer Michaeli.
arXiv 2023. [PDF]

$P+$: Extended Textual Conditioning in Text-to-Image Generation.
Andrey Voynov, Qinghao Chu, Daniel Cohen-Or, Kfir Aberman.
arXiv 2023. [PDF]

Unsupervised Discovery of Semantic Latent Directions in Diffusion Models.
Yong-Hyun Park, Mingi Kwon, Junghyo Jo, Youngjung Uh.
arXiv 2023. [PDF]

Diffusion Models Already Have A Semantic Latent Space.
Mingi Kwon, Jaeseok Jeong, Youngjung Uh.
ICLR2023. [PDF]

The Stable Artist: Steering Semantics in Diffusion Latent Space.
Manuel Brack, Patrick Schramowski, Felix Friedrich, Dominik Hintersdorf, Kristian Kersting.
arXiv 2022. [PDF]

Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidance.
Chen Henry Wu, Fernando De la Torre.
arXiv 2022. [PDF]

Generative Visual Prompt: Unifying Distributional Control of Pre-Trained Generative Models.
Chen Henry Wu, Saman Motamed, Shaunak Srivastava, Fernando De la Torre.
NeurIPS 2022. [PDF]

🔝

latent space traversal

Inversion is not the ultimate goal. The reason that we invert a real image into the latent space of a trained GAN model is that we can manipulate the inverted image in the latent space by discovering the desired code with certain attributes. This technique is usually known as latent space navigation, latent space traversal, GAN steerability, latent code manipulation, or other names in the literature. Although often regarded as an independent research field, it acts as an indispensable component of GAN inversion for manipulation. Many inversion methods also involve efficient discovery of a desired latent code.

Deep Curvilinear Editing: Commutative and Nonlinear Image Manipulation for Pretrained Deep Generative Model.
Takehiro Aoshima, Takashi Matsubara.
CVPR 2023. [PDF]

Latent Traversals in Generative Models as Potential Flows.
Yue Song, Andy Keller, Nicu Sebe, Max Welling.
ICML 2023. [PDF] [Github]

PandA: Unsupervised Learning of Parts and Appearances in the Feature Maps of GANs.
James Oldfield, Christos Tzelepis, Yannis Panagakis, Mihalis A. Nicolaou, Ioannis Patras.
ICLR 2023. [PDF] [Project] [Github]

Rayleigh EigenDirections (REDs): Nonlinear GAN Latent Space Traversals for Multidimensional Features.
Guha Balakrishnan, Raghudeep Gadde, Aleix Martinez, Pietro Perona.
ECCV 2022. [PDF]

Exploring Gradient-based Multi-directional Controls in GANs.
Zikun Chen, Ruowei Jiang, Brendan Duke, Han Zhao, Parham Aarabi.
ECCV 2022 (oral). [PDF] [Project]

Hierarchical Semantic Regularization of Latent Spaces in StyleGANs.
Tejan Karmali, Rishubh Parihar, Susmit Agrawal, Harsh Rangwani, Varun Jampani, Maneesh Singh, R. Venkatesh Babu.
ECCV 2022. [PDF] [Project]

CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directions.
Rameen Abdal, Peihao Zhu, John Femiani, Niloy J. Mitra, Peter Wonka.
SIGGRAPH 2022. [PDF] [Github]

Region-Based Semantic Factorization in GANs.
Jiapeng Zhu, Yujun Shen, Yinghao Xu, Deli Zhao, Qifeng Chen.
ICML 2022. [PDF] [Github]

Latent Image Animator: Learning to Animate Image via Latent Space Navigation.
Yaohui Wang, Di Yang, Francois Bremond, Antitza Dantcheva.
ICLR 2022. [PDF] [Project] [Github]

Do Not Escape From the Manifold: Discovering the Local Coordinates on the Latent Space of GANs.
Jaewoong Choi, Changyeon Yoon, Junho Lee, Jung Ho Park, Geonho Hwang, Myungjoo Kang.
ICLR 2022. [PDF]

Tensor-based Emotion Editing in the StyleGAN Latent Space.
René Haas, Stella Graßhof, Sami S. Brandt.
CVPR 2022 Workshop on AI for Content Creation. [PDF]

PaintInStyle: One-Shot Discovery of Interpretable Directions by Painting.
Berkay Doner, Elif Sema Balcioglu, Merve Rabia Barin, Umut Kocasari, Mert Tiftikci, Pinar Yanardag.
CVPR 2022 Workshop. [PDF]

Rank in Style: A Ranking-Based Approach To Find Interpretable Directions.
Umut Kocasari, Kerem Zaman, Mert Tiftikci, Enis Simsar, Pinar Yanardag.
CVPR 2022 Workshop. [PDF]

LARGE: Latent-Based Regression through GAN Semantics.
Yotam Nitzan, Rinon Gal, Ofir Brenner, Daniel Cohen-Or.
CVPR 2022. [PDF] [Github] [Project]

StyleFusion: Disentangling Spatial Segments in StyleGAN-Generated Images.
Omer Kafri, Or Patashnik, Yuval Alaluf, Daniel Cohen-Or.
TOG 2022. [PDF] [Github]

Tensor Component Analysis for Interpreting the Latent Space of GANs.
James Oldfield, Markos Georgopoulos, Yannis Panagakis, Mihalis A. Nicolaou, Ioannis Patras.
BMVC 2021. [PDF] [Project] [Github]

Tensor-based Subspace Factorization for StyleGAN.
Rene Haas, Stella Graßhof and Sami S. Brandt.
FG 2021. [PDF]

Exploratory Search of GANs with Contextual Bandits.
Ivan Kropotov, Alan Medlar, Dorota Glowacka.
CIKM 2021. [PDF]

LowRankGAN: Low-Rank Subspaces in GANs.
Jiapeng Zhu, Ruili Feng, Yujun Shen, Deli Zhao, Zhengjun Zha, Jingren Zhou, Qifeng Chen.
NeurIPS 2021. [PDF] [Github]

Controllable and Compositional Generation with Latent-Space Energy-Based Models.
Weili Nie, Arash Vahdat, Anima Anandkumar.
NeurIPS 2021. [PDF]

A Latent Transformer for Disentangled Face Editing in Images and Videos.
Xu Yao, Alasdair Newson, Yann Gousseau, Pierre Hellier.
ICCV 2021. [PDF] [ArXiV] [Github]

Toward a Visual Concept Vocabulary for GAN Latent Space.
Sarah Schwettmann, Evan Hernandez, David Bau, Samuel Klein, Jacob Andreas, Antonio Torralba.
ICCV 2021. [PDF] [Project]

WarpedGANSpace: Finding Non-linear RBF Paths in GAN Latent Space.
Christos Tzelepis, Georgios Tzimiropoulos, Ioannis Patras.
ICCV 2021. [PDF] [Github]

Latent Transformations via NeuralODEs for GAN-based Image Editing.
Valentin Khrulkov, Leyla Mirvakhabova, Ivan Oseledets, Artem Babenko.
ICCV 2021. [PDF] [Github]

OroJaR: Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generation.
Yuxiang Wei, Yupeng Shi, Xiao Liu, Zhilong Ji, Yuan Gao, Zhongqin Wu, Wangmeng Zuo.
ICCV 2021. [PDF] [Github]

EigenGAN: Layer-Wise Eigen-Learning for GANs.
Zhenliang He, Meina Kan, Shiguang Shan.
ICCV 2021. [PDF] [Github]

SSFlow: Style-guided Neural Spline Flows for Face Image Manipulation.
Hanbang Liang, Xianxu Hou, Linlin Shen.
ACM MM 2021. [PDF]

SalS-GAN: Spatially-Adaptive Latent Space in StyleGAN for Real Image Embedding.
Lingyun Zhang, Xiuxiu Bai, Yao Gao.
ACM MM 2021. [PDF]

Discovering Density-Preserving Latent Space Walks in GANs for Semantic Image Transformations.
Guanyue Li, Yi Liu, Xiwen Wei, Yang Zhang, Si Wu, Yong Xu, Hau San Wong.
ACM MM 2021. [PDF]

Discovering Interpretable Latent Space Directions of GANs Beyond Binary Attributes.
Huiting Yang, Liangyu Chai, Qiang Wen, Shuang Zhao, Zixun Sun, Shengfeng He.
CVPR 2021. [PDF]

Surrogate Gradient Field for Latent Space Manipulation.
Minjun Li, Yanghua Jin, Huachun Zhu.
CVPR 2021. [PDF]

SeFa: Closed-Form Factorization of Latent Semantics in GANs.
Yujun Shen, Bolei Zhou.
CVPR 2021. [PDF] [Github] [Project]

L2M-GAN: Learning To Manipulate Latent Space Semantics for Facial Attribute Editing.
Guoxing Yang, Nanyi Fei, Mingyu Ding, Guangzhen Liu, Zhiwu Lu, Tao Xiang.
CVPR 2021. [PDF] [Unofficial Pytorch]

MoCoGAN-HD: A Good Image Generator Is What You Need for High-Resolution Video Synthesis.
Yu Tian, Jian Ren, Menglei Chai, Kyle Olszewski, Xi Peng, Dimitris N. Metaxas, Sergey Tulyakov.
ICLR 2021. [PDF] [Github]

GAN Steerability without optimization.
Nurit Spingarn-Eliezer, Ron Banner, Tomer Michaeli.
ICLR 2021. [OpenReview] [PDF]

On the "steerability" of generative adversarial networks.
Ali Jahanian, Lucy Chai, Phillip Isola.
ICLR 2020. [PDF] [Project]

GANSpace: Discovering Interpretable GAN Controls.
Erik Härkönen, Aaron Hertzmann, Jaakko Lehtinen, Sylvain Paris.
NeurIPS 2020. [PDF] [Github]

Interpreting the Latent Space of GANs for Semantic Face Editing.
Yujun Shen, Jinjin Gu, Xiaoou Tang, Bolei Zhou.
CVPR 2020. [PDF] [Project] [Github]

Seeing What a GAN Cannot Generate.
David Bau, Jun-Yan Zhu, Jonas Wulff, William Peebles, Hendrik Strobelt, Bolei Zhou, Antonio Torralba.
ICCV 2019. [PDF] [PDF]

Unsupervised Discovery of Interpretable Directions in the GAN Latent Space.
Andrey Voynov, Artem Babenko.
ICML 2020. [PDF] [Github]

Fantastic Style Channels and Where to Find Them: A Submodular Framework for Discovering Diverse Directions in GANs.
Enis Simsar, Umut Kocasari, Ezgi Gülperi Er, Pinar Yanardag.
arxiv 2022. [PDF] [Project] [Demo]

Analyzing the Latent Space of GAN through Local Dimension Estimation.
Jaewoong Choi, Geonho Hwang, Hyunsoo Cho, Myungjoo Kang.
arxiv 2022. [PDF]

Multi-level Latent Space Structuring for Generative Control.
Oren Katzir, Vicky Perepelook, Dani Lischinski, Daniel Cohen-Or.
arxiv 2022. [PDF]

Rayleigh EigenDirections (REDs): GAN Latent Space Traversals for Multidimensional Features.
Guha Balakrishnan, Raghudeep Gadde, Aleix Martinez, Pietro Perona.
arxiv 2021. [PDF]

Optimizing Latent Space Directions For GAN-based Local Image Editing.
Ehsan Pajouheshgar, Tong Zhang, Sabine Süsstrunk.
arxiv 2021. [PDF]

🔝

application

image and video generation and manipulation

NeRFInvertor: High Fidelity NeRF-GAN Inversion for Single-shot Real Image Animation.
Yu Yin, Kamran Ghasedi, HsiangTao Wu, Jiaolong Yang, Xin Tong, Yun Fu.
CVPR 2023. [PDF]

Fine-Grained Face Swapping via Regional GAN Inversion.
Zhian Liu, Maomao Li, Yong Zhang, Cairong Wang, Qi Zhang, Jue Wang, Yongwei Nie.
CVPR 2023. [PDF] [Project]

VIVE3D: Viewpoint-Independent Video Editing using 3D-Aware GANs.
Anna Frühstück, Nikolaos Sarafianos, Yuanlu Xu, Peter Wonka, Tony Tung.
CVPR 2023. [PDF] [Project] [Github]

Delving StyleGAN Inversion for Image Editing: A Foundation Latent Space Viewpoint.
Hongyu Liu, Yibing Song, Qifeng Chen.
CVPR 2023. [PDF] [Project] [Github]

Dr.3D: Adapting 3D GANs to Artistic Drawings.
Wonjoon Jin, Nuri Ryu, Geonung Kim, Seung-Hwan Baek, Sunghyun Cho.
SIGGRAPH Asia 2022. [PDF] [Project]

Stitch it in Time: GAN-Based Facial Editing of Real Videos.
Rotem Tzaban, Ron Mokady, Rinon Gal, Amit H. Bermano, Daniel Cohen-Or.
SIGGRAPH Asia 2022. [PDF] [Project] [Github]

DyStyle: Dynamic Neural Network for Multi-Attribute-Conditioned Style Editing.
Bingchuan Li, Shaofei Cai, Wei Liu, Peng Zhang, Miao Hua, Qian He, Zili Yi.
WACV 2023. [PDF] [Github]

Generative Visual Prompt: Unifying Distributional Control of Pre-Trained Generative Models.
Chen Henry Wu, Saman Motamed, Shaunak Srivastava, Fernando De la Torre.
NeurIPS 2022. [PDF] [Github]

3D-FM GAN: Towards 3D-Controllable Face Manipulation.
Yuchen Liu, Zhixin Shu, Yijun Li, Zhe Lin, Richard Zhang, and Sun-Yuan Kung.
ECCV 2022. [PDF] [Project]

JoJoGAN: One Shot Face Stylization.
Min Jin Chong, David Forsyth.
ECCV 2022. [PDF] [Github]

Generative Multiplane Images: Making a 2D GAN 3D-Aware.
Xiaoming Zhao, Fangchang Ma, David Güera, Zhile Ren, Alexander G. Schwing, Alex Colburn.
ECCV 2022. [PDF] [Project] [Github]

Injecting 3D Perception of Controllable NeRF-GAN into StyleGAN for Editable Portrait Image Synthesis.
Jeong-gi Kwak, Yuanming Li, Dongsik Yoon, Donghyeon Kim, David Han, Hanseok Ko.
ECCV 2022. [PDF] [Project]

Temporally Consistent Semantic Video Editing.
Yiran Xu, Badour AlBahar, Jia-Bin Huang.
ECCV 2022. [PDF] [Project]

Sound-Guided Semantic Video Generation.
Seung Hyun Lee, Gyeongrok Oh, Wonmin Byeon, Jihyun Bae, Chanyoung Kim, Won Jeong Ryoo, Sang Ho Yoon, Jinkyu Kim, Sangpil Kim.
ECCV 2022. [PDF] [Project] [Github]

Third Time's the Charm? Image and Video Editing with StyleGAN3.
Yuval Alaluf, Or Patashnik, Zongze Wu, Asif Zamir, Eli Shechtman, Dani Lischinski, Daniel Cohen-Or.
ECCV 2022 Workshop on Advances in Image Manipulation. [PDF] [Project] [Github]

Everything is There in Latent Space: Attribute Editing and Attribute Style Manipulation by StyleGAN Latent Space Exploration.
Rishubh Parihar, Ankit Dhiman, Tejan Karmali, R. Venkatesh Babu.
ACM MM 2022. [PDF] [Project]

SD-GAN: Semantic Decomposition for Face Image Synthesis with Discrete Attribute.
Zhou Kangneng, Zhu Xiaobin, Gao Daiheng, Lee Kai, Li Xinjie, Yin Xu-Cheng.
ACM MM 2022. [PDF]

Identity-Guided Face Generation with Multi-modal Contour Conditions.
Qingyan Bai, Weihao Xia, Fei Yin, Yujiu Yang.
ICIP 2022. [PDF]

Self-Conditioned Generative Adversarial Networks for Image Editing.
Yunzhe Liu, Rinon Gal, Amit H. Bermano, Baoquan Chen, Daniel Cohen-Or.
SIGGRAPH 2022. [PDF] [Project]

StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generators.
Rinon Gal, Or Patashnik, Haggai Maron, Gal Chechik, Daniel Cohen-Or.
SIGGRAPH 2022. [PDF] [Project] [Github]

SphericGAN: Semi-Supervised Hyper-Spherical Generative Adversarial Networks for Fine-Grained Image Synthesis.
Tianyi Chen, Yunfei Zhang, Xiaoyang Huo, Si Wu, Yong Xu, Hau San Wong.
CVPR 2022. [PDF]

Sound-Guided Semantic Image Manipulation.
Seung Hyun Lee, Wonseok Roh, Wonmin Byeon, Sang Ho Yoon, Chan Young Kim, Jinkyu Kim, Sangpil Kim.
CVPR 2022. [PDF]

HairCLIP: Design Your Hair by Text and Reference Image.
Tianyi Wei, Dongdong Chen, Wenbo Zhou, Jing Liao, Zhentao Tan, Lu Yuan, Weiming Zhang, Nenghai Yu.
CVPR 2022. [PDF] [Github]

HairMapper: Removing Hair from Portraits Using GANs.
Yiqian Wu, Yong-Liang Yang, Xiaogang Jin.
CVPR 2022. [PDF] [Project] [Github] [Non-hair-FFHQ Data]

Attribute Group Editing for Reliable Few-shot Image Generation.
Guanqi Ding, Xinzhe Han, Shuhui Wang, Shuzhe Wu, Xin Jin, Dandan Tu, Qingming Huang.
CVPR 2022. [PDF] [Github]

InsetGAN for Full-Body Image Generation.
Anna Frühstück, Krishna Kumar Singh, Eli Shechtman, Niloy J. Mitra, Peter Wonka, Jingwan Lu.
CVPR 2022. [PDF] [Project] [Unofficial]

SpaceEdit: Learning a Unified Editing Space for Open-Domain Image Editing.
Jing Shi, Ning Xu, Haitian Zheng, Alex Smith, Jiebo Luo, Chenliang Xu.
CVPR 2022. [PDF]

In&Out: Diverse Image Outpainting via GAN Inversion.
Yen-Chi Cheng, Chieh Hubert Lin, Hsin-Ying Lee, Jian Ren, Sergey Tulyakov, Ming-Hsuan Yang.
CVPR 2022. [PDF] [Github]

InfinityGAN: Towards Infinite-Resolution Image Synthesis.
Chieh Hubert Lin, Hsin-Ying Lee, Yen-Chi Cheng, Sergey Tulyakov, Ming-Hsuan Yang.
ICLR 2022. [PDF] [Project]

Latent to Latent: A Learned Mapper for Identity Preserving Editing of Multiple.
Siavash Khodadadeh, Shabnam Ghadar, Saeid Motiian, Wei-An Lin, Ladislau Bölöni, Ratheesh Kalarot.
WACV 2022. [PDF]

StyleVideoGAN: A Temporal Generative Model using a Pretrained StyleGAN.
Gereon Fox, Ayush Tewari, Mohamed Elgharib, Christian Theobalt.
BMVC 2021 (Oral). [PDF]

Face Image Retrieval With Attribute Manipulation.
Alireza Zaeemzadeh, Shabnam Ghadar, Baldo Faieta, Zhe Lin, Nazanin Rahnavard, Mubarak Shah, Ratheesh Kalarot.
ICCV 2021. [PDF]

StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation.
Wongjong Jang, Gwangjin Ju, Yucheol Jung, Jiaolong Yang, Xin Tong, Seungyong Lee.
TOG 2021. [PDF] [Github]

Coarse-to-Fine: Facial Structure Editing of Portrait Images via Latent Space Classifications.
Yiqian Wu, Yongliang Yang, Qinjie Xiao, Xiaogang Ji.
TOG 2021. [PDF] [Project]

SAM: Only a Matter of Style-Age Transformation Using a Style-Based Regression Model.
Yuval Alaluf, Or Patashnik, Daniel Cohen-Or.
TOG 2021. [PDF] [Github]

Barbershop: GAN-based Image Compositing using Segmentation Masks.
Peihao Zhu, Rameen Abdal, John Femiani, Peter Wonka.
SIGGRAPH Asia 2021. [PDF] [Project] [Github]

Constrained Graphic Layout Generation via Latent Optimization.
Kotaro Kikuchi, Edgar Simo-Serra, Mayu Otani, Kota Yamaguchi.
ACM MM 2021. [PDF] [Github]

CI-GAN: Cycle-Consistent Inverse GAN for Text-to-Image Synthesis.
Hao Wang, Guosheng Lin, Steven C. H. Hoi, Chunyan Miao.
ACM MM 2021. [PDF]

Exploring Adversarial Fake Images on Face Manifold.
Dongze Li, Wei Wang, Hongxing Fan, Jing Dong.
CVPR 2021. [PDF]

HistoGAN: Controlling Colors of GAN-Generated and Real Images via Color Histograms.
Mahmoud Afifi, Marcus A. Brubaker, Michael S. Brown.
CVPR 2021. [PDF] [Github] [4K Landscape]

One Shot Face Swapping on Megapixels.
Yuhao Zhu, Qi Li, Jian Wang, Chengzhong Xu, Zhenan Sun.
CVPR 2021. [PDF] [Github]

LOHO: Latent Optimization of Hairstyles via Orthogonalization.
Rohit Saha, Brendan Duke, Florian Shkurti, Graham W. Taylor, Parham Aarabi.
CVPR 2021. [PDF] [Github]

StyleMapGAN: Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editing.
Hyunsu Kim, Yunjey Choi, Junho Kim, Sungjoo Yoo, Youngjung Uh.
CVPR 2021. [PDF] [Github]

TediGAN: Text-Guided Diverse Image Generation and Manipulation.
Weihao Xia, Yujiu Yang, Jing-Hao Xue, Baoyuan Wu.
CVPR 2021. [PDF] [Data] [Github]

DeepI2I: Enabling Deep Hierarchical Image-to-Image Translation by Transferring from GANs.
yaxing wang, Lu Yu, Joost van de Weijer.
NeurIPS 2020. [PDF] [Github]

DeepLandscape: Adversarial Modeling of Landscape Videos.
E. Logacheva, R. Suvorov, O. Khomenko, A. Mashikhin, and V. Lempitsky.
ECCV 2020. [PDF] [Github] [Project]

Balancing Reconstruction and Editing Quality of GAN Inversion for Real Image Editing with StyleGAN Prior Latent Space.
Kai Katsumata, Duc Minh Vo, Bei Liu, Hideki Nakayama.
CVPR 2023 Workshop on AI4CC. [PDF]

Make It So: Steering StyleGAN for Any Image Inversion and Editing.
Anand Bhattad, Viraj Shah, Derek Hoiem, D. A. Forsyth.
arXiv 2023. [PDF] [Project]

One-Shot Face Video Re-enactment using Hybrid Latent Spaces of StyleGAN2.
Trevine Oorloff, Yaser Yacoob.
arXiv 2023. [PDF] [Project]

In-N-Out: Face Video Inversion and Editing with Volumetric Decomposition.
Yiran Xu, Zhixin Shu, Cameron Smith, Jia-Bin Huang, Seoung Wug Oh.
arXiv 2023. [PDF] [Project]

Modeling the Latent Dynamics of StyleGAN using Neural ODEs.
Weihao Xia, Yujiu Yang, Jing-Hao Xue.
arXiv 2022. [PDF] [Github]

Unsupervised Domain Adaptation GAN Inversion for Image Editing.
Siyu Xing, Chen Gong, Hewei Guo, Xiao-Yu Zhang, Xinwen Hou, Yu Liu.
arXiv 2022. [PDF]

MultiStyleGAN: Multiple One-shot Face Stylizations using a Single GAN.
Viraj Shah, Svetlana Lazebnik.
arxiv 2022. [PDF] [Project] [Github]

Generalized One-shot Domain Adaption of Generative Adversarial Networks.
Zicheng Zhang, Yinglu Liu, Congying Han, Tiande Guo, Ting Yao, Tao Mei.
arxiv 2022. [PDF]

Robust Sound-Guided Image Manipulation.
Seung Hyun Lee, Chanyoung Kim, Wonmin Byeon, Gyeongrok Oh, Jooyoung Lee, Sang Ho Yoon, Jinkyu Kim, Sangpil Kim.
arxiv 2022. [PDF]

Few-shot Semantic Image Synthesis Using StyleGAN Prior.
Yuki Endo, Yoshihiro Kanamori.
arxiv 2021. [PDF] [Github]

Heredity-aware Child Face Image Generation with Latent Space Disentanglement.
Xiao Cui, Wengang Zhou, Yang Hu, Weilun Wang, Houqiang Li.
arxiv 2021. [PDF]

Paint by Word.
David Bau, Alex Andonian, Audrey Cui, YeonHwan Park, Ali Jahanian, Aude Oliva, Antonio Torralba.
arxiv 2021. [PDF] [Project] [Project]

image restoration

High-Fidelity Image Inpainting with GAN Inversion.
Yongsheng Yu, Libo Zhang, Heng Fan, Tiejian Luo.
ECCV 2022. [PDF]

Escaping Data Scarcity for High-Resolution Heterogeneous Face Hallucination.
Yiqun Mei, Pengfei Guo, Vishal M. Patel.
CVPR 2022. [PDF]

Towards High-Fidelity Face Self-Occlusion Recovery via Multi-View Residual-Based GAN Inversion.
Jinsong Chen, Hu Han, Shiguang Shan.
AAAI 2022. [PDF]

Time-Travel Rephotography.
Xuan Luo, Xuaner Zhang, Paul Yoo, Ricardo Martin-Brualla, Jason Lawrence, Steven M. Seitz.
SIGGRAPH Asia 2021 (TOG). [PDF] [Project] [Github]

GPEN: GAN Prior Embedded Network for Blind Face Restoration in the Wild.
Tao Yang, Peiran Ren, Xuansong Xie, Lei Zhang.
CVPR 2021. [PDF] [Github]

GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution.
Kelvin C.K. Chan, Xintao Wang, Xiangyu Xu, Jinwei Gu, Chen Change Loy.
CVPR 2021. [PDF] [Project] [Github]

GFP-GAN: Towards Real-World Blind Face Restoration with Generative Facial Prior.
Xintao Wang, Yu Li, Honglun Zhang, Ying Shan.
CVPR 2021. [PDF] [Project]

PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models.
Sachit Menon, Alexandru Damian, Shijia Hu, Nikhil Ravi, Cynthia Rudin.
CVPR 2020. [PDF] [Github]

Semantic uncertainty intervals for disentangled latent spaces.
Swami Sankaranarayanan, Anastasios N. Angelopoulos, Stephen Bates, Yaniv Romano, Phillip Isola.
arxiv 2021. [PDF] [Github]

LTT-GAN: Looking Through Turbulence by Inverting GANs.
Kangfu Mei, Vishal M. Patel.
arxiv 2021. [PDF] [Project]

Style Generator Inversion for Image Enhancement and Animation.
Aviv Gabbay, Yedid Hoshen.
arxiv 2019. [PDF] [Project] [Github]

image understanding

Segmentation in Style: Unsupervised Semantic Image Segmentation with Stylegan and CLIP.
Daniil Pakhomov, Sanchit Hira, Narayani Wagle, Kemar E. Green, Nassir Navab.
arxiv 2021. [PDF] [Project] [Github]

Finding an Unsupervised Image Segmenter in each of your Deep Generative Models.
Luke Melas-Kyriazi, Christian Rupprecht, Iro Laina, Andrea Vedaldi.
ICLR 2022. [PDF]

Labels4Free: Unsupervised Segmentation using StyleGAN.
Rameen Abdal, Peihao Zhu, Niloy Mitra, Peter Wonka.
ICCV 2021. [PDF] [Project] [Github]

DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort.
Yuxuan Zhang, Huan Ling, Jun Gao, Kangxue Yin, Jean-Francois Lafleche, Adela Barriuso, Antonio Torralba, Sanja Fidler.
CVPR 2021. [PDF] [Github] [Project]

Repurposing GANs for One-shot Semantic Part Segmentation.
Nontawat Tritrong, Pitchaporn Rewatbowornwong, Supasorn Suwajanakorn.
CVPR 2021 (oral). [PDF] [Project] [Github]

recognition

How to Boost Face Recognition with StyleGAN?
Artem Sevastopolsky, Yury Malkov, Nikita Durasov, Luisa Verdoliva, Matthias Nießner.
arxiv 2022. [PDF]

3D

StyleGAN Knows Normal, Depth, Albedo, and More.
Anand Bhattad, Daniel McKee, Derek Hoiem, D. A. Forsyth.
arXiv 2023. [PDF]

Shape, Pose, and Appearance from a Single Image via Bootstrapped Radiance Field Inversion.
Dario Pavllo, David Joseph Tan, Marie-Julie Rakotosaona, Federico Tombari.
CVPR 2023. [PDF] [Github]

Monocular 3D Object Reconstruction with GAN Inversion.
Junzhe Zhang, Daxuan Ren, Zhongang Cai, Chai Kiat Yeo, Bo Dai, Chen Change Loy.
ECCV 2022. [PDF] [Project] [Github]

CoordGAN: Self-Supervised Dense Correspondences Emerge from GANs.
Jiteng Mu, Shalini De Mello, Zhiding Yu, Nuno Vasconcelos, Xiaolong Wang, Jan Kautz, Sifei Liu.
CVPR 2022. [PDF] [Project]

GAN2Shape: Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANs.
Xingang Pan, Bo Dai, Ziwei Liu, Chen Change Loy, Ping Luo.
ICLR 2021 (oral). [PDF] [Github] [Project]

Normalized Avatar Synthesis Using StyleGAN and Perceptual Refinement.
Huiwen Luo, Koki Nagano, Han-Wei Kung, Mclean Goldwhite, Qingguo Xu, Zejian Wang, Lingyu Wei, Liwen Hu, Hao Li.
CVPR 2021. [PDF]

Unsupervised 3D Shape Completion through GAN-Inversion.
Junzhe Zhang, Xinyi Chen, Zhongang Cai, Liang Pan, Haiyu Zhao, Shuai Yi, Chai Kiat Yeo, Bo Dai, Chen Change Loy.
CVPR 2021. [PDF] [Project]

Unsupervised 3D Shape Completion through GAN-Inversion.
Junzhe Zhang, Xinyi Chen, Zhongang Cai, Liang Pan, Haiyu Zhao, Shuai Yi, Chai Kiat Yeo, Bo Dai, Chen Change Loy.
CVPR 2021. [PDF] [Project]

OSTeC: One-Shot Texture Completion.
Baris Gecer, Jiankang Deng, Stefanos Zafeiriou.
CVPR 2021. [PDF] [Github]

StylePart: Image-based Shape Part Manipulation.
I-Chao Shen, Li-Wen Su, Yu-Ting Wu, Bing-Yu Chen.
arxiv 2021. [PDF]

compressed sensing

Generator Surgery for Compressed Sensing.
Niklas Smedemark-Margulies, Jung Yeon Park, Max Daniels, Rose Yu, Jan-Willem van de Meent, Paul Hand.
NeurIPS 2020 Workshop on Deep Inverse. [PDF] [Github]

Task-Aware Compressed Sensing with Generative Adversarial Networks.
Maya Kabkab, Pouya Samangouei, Rama Chellappa.
AAAI 2018. [PDF]

medical imaging

Controllable Medical Image Generation via Generative Adversarial Networks.
Zhihang Ren, Stella X. Yu, David Whitney.
Human Vision and Electronic Imaging 2021. [PDF]

High-resolution Controllable Prostatic Histology Synthesis using StyleGAN.
Gagandeep B. Daroach, Josiah A. Yoder, Kenneth A. Iczkowski, Peter S. LaViolette.
BIOIMAGING 2021. [PDF]

compression, fairness, and security

FairStyle: Debiasing StyleGAN2 with Style Channel Manipulations.
Cemre Karakas, Alara Dirik, Eylul Yalcinkaya, Pinar Yanardag.
ECCV 2022. [PDF] [Project] [Code]

Video Coding Using Learned Latent GAN Compression.
Mustafa Shukor, Bharath Bhushan Damodaran, Xu Yao, Pierre Hellier.
ACM Multimedia 2022. [PDF]

Differentially Private Imaging via Latent Space Manipulation.
Tao Li, Chris Clifton.
IEEE Symposium on Security & Privacy (S&P) 2021. [PDF]

acknowledgement

Thanks for the constructive comments from anonymous reviewers and feedback from Jun-Yan Zhu, Andrey Voynov, and Rushil Anirudh.

If you find this repo or our paper is helpful for your research, please consider to cite:

@article{xia2022gan,
    author  = {Xia, Weihao and Zhang, Yulun and Yang, Yujiu and Xue, Jing-Hao and Zhou, Bolei and Yang, Ming-Hsuan},
    title   = {GAN Inversion: A Survey},
    journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
    year={2022}
  }

🔝