Official code for ECCV2022 paper "Single Stage Virtual Try-on via Deformable Attention Flows"
We propose a single-stage try-on framework by developing a novel Deformable Attention Flow (DAFlow), which applies the deformable attention scheme to multi-flow estimation. With pose keypoints as the guidance only, the self- and cross-deformable attention flows are estimated for the reference person and the garment images, respectively. By sampling multiple flow fields, the feature-level and pixel-level information from different semantic areas are simultaneously extracted and merged through the attention mechanism. It enables clothes warping and body synthesizing at the same time which leads to photo-realistic results in an end-to-end manner.
- Datasets: Train_data Test_data, put the folders into
data/VITON
- Checkpoints: ckpt_viton.pt
- python 3.6
- pytorch1.7
- torchvision 0.8
The data process scripts in data/data_process
, the original data is from CPVTON-plus. Unzip the viton_plus.zip
from CPVTON-plus.
Drawing img_agnostic:
python get_img_agnostic.py --data_path img_path --data_txt data.txt
Drawing skeleton img (json file as the openpose format):
python draw_keypoints.py --json_path json_files_path --save_path save_path
Paired setting: In the paired setting, we have ground truth images.
python -u test_SDAFNet_viton.py -b 8 --name TEST_PAIR --mode test --dataset_list VITON/test_pairs.txt
Unpaired setting: In the unpaired setting, we do not have ground truth images. The test_unpairs.txt
is same with CPVTON-plus and PFAFN
python -u test_SDAFNet_viton.py -b 8 --name TEST_UNPAIR --mode test --dataset_list VITON/test_unpairs.txt
python train_SDAFNet_viton.py -b 8 --name VITON
- VITON dataset
Download the checkpoint, which gets higher scores than reported in paper.
Paired setting. SSIM: 0.8539 using the pytorch SSIM repo. https://github.com/Po-Hsun-Su/pytorch-ssim
UnPaired setting. FID: 10.55 using the implementation to evaluate. https://github.com/toshas/torch-fidelity
- VITON dataset
- MPV dataset
- FashionVideo
- ShapeNet
Our code references the implementation of ClotFlow and PFAPN, including the feature extractors, feature pyramid networks (FPN) , and the design of the cascaded structure. Thanks for their awesome works.
The use of this code is RESTRICTED to non-commercial research and educational purposes.