PASSRnet: Parallax Attention Stereo Super-Resolution Network
Pytorch implementation of "Learning Parallax Attention for Stereo Image Super-Resolution", CVPR 2019
Overview
Figure 1. Overview of our PASSRnet network.
Figure 2. Illustration of our parallax-attention mechanism.
Figure 3. A toy example illustration of the parallax-attention and cycle-attention maps generated by our PAM. The attention maps (30ร30) correspond to the regions (1ร30) marked by a yellow stroke. In (a) and (b), the first row represents left/right stereo images, the second row stands for parallax-attention maps, and the last row represents cycle-attention maps.
Flickr1024 Dataset
Figure 4. The Flickr1024 dataset.
Requirements
- pytorch (0.4), torchvision (0.2) (Note: The code is tested with
python=3.6, cuda=9.0
) - Matlab (For training/test data generation)
Train
Prepare training data
- Download the Flickr1024 dataset and put the images in
data/train/Flickr1024
(Note: In our paper, we also use 60 images in the Middlebury dataset as the training set.) - Cd to
data/train
and rungenerate_trainset.m
to generate training data.
Begin to train
python train.py --scale_factor 4 --device cuda:0 --batch_size 32 --n_epochs 80 --n_steps 30
Test
Prepare test data
- Download the KITTI2012 dataset and put folders
testing/colored_0
andtesting/colored_1
indata/test/KITTI2012/original
- Cd to
data/test
and rungenerate_testset.m
to generate test data. - (optional) You can also download KITTI2015, Middlebury or other stereo datasets and prepare test data in
data/test
as below:
data
โโโ test
โโโ dataset_1
โโโ hr
โโโ scene_1
โโโ hr0.png
โโโ hr1.png
โโโ ...
โโโ scene_M
โโโ lr_x4
โโโ scene_1
โโโ lr0.png
โโโ lr1.png
โโโ ...
โโโ scene_M
โโโ ...
โโโ dataset_N
Demo
python demo_test.py --scale_factor 4 --device cuda:0 --dataset KITTI2012
Results
Figure 5. Visual comparison for 2ร SR. These results are achieved on โtest_image_013โ of the KITTI 2012 dataset and โtest_image_019โ of the KITTI 2015 dataset.
Figure 6. Visual comparison for 4ร SR. These results are achieved on โtest_image_004โ of the KITTI 2015 dataset.
Figure 7. Visual comparison for 2ร SR. These results are achieved on a stereo image pair acquired in our laboratory.
Citation
@InProceedings{Wang2019Learning,
author = {Longguang Wang and Yingqian Wang and Zhengfa Liang and Zaiping Lin and Jungang Yang and Wei An and Yulan Guo},
title = {Learning Parallax Attention for Stereo Image Super-Resolution},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2019},
}
Contact
For questions, please send an email to [email protected]