Hyperspectral-Image-Super-Resolution-Benchmark

A list of hyperspectral image super-resolution resources collected by Junjun Jiang. If you find that important resources are not included, please feel free to contact me.

Hyperspectral image super-resolution is a kind of technique that can generate a high spatial and high spectral resolution image from one of the following observed data (1) low-resolution multispectral image, e.g., LR RGB image, (2) high-resolution multispectral image, e.g., HR RGB image or other 2D measurement, (3) low-resolution hyperspectral image, or (4) high-resolution multispectral image and low-resolution hyperspectral image. According to kind of observed data, hyperspectral image super-resolution techniques can be divided into four classes: spatiospectral super-resolution (SSSR), spectral super-resolution (SSR), single hyperspectral image super-resolution (SHSR), and multispectral image and hyperspectral image fusion (MHF). Note that we take hyperspectral image reconstruction from 2D measurement as a class of SSR.

========================================================================

0. Pioneer Work and Technique Review

• Unmixing based multisensor multiresolution image fusion, TGRS1999, B. Zhukov et al.

• Application of the stochastic mixing model to hyperspectral resolution enhancement, TGRS2004, M. T. Eismann et al.

• Resolution enhancement of hyperspectral imagery using maximum a posteriori estimation with a stochastic mixing model, Ph.D. dissertation, 2004, M. T. Eismann et al.

• MAP estimation for hyperspectral image resolution enhancement using an auxiliary sensor, TIP2004, R. C. Hardie et al.

• Hyperspectral resolution enhancement using high-resolution multispectral imagery with arbitrary response functions, TGRS2005, M. T. Eismann et al.

• Hyperspectral pansharpening: a review. GRSM2015, L. Loncan et al. [PDF] [Code]

• Hyperspectral and multispectral data fusion: A comparative review of the recent literature, GRSM2017, N. Yokoya,et al. [PDF] [Code]

• A Survey of Hyperspectral Image Super-Resolution Technology, IGARSS2021, ML Zhang et al. [PDF]

• Recent Advances and New Guidelines on Hyperspectral and Multispectral Image Fusion, Information Fusion2021, RW Dian, et al. [PDF]

========================================================================

1. SpatioSpectral Super-Resolution (SSSR)

• Spatial and spectral joint super-resolution using convolutional neural network, TGRS 2020, S. Mei et al.
• 【Our work】Multi-task Interaction learning for Spatiospectral Image Super-Resolution, Q. Ma et al. submitted to IEEE TIP, in peer review.
• 【Our work】Deep Unfolding Network for Spatiospectral Image Super-Resolution, Q. Ma et al. IEEE TCI 2022. [Code]
• Ponet: A universal physical optimization-based spectral super-resolution network for arbitrary multispectral images. Information Fusion 2022. J He et al.

2. Spectral Super-Resolution (SSR)

• NTIRE 2018 Challenge on Spectral Reconstruction from RGB Images, CVPRW 2018, Boaz Arad et al.

• NTIRE 2020 Challenge on Spectral Reconstruction from an RGB Image, CVPRW 2020, Boaz Arad et al.

• NTIRE 2022 Challenge on Stereo Image Super-Resolution: Methods and Results, CVPRW2022, L Wang et al.

• MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction, CVPRW 2022, Y. Cai et al. (Winner of NTIRE 2022 Challenge on Spectral Reconstruction from RGB)[PDF][Code]

• HDNet: High-resolution Dual-domain Learning for Spectral Compressive Imaging, CVPR 2022, Y. Cai et al. [PDF][Code]

• Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction, CVPR 2022, Y. Cai et al. [PDF][Code]

• HASIC-Net: Hybrid Attentional Convolutional Neural Network With Structure Information Consistency for Spectral Super-Resolution of RGB Images, TGRS 2022, J LI et al.

• Semisupervised spectral degradation constrained network for spectral super-resolution, GRSL 2022, W Chen, et al.

• A spectral–spatial jointed spectral super-resolution and its application to hj-1a satellite images, GRSL 2022, X Han, et al.

• DRCR Net: Dense Residual Channel Re-calibration Network with Non-local Purification for Spectral Super Resolution, CVPRW 2022, JJ LI, et al.

• DsTer: A dense spectral transformer for remote sensing spectral super-resolution, International Journal of Applied Earth Observation and Geoinformation 2022, J He, et al.

========================================================================

3. Single Hyperspectral Image Super-Resolution (SHSR)

• Super-resolution reconstruction of hyperspectral images, TIP2005, T. Akgun et al.

• Enhanced self-training superresolution mapping technique for hyperspectral imagery, GRSL2011, F. A. Mianji et al.

• A super-resolution reconstruction algorithm for hyperspectral images. Signal Process. 2012, H. Zhang et al.

• Super-resolution hyperspectral imaging with unknown blurring by low-rank and group-sparse modeling, ICIP2014, H. Huang et al.

• Super-resolution mapping via multi-dictionary based sparse representation, ICASSP2016, H. Huang et al.

• Super-resolution: An efficient method to improve spatial resolution of hyperspectral images, IGARSS2016, A. Villa, J. Chanussot et al.

• Hyperspectral image super resolution reconstruction with a joint spectral-spatial sub-pixel mapping model, IGARSS2016, X. Xu et al.

• Hyperspectral image super-resolution by spectral mixture analysis and spatial–spectral group sparsity, GRSL2016, J. Li et al.

• Super-resolution reconstruction of hyperspectral images via low rank tensor modeling and total variation regularization, IGARSS2016, S. He et al. [PDF]

• Hyperspectral image super-resolution by spectral difference learning and spatial error correction, GRSL2017, J. Hu et al.

• Super-Resolution for Remote Sensing Images via Local–Global Combined Network, GRSL2017, J. Hu et al.

• Hyperspectral image superresolution by transfer learning, Jstars2017, Y. Yuan et al. [PDF]

• Hyperspectral image super-resolution using deep convolutional neural network, Neurocomputing, 2017, Sen Lei et al. [PDF]

• Hyperspectral image super-resolution via nonlocal low-rank tensor approximation and total variation regularization, Remote Sensing, 2017, Yao Wang et al. [PDF]

• Hyperspectral Image Spatial Super-Resolution via 3D Full Convolutional Neural Network, Remote Sensing, 2017, Saohui Mei et al. [PDF] [Code]

• A MAP-Based Approach for Hyperspectral Imagery Super-Resolution, TIP2018, Hasan Irmak et al.

• Single Hyperspectral Image Super-resolution with Grouped Deep Recursive Residual Network, BigMM2018, Yong Li et al. [PDF] [Code]

• Hyperspectral image super-resolution with spectral–spatial network, IJRS2018, Jinrang Jia et al. [PDF]

• Separable-spectral convolution and inception network for hyperspectral image super-resolution, IJMLC 2019, Ke Zheng et al.

• Hyperspectral Image Super-Resolution Using Deep Feature Matrix Factorization, IEEE TGRS 2019, Weiying Xie et al. [PDF]

• Deep Hyperspectral Prior Single-Image Denoising, Inpainting, Super-Resolution, ICCVW2019, Oleksii Sidorov et al. [PDF]

• Spatial-Spectral Residual Network for Hyperspectral Image Super-Resolution, arXiv2020, Qi Wang et al. [PDF]

• CNN-Based Super-Resolution of Hyperspectral Images, IEEE TGRS 2020, P. V. Arun et al. [PDF]

• Hyperspectral Image Super-Resolution via Intrafusion Network, IEEE TGRS 2020, Jing Hu et al. [PDF]

• Mixed 2D/3D Convolutional Network for Hyperspectral Image Super-Resolution, Remote Sensing 2020, Qiang Li et al. [Code][Pdf]

• Hyperspectral Image Super-Resolution by Band Attention Through Adversarial Learning, IEEE TGRS 2020, Jiaojiao Li et al. [Pdf]

• 【Our work】Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery, IEEE TCI 2020, Junjun Jiang et al. [Code][Pdf] It achieves state-of-the-art performance for Single Hyperspectral Image Super-Resolution (SHSR) task

• Bidirectional 3D Quasi-Recurrent Neural Networkfor Hyperspectral Image Super-Resolution, IEEE JStars 2021, Ying Fu et al. [Web][Pdf]

• Hyperspectral Image Super-Resolution Using Spectrum and Feature Context, IEEE TIM 2021, Qi Wang et al. [Web][Pdf]

• Hyperspectral Image Super-Resolution with Spectral Mixup and Heterogeneous Datasets, arXiv2021, Ke Li et al. [Pdf]

• A Spectral Grouping and Attention-Driven Residual Dense Network for Hyperspectral Image Super-Resolution, IEEE TGRS 2021, Denghong Liu et al. [Web][Pdf]

• Spatial-Spectral Feedback Network for Super-Resolution of Hyperspectral Imagery, arXiv 2021, Enhai Liu et al. [Web][Pdf]

• Exploring the Relationship Between 2D/3D Convolution for Hyperspectral Image Super-Resolution, IEEE TGRS 2021, Qi Wang et al. [Web][Pdf]

• Hyperspectral Image Super-Resolution via Recurrent Feedback Embedding and Spatial-Spectral Consistency Regularization, IEEE RGS 2021, Xinya Wang et al. [Pdf]

• Hyperspectral Image Super-Resolution Using Spectrum and Feature Context, IEEE TIM 2021, Qi Wang et al. [Web][Pdf]

• Dilated projection correction network based on autoencoder for hyperspectral image super-resolution, Neural Networks 2022, X. Wang et al.

• Hyperspectral Image Super-Resolution with RGB Image Super-Resolution as an Auxiliary Task, WACV 2022, K Li et al. [PDF] [Code]

• 【Our work】From Less to More: Spectral Splitting and Aggregation Network for Hyperspectral Face Super-Resolution, CVPRW 2022, JJ Jiang, et al. [PDF]

• Interactformer: Interactive Transformer and CNN for Hyperspectral Image Super-Resolution, TGRS 2022, Y Liu. [PDF]

• Multiple Frame Splicing and Degradation Learning for Hyperspectral Imagery Super-Resolution, IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING 2022, C Deng, et al. [PDF]

• A Group-based Embedding Learning and Integration Network for Hyperspectral Image Super-resolution, TGRS 2022, X Wang, et al. [PDF]

• Hyperspectral image super-resolution using cluster-based deep convolutional networks, Signal Processing: Image Communication 2022, C Zou, et al. [PDF]

• Learning Deep Resonant Prior for Hyperspectral Image Super-Resolution, TGRS 2022, Z Gong, et al. [PDF]

• GJTD-LR: A Trainable Grouped Joint Tensor Dictionary With Low-Rank Prior for Single Hyperspectral Image Super-Resolution, TGRS 2022, C Liu, rt al. [PDF]

========================================================================

4. Multispectral and Hyperspectral Image Fusion (MHF)

1) Bayesian based approaches

• Blind Image Fusion for Hyperspectral Imaging with the Directional Total Variation, Inverse Problems, 2018, Leon Bungert et al. [PDF] [Code]

• Bayesian sparse representation for hyperspectral image super resolution, CVPR2015, N. Akhtar et al. [PDF] [Code]

• Hysure: A convex formulation for hyperspectral image superresolution via subspace-based regularization, TGRS2015, M. Simoes et al. [PDF] [Code]

• Hyperspectral and multispectral image fusion based on a sparse representation, TGRS2015, Q. Wei et al. [PDF] [Code]

• Bayesian fusion of multi-band images, Jstar2015, W. Qi et al. [PDF] [Code]

• Noise-resistant wavelet-based Bayesian fusion of multispectral and hyperspectral images, TGRS2009, Y. Zhang et al. [PDF]

• Weighted Low-rank Tensor Recovery for Hyperspectral Image Restoration, arXiv2018, Yi Chang et al. [PDF]

• Enhanced Hyperspectral Image Super-Resolution via RGB Fusion and TV-TV Minimization, ICIP 2021, Marija Vella et al. [PDF][Code]

2) Tensor based approaches

• Hyperspectral image superresolution via non-local sparse tensor factorization, CVPR2017, R. Dian et al. [PDF]

• Spatial–Spectral-Graph-Regularized Low-Rank Tensor Decomposition for Multispectral and Hyperspectral Image Fusion, Jstars2018, K. Zhang et al. [PDF]

• Fusing Hyperspectral and Multispectral Images via Coupled Sparse Tensor Factorization, TIP2108, S. Li et al. [PDF] [Code]

• Hyperspectral Super-Resolution: A Coupled Tensor Factorization Approach, arXiv2018, Charilaos I. Kanatsoulis et al. [PDF]

• Nonlocal Patch Tensor Sparse Representation for Hyperspectral Image Super-Resolution, TIP2019, Yang Xu et al. [PDF] [Web]

• Learning a Low Tensor-Train Rank Representation for Hyperspectral Image Super-Resolution, TNNLS2019, Renwei Dian et al. [PDF] [Web]

• Nonnegative and Nonlocal Sparse Tensor Factorization-Based Hyperspectral Image Super-Resolution, IEEE TGRS2020, Wei Wan et al. [PDF]

• Nonlocal Coupled Tensor CP Decomposition for Hyperspectral and Multispectral Image Fusion, IEEE TGRS2020, Xu Yang et al. [PDF]

• Hyperspectral Super-Resolution via Coupled Tensor Ring Factorization, IEEE TGRS2020, Wei He et al. [PDF]

• Spatial-Spectral Structured Sparse Low-Rank Representation for Hyperspectral Image Super-Resolution, IEEE TIP2021, Jize Xue et al., [PDF]

• Hyperspectral Images Super-Resolution via Learning High-Order Coupled Tensor Ring Representation, IEEE TNNLS 2020, Y. Xu et al. [Pdf]

• Hyperspectral Image Superresolution Using Global Gradient Sparse and Nonlocal Low-Rank Tensor Decomposition With Hyper-Laplacian Prior, IEEE JStars 2021, Y. Peng et al. [Pdf]

• Hyperspectral Image Superresolution via Structure-Tensor-Based Image Matting, IEEE JStars 2021, H. Gao et al. [Pdf]

• Hyperspectral super-resolution via coupled tensor ring factorization, PR 2022, W He, et al. [PDF] [Code]

• Coupled Tensor Block Term Decomposition with Superpixel-Based Graph Laplacian Regularization for Hyperspectral Super-Resolution, RS 2022, H Liu, et al. [PDF]

3) Matrix factorization based approaches

• High-resolution hyperspectral imaging via matrix factorization, CVPR2011, R. Kawakami et al. [PDF] [Code]

• Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion, TGRS2012, N. Yokoya et al. [PDF] [Code]

• Sparse spatio-spectral representation for hyperspectral image super-resolution, ECCV2014, N. Akhtar et al. [PDF] [Code]

• Hyper-sharpening: A first approach on SIM-GA data, Jstars2015, M. Selva et al.

• Hyperspectral super-resolution by coupled spectral unmixing, ICCV2015, C Lanaras. [PDF] [Code]

• RGB-guided hyperspectral image upsampling, CVPR2015, H. Kwon et al. [PDF] [Code]

• Multiband image fusion based on spectral unmixing, TGRS2016, Q. Wei et al. [PDF] [Code]

• Hyperspectral image super-resolution via non-negative structured sparse representation, TIP2016, W. Dong, et al. [PDF] [Code]

• Hyperspectral super-resolution of locally low rank images from complementary multisource data, TIP2016, M. A. Veganzones et al. [PDF]

• Multispectral and hyperspectral image fusion based on group spectral embedding and low-rank factorization, TGRS2017, K. Zhang et al.

• Hyperspectral Image Super-Resolution Based on Spatial and Spectral Correlation Fusion, TRGS2018, C. Yi et al.

• Self-Similarity Constrained Sparse Representation for Hyperspectral Image Super-Resolution, TIP2108, X. Han et al.

• Exploiting Clustering Manifold Structure for Hyperspectral Imagery Super-Resolution, TIP2018, L. Zhang et al. [Code]

• Hyperspectral Image Super-Resolution With a Mosaic RGB Image, TIP2018, Y. Fu et al. [PDF]

• Fusing Hyperspectral and Multispectral Images via Coupled Sparse Tensor Factorization, TIP2018, S. Li et al. [PDF][Code]

• Multispectral Image Super-Resolution via RGB Image Fusion and Radiometric Calibration, TIP2019, Zhi-Wei Pan et al. [PDF] [Web]

• Hyperspectral Image Super-resolution via Subspace-Based Low Tensor Multi-Rank Regularization, TIP2019, Renwei Dian et al. [PDF]

• Hyperspectral Image Super-Resolution With Optimized RGB Guidance, Ying Fu et al., CVPR2019. [PDF]

• Super-Resolution for Hyperspectral and Multispectral Image Fusion Accounting for Seasonal Spectral Variability, TIP2020, R.A. Borsoi et al. [PDF]

• A Truncated Matrix Decomposition for Hyperspectral Image Super-Resolution, TIP2020, Jianjun Liu et al. [PDF]

• Adaptive Nonnegative Sparse Representation for Hyperspectral Image Super-Resolution, IEEE JStars 2021, X. Li et al. [Pdf]

4) Deep Learning based approaches

• Deep Residual Convolutional Neural Network for Hyperspectral Image Super-Resolution, ICIG2017, C. Wang et al. [PDF]

• SSF-CNN: Spatial and Spectral Fusion with CNN for Hyperspectral Image Super-Resolution, ICIP2018, X. Han et al. [PDF]

• Deep Hyperspectral Image Sharpening, TNNLS2018, R. Dian et al. [PDF] [Code]

• HSI-DeNet: Hyperspectral Image Restoration via Convolutional Neural Network, TGRS2018, Y. Chang et al. [Web]

• Unsupervised Sparse Dirichlet-Net for Hyperspectral Image Super-Resolution, CVPR2018, Y. Qu et al. [PDF] [Code]

• Deep Hyperspectral Prior: Denoising, Inpainting, Super-Resolution, arXiv2019, Oleksii Sidorov et al. [PDF] [Code]

• Multi-level and Multi-scale Spatial and Spectral Fusion CNN for Hyperspectral Image Super-resolution, ICCVW 2019, Xianhua Han et al. [PDF]

• Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net, CVPR2019, Xie Qi et al. [PDF] [Web]

• Hyperspectral Image Reconstruction Using Deep External and Internal Learning,ICCV2019, Zhang Tao et al. [PDF] [Web]

• Deep Blind Hyperspectral Image Super-Resolution, IEEE TNNLS 2020, Lei Zhang et al. [Pdf]

• Deep Recursive Network for Hyperspectral Image Super-Resolution, IEEE TCI2020, Wei Wei, et al. [PDF][Web]

• Coupled Convolutional Neural Network With Adaptive Response Function Learning for Unsupervised Hyperspectral Super Resolution, IEEE TGRS 2020, K. Zheng et al. [Pdf]

• Unsupervised Adaptation Learning for Hyperspectral Imagery Super-Resolution, CVPR 2020, L. Zhang et al. [Pdf]

• Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral Super-Resolution, ECCV 2020, J. Yao et al. [Pdf]

• Unsupervised Recurrent Hyperspectral Imagery Super-Resolution Using Pixel-Aware Refinement, IEEE TGRS2021, Wei Wei, et al. [PDF][Web]

• A Band Divide-and-Conquer Multispectral and Hyperspectral Image Fusion Method, IEEE TGRS 2021, Weiwei Sun et al. [Pdf]

• Hyperspectral Image Super-Resolution via Deep Progressive Zero-Centric Residual Learning, IEEE TIP 2021, Zhiyu Zhu et al. [Pdf]

• Hyperspectral Image Super-Resolution via Deep Prior Regularization with Parameter Estimation, IEEE TCSVT 2021, X. Wang et al. [Pdf][Code]

• Hyperspectral Image Super-Resolution with Self-Supervised Spectral-Spatial Residual Network, RS 2021, W. Chen et al. [Pdf]

• Hyperspectral Image Super-Resolution via Deep Spatiospectral Attention Convolutional Neural Networks, IEEE TNNLS 2021, J. Hu et al. [Pdf]

• Model-Guided Deep Hyperspectral Image Super-Resolution, IEEE TIP 2021, W. Dong et al. [Pdf] [Web]

• 【Our work】Learning A 3D-CNN and Transformer Prior for hyperspectral Image Super-Resolution, arXiv 2021, Q. Ma et al. [Pdf] It achieves state-of-the-art performance for Multispectral and Hyperspectral Image Fusion (MHF) task

• Hyperspectral Image Super-resolution with Deep Priors and Degradation Model Inversion, ICASSP 2022, X. Wang et al. [Pdf][Code]

• Model Inspired Autoencoder for Unsupervised Hyperspectral Image Super-Resolution, TGRS 2022, J Liu et al. [Pdf] [Code]

• Fusformer: A Transformer-Based Fusion Network for Hyperspectral Image Super-Resolution, GRSL 2022, J Hu, et al. [Pdf] [Code]

• External-Internal Attention for Hyperspectral Image Super-Resolution, TGRS 2022, Z Guo, et al.

• Model inspired autoencoder for unsupervised hyperspectral image super-resolution, TGRS 2022, J Liu, et al.

• Symmetrical Feature Propagation Network for Hyperspectral Image Super-Resolution, TGRS 2022, Q Li, et al.

• Context-Aware Guided Attention Based Cross-Feedback Dense Network for Hyperspectral Image Super-Resolution, TGRS 2022, W Dong, et al.

5) Simulations registration and super-resolution approaches

• An Integrated Approach to Registration and Fusion of Hyperspectral and Multispectral Images, TRGS 2019, Yuan Zhou et al.

• Deep Blind Hyperspectral Image Fusion, ICCV2019, Wu Wang et al. [PDF]

• Unsupervised and Unregistered Hyperspectral Image Super-Resolution With Mutual Dirichlet-Net, IEEE TGRS 2021, Y. Qu et al. [Pdf]

========================================================================

Databases

• CAVE dataset
• Harvard dataset
• iCVL dataset
• NUS datase
• NTIRE18 dataset
• Chikusei dataset
• Indian Pines, Salinas, KSC et al.

=========================================================================

Image Quality Measurement

• Peak Signal to Noise Ratio (PSNR)
• Root Mean Square Error (RMSE)
• Structural SIMilarity index (SSIM)
• Spectral Angle Mapper (SAM)
• Erreur Relative Globale Adimensionnelle de Synthèse (ERGAS)
• Universal Image Quality Index (UIQI)

Since 2022/5/7