• Stars
    star
    124
  • Rank 288,207 (Top 6 %)
  • Language
    MATLAB
  • License
    MIT License
  • Created over 4 years ago
  • Updated almost 3 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

[IEEE TIP'2021] "UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content", Zhengzhong Tu, Yilin Wang, Neil Birkbeck, Balu Adsumilli, Alan C. Bovik

PWC PWC PWC PWC

VIDEVAL

A MATLAB implementation of feature fused VIDeo quality EVALuator (VIDEVAL) proposed in [IEEE TIP2021] UGC-VQA: Benchmarking blind video quality assessment for user generated content. IEEEXplore

Check out our BVQA resource list and performance benchmark/leaderboard results in https://github.com/tu184044109/BVQA_Benchmark.

็ ไบ‘Gitee mirror: https://gitee.com/zhengzhong-tu/VIDEVAL_release

The recommended system is Linux, than Windows. Mac is not supported though due to FRIQUEE issue.

Should you find any problems, please feel free to send an issue or email me.

Updates

  • [10-21-2021] All the features I used in the paper can be downloaded here: Google Drive
  • [10-21-2021] Added the code for calibrating dataset MOSs. Check inlsa/ for more details.
  • ๐Ÿ› [12-17-2020] Mac system is not supported since there was an error on calling the mex files here. It also means that the FRIQUEE model cannot be run on Mac too. Thanks to @CXMANDTXW for finding this in issue.
  • โœจ [12-20-2020] A light version VIDEVAL_light was provided as a speed-up version of vanilla VIDEVAL (scales better for high resolution and high fps), albeit the performance may drop. Please check Performances for the performance-speed tradeoff. Check Demos for the running of light VIDEVAL.
  • ๐Ÿ› [04-22-2021] Fixed nan bug by using nanmean(). Thanks to @Sissuire.

Performances

SRCC / PLCC

VIDEVAL means the original VIDEVAL in demo_compute_VIDEVAL_feats.m. It operates on the original frame resolution sampled at every second frame. VIDEVAL_light_{res}s_{fps}fps is the light version of VIDEVAL where input video is spatially downscaled to {res} at a frame sampling rate of {fps} fps. Check demo_compute_VIDEVAL_light_feats.m. Note that speed-up parameters [max_reso,frs_per_blk] can be played with for specific application scenarios.

Methods KoNViD-1k LIVE-VQC YouTube-UGC All-Combined
VIDEVAL 0.7832 / 0.7803 0.7522 / 0.7514 0.7787 / 0.7733 0.7960 / 0.7939
VIDEVAL_light_720s_6fps 0.7510 / 0.7510 0.7429 / 0.7453 0.7514 / 0.7477 0.7621 / 0.7689
VIDEVAL_light_720s_3fps 0.7492 / 0.7508 0.7174 / 0.7225 0.7445 / 0.7413 0.7584 / 0.7666
VIDEVAL_light_480s_3fps 0.7281 / 0.7338 0.7144 / 0.7209 0.7140 / 0.7134 0.7462 / 0.7537

Speed

Speed was evaluated on the whole calc_VIDEVAL_feats.m function. The unit is average secs/video.

Methods 540p 720p 1080p 4k@60 scalability
VIDEVAL 61.9 146.5 354.5 1716.3 ๐ŸŒ ๐Ÿคฌ
VIDEVAL_light_720s_6fps 29.9 68.2 72.6 205.2 ๐Ÿš… ๐Ÿ˜…
VIDEVAL_light_720s_3fps 15.7 33.6 40.9 115.9 โœˆ๏ธ ๐Ÿ˜ฒ
VIDEVAL_light_480s_3fps 12.2 16.6 20.4 77.9 ๐Ÿš€ ๐Ÿ˜Š

Note:

  • 540p: 540p@30fps@8sec in KoNViD-1k
  • 720p: 720p@30fps@10sec in LIVE-VQC
  • 1080p: 1080p@30fps@10sec in LIVE-VQC
  • 4k@60: 4k@60fps@20s in YouTube-UGC

Our empirical observations

  • Aggressive spatial downsampling will harm the performance on spatially-dominated datasets, KoNViD-1k, YouTube-UGC.
  • Increasing frame sampling rate benefits the performance on temporal-distorted or motion-intensive videos, as those in LIVE-VQC.

Installation

[Note] Recommended system is Linux. Windows MATLAB users may suffer from WARNING: You should compile the MEX version of "*.c" and was slower from our tests.

  • MATLAB
  • FFmpeg
  • python3
  • sklearn

Demos

Feature Extraction Only

demo_compute_VIDEVAL_feats.m

or light version:

demo_compute_VIDEVAL_light_feats.m

You need to specify the parameters

Quality Prediction with Pre-trained Model

This pre-trained model was trained on the combined dataset.

You need first extract features:

demo_compute_VIDEVAL_feats.m

or light version:

demo_compute_VIDEVAL_light_feats.m

Then run:

demo_pred_MOS_pretrained_VIDEVAL.py

or light version:

demo_pred_MOS_pretrained_VIDEVAL_light.py

Evaluation of BVQA Model on One Dataset

demo_eval_BVQA_feats_one_dataset.py

You need to specify the parameters

Evaluation of BVQA Model on All-Combined Dataset

demo_eval_BVQA_feats_all_combined.py

You need to specify the parameters

Citation

If you use this code for your research, please cite our papers.

@article{tu2021ugc,
  title={UGC-VQA: Benchmarking blind video quality assessment for user generated content},
  author={Tu, Zhengzhong and Wang, Yilin and Birkbeck, Neil and Adsumilli, Balu and Bovik, Alan C},
  journal={IEEE Transactions on Image Processing},
  year={2021},
  publisher={IEEE}
}

Contact

Zhengzhong TU, [email protected]

More Repositories

1

maxim-pytorch

[CVPR 2022 Oral] PyTorch re-implementation for "MAXIM: Multi-Axis MLP for Image Processing", with *training code*. Official Jax repo: https://github.com/google-research/maxim
Python
168
star
2

BVQA_Benchmark

A resource list and performance benchmark for blind video quality assessment (BVQA) models on user-generated content (UGC) datasets. [IEEE TIP'2021] "UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content", Zhengzhong Tu, Yilin Wang, Neil Birkbeck, Balu Adsumilli, Alan C. Bovik
Python
116
star
3

RAPIQUE

[IEEE OJSP'2021] "RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Content", Zhengzhong Tu, Xiangxu Yu, Yilin Wang, Neil Birkbeck, Balu Adsumilli, Alan C. Bovik
MATLAB
48
star
4

COVER

๐Ÿ† [CVPRW 2024] COVER: A Comprehensive Video Quality Evaluator. ๐Ÿฅ‡ Winner solution for Video Quality Assessment Challenge at the 1st AIS 2024 workshop @ CVPR 2024
Python
37
star
5

MAXIM

[CVPR 2022] Unofficial repository for "MAXIM: Multi-Axis MLP for Image Processing". Official repo: https://github.com/google-research/maxim
20
star
6

Temporal_Pooling

Implementation of temporal pooling methods studied in [ICIP'20] A Comparative Evaluation Of Temporal Pooling Methods For Blind Video Quality Assessment
Python
7
star
7

VMEON-pytorch

This is a GitHub copy of [ACM Multimedia'18] End-to-End Blind Quality Assessment of Compressed Videos Using Deep Neural Networks.
Python
4
star
8

DebandingNet

Python
3
star
9

LIVE-YT-Banding-Database

This is the repository for LIVE-YouTube banding database.
2
star
10

zhengzhongtu

Personal Website
TypeScript
1
star