VIDEVAL
A MATLAB implementation of feature fused VIDeo quality EVALuator (VIDEVAL) proposed in [IEEE TIP2021] UGC-VQA: Benchmarking blind video quality assessment for user generated content. IEEEXplore
Check out our BVQA resource list and performance benchmark/leaderboard results in https://github.com/tu184044109/BVQA_Benchmark.
码云Gitee mirror: https://gitee.com/zhengzhong-tu/VIDEVAL_release
The recommended system is Linux, than Windows. Mac is not supported though due to FRIQUEE issue.
Should you find any problems, please feel free to send an issue or email me.
Updates
- [10-21-2021] All the features I used in the paper can be downloaded here: Google Drive
- [10-21-2021] Added the code for calibrating dataset MOSs. Check
inlsa/
for more details. 🐛 [12-17-2020] Mac system is not supported since there was an error on calling themex
files here. It also means that the FRIQUEE model cannot be run on Mac too. Thanks to @CXMANDTXW for finding this in issue.✨ [12-20-2020] A light versionVIDEVAL_light
was provided as a speed-up version of vanilla VIDEVAL (scales better for high resolution and high fps), albeit the performance may drop. Please check Performances for the performance-speed tradeoff. Check Demos for the running of light VIDEVAL.🐛 [04-22-2021] Fixed nan bug by using nanmean(). Thanks to @Sissuire.
Performances
SRCC / PLCC
VIDEVAL means the original VIDEVAL in demo_compute_VIDEVAL_feats.m
. It operates on the original frame resolution sampled at every second frame.
VIDEVAL_light_{res}s_{fps}fps is the light version of VIDEVAL where input video is spatially downscaled to {res} at a frame sampling rate of {fps} fps. Check demo_compute_VIDEVAL_light_feats.m
. Note that speed-up parameters [max_reso,frs_per_blk]
can be played with for specific application scenarios.
Methods | KoNViD-1k | LIVE-VQC | YouTube-UGC | All-Combined |
---|---|---|---|---|
VIDEVAL | 0.7832 / 0.7803 | 0.7522 / 0.7514 | 0.7787 / 0.7733 | 0.7960 / 0.7939 |
VIDEVAL_light_720s_6fps | 0.7510 / 0.7510 | 0.7429 / 0.7453 | 0.7514 / 0.7477 | 0.7621 / 0.7689 |
VIDEVAL_light_720s_3fps | 0.7492 / 0.7508 | 0.7174 / 0.7225 | 0.7445 / 0.7413 | 0.7584 / 0.7666 |
VIDEVAL_light_480s_3fps | 0.7281 / 0.7338 | 0.7144 / 0.7209 | 0.7140 / 0.7134 | 0.7462 / 0.7537 |
Speed
Speed was evaluated on the whole calc_VIDEVAL_feats.m
function. The unit is average secs/video
.
Methods | 540p | 720p | 1080p | 4k@60 | scalability |
---|---|---|---|---|---|
VIDEVAL | 61.9 | 146.5 | 354.5 | 1716.3 | |
VIDEVAL_light_720s_6fps | 29.9 | 68.2 | 72.6 | 205.2 | |
VIDEVAL_light_720s_3fps | 15.7 | 33.6 | 40.9 | 115.9 | |
VIDEVAL_light_480s_3fps | 12.2 | 16.6 | 20.4 | 77.9 |
Note:
- 540p: 540p@30fps@8sec in KoNViD-1k
- 720p: 720p@30fps@10sec in LIVE-VQC
- 1080p: 1080p@30fps@10sec in LIVE-VQC
- 4k@60: 4k@60fps@20s in YouTube-UGC
Our empirical observations
- Aggressive spatial downsampling will harm the performance on spatially-dominated datasets, KoNViD-1k, YouTube-UGC.
- Increasing frame sampling rate benefits the performance on temporal-distorted or motion-intensive videos, as those in LIVE-VQC.
Installation
[Note] Recommended system is Linux. Windows MATLAB users may suffer from
WARNING: You should compile the MEX version of "*.c"
and was slower from our tests.
- MATLAB
- FFmpeg
- python3
- sklearn
Demos
Feature Extraction Only
demo_compute_VIDEVAL_feats.m
or light version:
demo_compute_VIDEVAL_light_feats.m
You need to specify the parameters
Quality Prediction with Pre-trained Model
This pre-trained model was trained on the combined dataset.
You need first extract features:
demo_compute_VIDEVAL_feats.m
or light version:
demo_compute_VIDEVAL_light_feats.m
Then run:
demo_pred_MOS_pretrained_VIDEVAL.py
or light version:
demo_pred_MOS_pretrained_VIDEVAL_light.py
Evaluation of BVQA Model on One Dataset
demo_eval_BVQA_feats_one_dataset.py
You need to specify the parameters
Evaluation of BVQA Model on All-Combined Dataset
demo_eval_BVQA_feats_all_combined.py
You need to specify the parameters
Citation
If you use this code for your research, please cite our papers.
@article{tu2021ugc,
title={UGC-VQA: Benchmarking blind video quality assessment for user generated content},
author={Tu, Zhengzhong and Wang, Yilin and Birkbeck, Neil and Adsumilli, Balu and Bovik, Alan C},
journal={IEEE Transactions on Image Processing},
year={2021},
publisher={IEEE}
}
Contact
Zhengzhong TU, [email protected]