BVI-VFI: A Video Quality Database for Video Frame Interpolation

Abstract

Video frame interpolation (VFI) is a fundamental research topic in video processing, which is currently attracting increased attention across the research community. While the development of more advanced VFI algorithms has been extensively researched, there remains little understanding of how humans perceive the quality of interpolated content and how well existing objective quality assessment methods perform when measuring the perceived quality. In order to narrow this research gap, we have developed a new video quality database named BVI-VFI, which contains 540 distorted sequences generated by applying five commonly used VFI algorithms to 36 diverse source videos with various spatial resolutions and frame rates. We collected more than 10,800 quality ratings for these videos through a large scale subjective study involving 189 human subjects. Based on the collected subjective scores, we further analysed the influence of VFI algorithms and frame rates on the perceptual quality of interpolated videos. Moreover, we benchmarked the performance of 33 classic and state-of-the-art objective image/video quality metrics on the new database, and demonstrated the urgent requirement for more accurate bespoke quality assessment methods for VFI.

TL;DR

• A video database with subjective quality labels was collected.
• Analysis on quality labels reveals that
• Recent deep learning-based VFI methods have a clear advantage over simple frame repeating and averating, but the performance gap narrows at frame rate increases.
• Simple frame repeating and averating can outperform DL methods on dynamic textures (rapid and irregular motion).
• Motion magnitude and complexity jointly impacts VFI difficulty.
• Higher spatial resolution also makes VFI more challenging.
• 33 existing image/video quality assessment methods were benchmarked, and best performance (SRCC=0.7) was achieved by FAST.
• There is a need for a better quality evaluator for VFI, and the proposed database can be used for development and benchmarking of them.

Links

Please fill the [registration form] to get access to the download link. I'll then share the download link ASAP.
Please check Github link for more detailed description of the dataset.
[paper] [arXiv] [Github]

Sample frames from BVI-VFI

Evaluation results of selected metrics

See more experimental results in the paper.

Copyright disclaimer

The sequences contained in the BVI-VFI dataset are obtained from various sources, including (1) BVI-HFR dataset (https://data.bris.ac.uk/data/dataset/k8bfn0qsj9fs1rwnc2x75z6t7). (2) LIVE-YT-HFR dataset (https://live.ece.utexas.edu/research/LIVE_YT_HFR/LIVE_YT_HFR/index.html). This database has been compiled by the University of Bristol, Bristol, UK. All intellectual property rights remain with the University of Bristol. The dataset should only be used for academic purpose. This copyright and permission notice shall be duplicated whenever the data is copied. The University of Bristol makes no warranties with respect to the material and expressly disclaims any warranties regarding its fitness for any purpose. Unless the above conditions are agreed to by the recipient, no permission is granted for any use and copying of the data. By using the database and sequences, the user agrees to the conditions of this copyright and disclaimer.

Citation

@article{danier2023bvi,
    title={BVI-VFI: A Video Quality Database for Video Frame Interpolation},
    author={Danier, Duolikun and Zhang, Fan and Bull, David R},
    journal={IEEE Transactions on Image Processing},
    year={2023},
    publisher={IEEE}
}
@inproceedings{danier2022subjective,
    title={A subjective quality study for video frame interpolation},
    author={Danier, Duolikun and Zhang, Fan and Bull, David},
    booktitle={2022 IEEE International Conference on Image Processing (ICIP)},
    pages={1361--1365},
    year={2022},
    organization={IEEE}
}