A Subjective Quality Study for Video Frame Interpolation

Abstract

Video frame interpolation (VFI) is one of the fundamental research areas in video processing. While the development of novel interpolation algorithms has been widely researched, there is an important topic which has not been sufficiently investigated. This is the quality assessment of interpolated content. In this paper, we have conducted a subjective study for VFI based on a newly developed video database, BVI-VFI, which contains 36 reference at three different frame rates and 180 distorted videos generated using five conventional and learning based VFI algorithms. Subjective opinion scores have been collected from 60 human participants, and then employed to evaluate eight popular quality metrics, including PSNR, SSIM and LPIPS which are commonly used for assessing VFI methods. The results indicate that none of these metrics provide acceptable correlation with the perceived quality on interpolated content, with the best-performing metric, LPIPS, offering a SROCC value below 0.6. Our findings show that there is an urgent need to develop a bespoke perceptual quality metric for VFI.

Links

***Note: there is a newer and larger version of the BVI-VFI database available [here]. If you fill in the form below, the full database will be shared.***

[BVI-VFI database] (Please fill the registration form to get access to the download link. After the form is submitted, you should be able to access the dataset within 2 working days.)
[paper]

Sample frames from BVI-VFI

Evaluation results of selected metrics

See the quantitative results in the paper.

Copyright disclaimer

The sequences contained in the BVI-VFI dataset are obtained from the BVI-HFR dataset (https://data.bris.ac.uk/data/dataset/k8bfn0qsj9fs1rwnc2x75z6t7). This database has been compiled by the University of Bristol, Bristol, UK. All intellectual property rights remain with the University of Bristol. The dataset should only be used for academic purpose. This copyright and permission notice shall be duplicated whenever the data is copied. The University of Bristol makes no warranties with respect to the material and expressly disclaims any warranties regarding its fitness for any purpose. Unless the above conditions are agreed to by the recipient, no permission is granted for any use and copying of the data. By using the database and sequences, the user agrees to the conditions of this copyright and disclaimer.

Citation

@inproceedings{danier2022subjective,
    title={A subjective quality study for video frame interpolation},
    author={Danier, Duolikun and Zhang, Fan and Bull, David},
    booktitle={2022 IEEE International Conference on Image Processing (ICIP)},
    pages={1361--1365},
    year={2022},
    organization={IEEE}
}