FloLPIPS: A Bespoke Video Quality Metric for Frame Interpoation

Abstract

Video frame interpolation (VFI) serves as a useful tool for many video processing applications. Recently, it has also been applied in the video compression domain for enhancing both conventional video codecs and learning-based compression architectures. While there has been a surge in works on developing effective frame interpolation algorithms, the perceptual quality assessment of interpolated content remains an open field of research. In this paper, we present a bespoke full reference video quality metric for VFI, FloLPIPS, which is based on a popular perceptual image quality metric, LPIPS, which captures the perceptual degradation in extracted image feature space. In order to enhance the performance of LPIPS on evaluating frame interpolated videos, we re-designed the spatial feature aggregation step in LPIPS by using the temporal distortion (through comparing optical flows) to weight the feature difference maps in LPIPS. Evaluated on the BVI-VFI database, which contains 180 test sequences with various frame interpolation artefacts, FloLPIPS shows superior correlation performance (with statistical significance) with subjective ground truth over 12 popular quality assessors.

Links

Model

Results

Citation

@inproceedings{danier2022flolpips,
    title={Flolpips: A bespoke video quality metric for frame interpolation},
    author={Danier, Duolikun and Zhang, Fan and Bull, David},
    booktitle={2022 Picture Coding Symposium (PCS)},
    pages={283--287},
    year={2022},
    organization={IEEE}
}