BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond
3 CUHK – SenseTime Joint Lab, The Chinese University of Hong Kong
4 Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences
The BasicVSR framework. BasicVSR is a simple yet efficient baseline for video super-resolution. It consists of only generic components such as residual blocks and optical flow, but outperforms existing state of the arts with fewer parameters and faster inference speed. Its simplicity enables the possibility of serving as a backbone where sophisticated components can be added to it for further improvement.
Highlights
- BasicVSR outperforms existing works with fewer parameters and faster speed. It can serve as a strong baseline for video super-resolution.
- We build upon BasicVSR and propose IconVSR with two novel components. It outperforms our strong BasicVSR with similar speed.
Abstract
Video super-resolution (VSR) approaches tend to have more components than the image counterparts as they need to exploit the additional temporal dimension. Complex designs are not uncommon. In this study, we wish to untangle the knots and reconsider some most essential components for VSR guided by four basic functionalities, i.e., Propagation, Alignment, Aggregation, and Upsampling. By reusing some existing components added with minimal redesigns, we show a succinct pipeline, BasicVSR, that achieves appealing improvements in terms of speed and restoration quality in comparison to many state-of-the-art algorithms. We conduct systematic analysis to explain how such gain can be obtained and discuss the pitfalls. We further show the extensibility of BasicVSR by presenting an information refill mechanism and a coupled propagation scheme to facilitate information aggregation. The BasicVSR and its extension, IconVSR, can serve as strong baselines for future VSR approaches.
Network Details and IconVSR
Left: The propagation branches in BasicVSR and IconVSR; Right: The proposed components in IconVSR. See our paper for more details.
Results
Results of 4x super-resolution: BasicVSR and IconVSR reconstruct finer details when compared to existing state of the arts. With information-refill and coupled propagation, IconVSR further restores sharper edges and more details.
Citation
@InProceedings{chan2021basicvsr,
author = {Chan, Kelvin CK and Wang, Xintao and Yu, Ke and Dong, Chao and Loy, Chen Change},
title = {BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond},
booktitle = {Proceedings of the IEEE conference on computer vision and pattern recognition},
year = {2021}
}
Contact
If you have any question, please contact Kelvin Chan at chan0899@e.ntu.edu.sg.