Learning lightweight Multi-Scale Feedback Residual network for single image super-resolution

摘要

In the past years, convolutional neural networks (CNNs) have demonstrated great success for single image super-resolution (SISR). However, existing CNNs for SISR generally have two limitations: (1) the network depth is very deep, which not only weakens the information flow from bottom to top but also has a heavy model capacity; (2) the network architectures are often feed-forward, which prevent the previous layers capturing the useful information from the following layers, limiting the feature learning capability. To address these issues, this paper presents a lightweight Multi-scale Feedback Residual network for SISR. Specifically, we design a lightweight feedback-based recurrent neural network (FRNN) tailored to SISR. The FRNN is consists of a series of recursive Densely-Connected Blocks (DCBs) with the Low-Resolution (LR) image features and the output of the former DCB as inputs. Each DCB adaptively fuses multi-level features from the side-output intermediate feature maps to generate a powerful feature representation. Meanwhile, the DCB cascades a set of Multi-scale Residual Blocks (MRBs), each of which has an enlarged field of view to fully capture multi-scale context information. Moreover, the MRB has a novel Multi-Kernel Fusion Block (MKFB) design, which can dynamically adjust the receptive field size of the output feature representation based on the multi-scale inputs. The whole network of our MFRSR is lightweight with only ∼4.5M parameters, but achieves favorable performance on five benchmark datasets compared to the state-of-the-art methods in terms of PSNR and SSIM.