Factorized multi-scale multi-resolution residual network for single image deraining

摘要

The performance of vision systems can be affected when used in severe weather conditions such as heavy rain or snow. Rain streak removal is an ill posed problem as they can vary in shape, size, and density across the image. In this paper, a single-image deraining network named Factorized Multi-scale Multi-resolution Residual Network (FMMRNet), which follows a U-Net backbone, is proposed. As rain streaks affect non-local regions of the image, larger receptive fields are beneficial to capture these non-local dependencies. We propose the use of multi-scale grouped convolutions to integrate information from global and local scales. In the proposed FMMRNet, multi-scale convolutions are factorized so that they can have large effective kernel sizes while reducing the computational complexity. During training, intermediate multi-resolution outputs are produced for loss computation which improves gradient flow in the deeper layers of the network and promotes better learning. A channel-wise attention mechanism is included to recalibrate feature maps before fusion instead of direct concatenation at each stage of the decoder. A higher-level reconstruction loss called perceptual loss is included for effective training to improve the visual quality of the derained images. The performance of the proposed FMMRNet is quantitatively and qualitatively compared on public benchmark datasets, and it outperforms the state-of-the-art deraining methods. Furthermore, to show the practical applicability of the proposed network, we demonstrate that when FMMRNet is used as a preprocessing step for object-detection methods such as Faster-RCNN and YOLO, it improves their performance on images degraded by rain streaks by 12.6% and 75.2% respectively.