Depth image super-resolution using correlation-controlled color guidance and multi-scale symmetric network

Highlights：

• We develop an effective symmetric unit (SU) with the ability of residual learning to reconstruct edge details and restore edge sharpness.

• We use chains of SUs to construct a multi-scale symmetric network architecture (MSSNet) with dense color guidance to progressively up-sample depth images.

• A novel structure called correlation-controlled color guidance block (CCGB) is introduced by investigating the inter-channel correlation between depth inference network and color guidance network to improve the color guidance accuracy.

• We integrate the MSSNet and the CCGB into a unified framework to effectively resolve the problem of depth image super resolution.

摘要

•We develop an effective symmetric unit (SU) with the ability of residual learning to reconstruct edge details and restore edge sharpness.•We use chains of SUs to construct a multi-scale symmetric network architecture (MSSNet) with dense color guidance to progressively up-sample depth images.•A novel structure called correlation-controlled color guidance block (CCGB) is introduced by investigating the inter-channel correlation between depth inference network and color guidance network to improve the color guidance accuracy.•We integrate the MSSNet and the CCGB into a unified framework to effectively resolve the problem of depth image super resolution.

论文关键词：Depth image super-resolution,Deep convolutional neural network,Encoder-decoder structure,Color guidance,Channel correlation

论文评审过程：Received 11 July 2019, Revised 24 May 2020, Accepted 22 June 2020, Available online 24 June 2020, Version of Record 29 June 2020.