Deep Learning Geometry Compression Artifacts Removal for Video-Based Point Cloud Compression

摘要

Point cloud is an essential format for three-dimensional (3-D) object modelling and interaction in Augmented Reality and Virtual Reality applications. In the current state of the art video-based point cloud compression (V-PCC), a dynamic point cloud is projected onto geometry and attribute videos patch by patch, each represented by its texture, depth, and occupancy map for reconstruction. To deal with occlusion, each patch is projected onto near and far depth fields in the geometry video. Once there are artifacts on the compressed two-dimensional (2-D) geometry video, they would be propagated to the 3-D point-cloud frames. In addition, in the lossy compression, there always exists a tradeoff between the rate of bitstream and distortion. Although some geometry-related methods were proposed to attenuate these artifacts and improve the coding efficiency, the interactive correlation between projected near and far depth fields has been ignored. Moreover, the non-linear representation ability of Convolutional Neural Network has not been fully considered. Therefore, we propose a learning-based approach to remove the geometry artifacts and improve the compressing efficiency. We have the following contributions. We devise a two-step method working on the near and far depth fields decomposed from geometry. The first stage is learning-based Pseudo-Motion Compensation. The second stage exploits the potential of the strong correlations between near and far depth fields. Our proposed algorithm is embedded in the V-PCC reference software. To the best of our knowledge, this is the first learning-based solution of the geometry artifacts removal in V-PCC. The extensive experimental results show that the proposed approach achieves significant gains on geometry artifacts removal and quality improvement of 3-D point-cloud reconstruction compared to the state-of-the-art schemes.