Weakly supervised instance segmentation using multi-prior fusion

摘要

Instance segmentation usually requires mask annotation, which has a much higher labeling cost than bounding box annotation. In recent years, weakly supervised instance segmentation (WSIS) has drawn great attention with only bounding box annotation needed during training. However, bounding box annotation can only provide limited information for the segmentation task. Without pixel-wise fine-grained supervision, predicted segmentation masks usually have blurred boundaries with poor performance. To address the issue caused by weak bounding box annotation, in this paper, two types of prior knowledge, i.e., bounding box tightness prior and contour prior, are employed to guide the mask prediction. For the bounding box tightness prior, we assume that the objects are annotated tightly with bounding boxes. A multi-instance learning (MIL) approach is adopted with the tightness constraint. For the contour prior, we assume that the mask boundaries should align with strong image gradients. We maximize the inner product of the gradients between the mask proposal and the corresponding image region, which is treated as the contour prior constraint. Finally, both prior constraints are combined for separating targets and background. The experiment results on the PASCAL VOC dataset and the augmented dataset demonstrate that our method outperforms most state-of-the-art WSIS methods.