Certainty driven consistency loss on multi-teacher networks for semi-supervised learning

Highlights：

• The prediction of the unlabeled data given by the teacher model can be noisy and unreliable.

• We present a certainty-driven consistency loss (CCL) that exploits the uncertainty of the model predictions for the consistency regularization.

• Our Filtering CCL enforces consistency on the reliable targets by filtering out uncertain predictions.

• Our Temperature CCL reduces the magnitudes of gradients on the uncertain targets.

• We further introduce a decoupled multi-teacher framework to encourage the teacher provide additional new knowledge for the student.