A Bayesian evaluation framework for subjectively annotated visual recognition tasks

作者：

Highlights：

• Addresses problems in current evaluation procedures for subjectively annotated problems in computer vision.

• Proposes a framework for evaluation of any black box predictor’s epistemic uncertainty through Bayesian modeling of PY^(Y^|Y).

• Applies the framework to 4 practical use cases of subjectively annotated tasks where humans provide subjective annotations.

摘要

•Addresses problems in current evaluation procedures for subjectively annotated problems in computer vision.•Proposes a framework for evaluation of any black box predictor’s epistemic uncertainty through Bayesian modeling of PY^(Y^|Y).•Applies the framework to 4 practical use cases of subjectively annotated tasks where humans provide subjective annotations.

论文关键词：Uncertainty estimation,Epistemic uncertainty,Supervised learning,Bayesian inference,Bayesian modeling

论文评审过程：Received 11 February 2021, Revised 27 August 2021, Accepted 19 October 2021, Available online 28 October 2021, Version of Record 12 November 2021.

论文官网地址：https://doi.org/10.1016/j.patcog.2021.108395