Separate or joint? Estimation of multiple labels from crowdsourced annotations
作者：
Highlights：
• We propose estimating multiple true labels for multi-labeled instances.
• We flexibly incorporate label dependency into the label-generation process.
• It is effective to simultaneously estimate the states of strongly related labels.
• Reliable results are estimated using the opinions of a few crowdsourcing workers.
• Our models can reduce the cost of multi-label data collection for AI techniques.
摘要
•We propose estimating multiple true labels for multi-labeled instances.•We flexibly incorporate label dependency into the label-generation process.•It is effective to simultaneously estimate the states of strongly related labels.•Reliable results are estimated using the opinions of a few crowdsourcing workers.•Our models can reduce the cost of multi-label data collection for AI techniques.
论文关键词：Multi-label estimation,Crowdsourced annotation,Label dependency,Quality control,Human computation
论文评审过程：Available online 13 April 2014.
论文官网地址：https://doi.org/10.1016/j.eswa.2014.03.048
