Dynamic estimation of worker reliability in crowdsourcing for regression tasks: Making it work
作者：
Highlights：
• We propose DER3, a novel approach to estimate worker reliability in crowdsourcing dynamically.
• Constant rater availability is not required.
• A large number of tasks with correct answers known in advance is not required.
• DER3 is specifically suited for regression tasks, but can also be used in different settings.
• DER3 leads to the results of high accuracy and low costs in different application domains.
摘要
•We propose DER3, a novel approach to estimate worker reliability in crowdsourcing dynamically.•Constant rater availability is not required.•A large number of tasks with correct answers known in advance is not required.•DER3 is specifically suited for regression tasks, but can also be used in different settings.•DER3 leads to the results of high accuracy and low costs in different application domains.
论文关键词：Supervised machine learning,Crowdsourcing,Worker reliability,Multi-armed bandits,Emotion recognition
论文评审过程：Available online 24 April 2014.
论文官网地址：https://doi.org/10.1016/j.eswa.2014.04.012
