Effect of verbal comprehension skill and self-reported features on reliability of crowdsourced relevance judgments

作者：

Highlights：

• Exploring the reliability of using crowdsourcing to create relevance judgments.

• Investigating effects of cognitive skills on the reliability of judgments.

• Investigating effects of cognitive skills on system rankings.

• Investigating effects of self-reported competence on the reliability of judgments.

摘要

•Exploring the reliability of using crowdsourcing to create relevance judgments.•Investigating effects of cognitive skills on the reliability of judgments.•Investigating effects of cognitive skills on system rankings.•Investigating effects of self-reported competence on the reliability of judgments.

论文关键词：Information retrieval evaluation,Crowdsourcing,Verbal comprehension,Relevance judgment,Reliability

论文评审过程：Received 22 July 2015, Revised 6 July 2016, Accepted 27 July 2016, Available online 8 August 2016, Version of Record 8 August 2016.

论文官网地址：https://doi.org/10.1016/j.chb.2016.07.058