A comprehensive examination of the relation of three citation-based journal metrics to expert judgment of journal quality

作者：

Highlights：

• Using the journal rating system produced by the Excellence in Research for Australia (ERA) exercise, we examine the relationship over a set of more than 10,000 journals categorized into 27 subject areas.

• We analyze the relationship from the dimensions of correlation, distribution of the metrics over the rating tiers, and ROC analysis.

• Our results show that SNIP consistently has stronger agreement with the ERA rating, followed by RIP and then JIF along every dimension measured.

• Our results suggest that SNIP may be a better choice than RIP or JIF in evaluation of journal quality in situations where agreement with expert judgment is an important consideration.

摘要

•In this paper we study the relation between the three journal metrics source normalized impact per paper (SNIP), raw impact per paper (RIP) and Journal Impact Factor (JIF) and human expert judgement.•Using the journal rating system produced by the Excellence in Research for Australia (ERA) exercise, we examine the relationship over a set of more than 10,000 journals categorized into 27 subject areas.•We analyze the relationship from the dimensions of correlation, distribution of the metrics over the rating tiers, and ROC analysis.•Our results show that SNIP consistently has stronger agreement with the ERA rating, followed by RIP and then JIF along every dimension measured.•Our results suggest that SNIP may be a better choice than RIP or JIF in evaluation of journal quality in situations where agreement with expert judgment is an important consideration.

论文关键词：Citation analysis,Peer review,Source normalized impact per paper,Raw impact per paper,Journal impact factor,ERA

论文评审过程：Received 12 August 2015, Revised 15 December 2015, Accepted 15 December 2015, Available online 15 January 2016, Version of Record 15 January 2016.

论文官网地址：https://doi.org/10.1016/j.joi.2015.12.005