Do systems pass university entrance exams?
作者：
Highlights：
• We describe a Question Answering task over real exams to enter University.
• We define an evaluation methodology that can be reused for developing new systems.
• We include an in-depth analysis of results from different points of view.
• We show that current technologies are unable to pass the proposed exams.
• We identify the most difficult questions and offer future directions.
摘要
•We describe a Question Answering task over real exams to enter University.•We define an evaluation methodology that can be reused for developing new systems.•We include an in-depth analysis of results from different points of view.•We show that current technologies are unable to pass the proposed exams.•We identify the most difficult questions and offer future directions.
论文关键词：Question answering,Reading comprehension,Semantic understanding
论文评审过程：Received 27 July 2017, Revised 2 March 2018, Accepted 6 March 2018, Available online 27 March 2018, Version of Record 27 March 2018.
论文官网地址：https://doi.org/10.1016/j.ipm.2018.03.002
