How humans obtain information from AI: Categorizing user messages in human-AI collaborative conversations

作者：

Highlights：

• We use a real-world customer service log, which contains three kinds of conversations: human-human conversation, human-AI conversation, human-human conversation with AI assist.

• We have concluded a categorization system for user messages in human-AI collaborative conversations based on users’ information needs after a thorough examination of a real-world customer service log. The responses included both human and AI chatbot outputs.

• We summarize and report the characteristics of different message types and compare their usage in sessions with only human, AI, or both representatives.

摘要

•We use a real-world customer service log, which contains three kinds of conversations: human-human conversation, human-AI conversation, human-human conversation with AI assist.•We have concluded a categorization system for user messages in human-AI collaborative conversations based on users’ information needs after a thorough examination of a real-world customer service log. The responses included both human and AI chatbot outputs.•We summarize and report the characteristics of different message types and compare their usage in sessions with only human, AI, or both representatives.

论文关键词：Human-AI collaboration,Conversational agent,Informational conversation,Customer service

论文评审过程：Received 12 August 2021, Revised 6 November 2021, Accepted 21 November 2021, Available online 20 December 2021, Version of Record 20 December 2021.

论文官网地址：https://doi.org/10.1016/j.ipm.2021.102838