Abstract
Conversational agents have gained increasing popularity over the last decade in a variety of personal, public, and occupational settings due to rapid advances of artificial intelligence (AI) and natural language processing (NLP). However, how users can interact with such technologies is still understudied. The objective of this study was to investigate type of conversation (presence and absence of continuous conversation) and task complexity (high vs. low) on usability metrics (i.e., task completion time, number of queries used in completing tasks, and perceived system usability) with conversational agents in smart home environments. Eighteen participants joined this study and completed required tasks. The results showed that there was a significant effect of type of conversation on task completion time and number of queries per task. Task complexity significantly extended task completion time and increased number of queries per task. The results may help with the design of more usable conversational agents.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sciuto A, Saini A, Forlizzi J, Hong JI (eds) (2018) Hey Alexa, what’s up?: a mixed-methods studies of in-home conversational agent usage. In: Proceedings of the 2018 on designing interactive systems conference 2018. ACM
Toxtli C, Cranshaw J (eds) (2018) Understanding chatbot-mediated task management. In: Proceedings of the 2018 CHI conference on human factors in computing systems. ACM
Shum H-Y, He X-D, Li DJ, Fo IT (2018) From Eliza to XiaoIce: challenges and opportunities with social chatbots. Electron Eng 19(1):10–26
Kopp S, Gesellensetter L, Krämer NC, Wachsmuth I (eds) (2005) A conversational agent as museum guide–design and evaluation of a real-world application. Intelligent virtual agents. Springer
López G, Quesada L, Guerrero LA (2018) Alexa versus Siri versus Cortana versus google assistant: a comparison of speech-based natural user interfaces. Advances in Human Factors and Systems Interaction. Advances in Intelligent Systems and Computing, pp 241–50
McLoughlin I, Sharifzadeh HR (2008) Speech recognition for smart homes. InTech, Speech Recognition
Ortiz CL (2014) The road to natural conversational speech interfaces. IEEE Internet Computing 18(2):74–78
Luger E, Sellen A (eds) (2016) Like having a really bad PA: the gulf between user expectation and experience of conversational agents. In: Proceedings of the 2016 CHI conference on human factors in computing systems. ACM
Kaufmann E, Bernstein A (2010) Evaluating the usability of natural language query languages and interfaces to Semantic Web knowledge bases. J Web Semant 8(4):377–393
Ghosh S, Pherwani J (eds) (2015) Designing of a natural voice assistants for mobile through user centered design approach. Springer International Publishing, Cham
Lewis JR, Sauro J (eds) (2009) The factor structure of the system usability scale. First International Conference on Human Centered Design, Held as Part of HCI International, San Diego, CA, USA, July 19–24, 2009
Shawar BA, Atwell ES (2007) Chatbots: are they really useful? J Lang Technol and Comput Linguist 22(1):29–49
Compliance with Ethical Standards
The study was approved by the Logistics Department for Civilian Ethics Committee of Alibaba Group. All subjects who participated in the experiment were provided with and signed an informed consent form. All relevant ethical safeguards have been met with regard to subject protection.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Guo, J., Tao, D., Yang, C. (2020). The Effects of Continuous Conversation and Task Complexity on Usability of an AI-Based Conversational Agent in Smart Home Environments. In: Long, S., Dhillon, B. (eds) Man–Machine–Environment System Engineering . MMESE 2019. Lecture Notes in Electrical Engineering, vol 576. Springer, Singapore. https://doi.org/10.1007/978-981-13-8779-1_79
Download citation
DOI: https://doi.org/10.1007/978-981-13-8779-1_79
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-8778-4
Online ISBN: 978-981-13-8779-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)