Putting Chatbots to the Test: Does the User Experience Score Higher with Chatbots Than Websites?

  • Amélie Beriault-Poirier
  • Sandrine Prom TepEmail author
  • Sylvain Sénécal
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 876)


Chatbots are robots that simulate conversation with human users through instant messaging services. Though the technology is innovative and trendy, what kind of user experience do they provide? To answer this, we conducted an exploratory study with chatbots from different sectors of activity on Facebook Messenger. We invited ten participants to complete six precise search tasks using these chatbots and the corresponding websites from each brand. Based on observation and interviews according to a qualitative approach, our study shows that the user experience scores higher with websites than chatbots, while abandonment rate is higher with chatbots, even though they generate more positive emotions. As a conclusion, until artificial intelligence improves the technology, chatbots have some catching up to do as they are do not score higher than websites to fulfill users’ expectations.


Chatbots User experience Online consumer behavior Websites 


  1. 1.
    Dabholkar, P.A.: Consumer evaluations of new technology-based self-service options: an investigation of alternative models of service quality. Int. J. Res. Mark. 13(1), 29–51 (1996)CrossRefGoogle Scholar
  2. 2.
    Lund, A.M.: Measuring usability with the USE questionnaire. Usability Interface 8(2), 3–6 (2001)Google Scholar
  3. 3.
    Vasileios, T., Moridis, C.N., Economides, A.A.: Measuring instant emotions based on facial expressions during computer-based assessment. Pers. Ubiquitous Comput. 17(1), 43–52 (2013)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Amélie Beriault-Poirier
    • 1
  • Sandrine Prom Tep
    • 2
    Email author
  • Sylvain Sénécal
    • 3
  1. 1.imarklabMontréalCanada
  2. 2.ESG Business SchoolUniversity of Quebec in MontrealMontréalCanada
  3. 3.HEC MontrealMontréalCanada

Personalised recommendations