Ascertain Quality Attributes for Design and Development of New Improved Chatbots to Assess Customer Satisfaction Index (CSI): A Preliminary Study

  • Nurul Muizzah Johari
  • Halimah Badioze ZamanEmail author
  • Puteri N. E. Nohuddin
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11870)


Chatbots are artificial intelligence applications that are used as tools to communicate and assist humans in any task designed. It uses knowledge that has been provided by the developer and continue to learn on its own through a Natural Language Processing (NLP) approach. This paper highlights a study that aims to investigate quality attributes for a new improved Chatbots to assess customer satisfaction. The preliminary study was conducted to acquire prior understanding on the characteristics and functionalities capable of Chatbots to capture potential customer satisfaction in the tourism domain before a prototype is developed. The findings from this study reveal seven (7) plausible dimensions with several sub-factors of quality attributes that can be applied to new improved Chatbots. These dimensions and sub-factors are useful inputs to the Systems Requirement Specifications (SRS) for the design and development of new improved Chatbots to assess Customer Satisfaction Index (CSI).


Chatbots Quality attributes Customer Satisfaction Index (CSI) 


  1. 1.
    Ranoliya, B.R., Raghuwanshi, N., Singh, S.: Chatbot for University Related FAQs (2017)Google Scholar
  2. 2.
    Xu, A., Liu, Z., Guo, Y., Sinha, V., Akkiraju, R.: A new chatbot for customer service on social media. In: Proceedings of the ACM Conference on Human Factors in Computing Systems (2017)Google Scholar
  3. 3.
    Delacroix, L.: Longman Advanced American Dictionary. Pearson Education, Edinburgh (2007)Google Scholar
  4. 4.
    Kowcika, A., Gupta, A., Sondhi, K., Shivhre, N., Kumar, R.: Sentiment analysis for social media. Int. J. Adv. Res. Comput. Sci. Softw. Eng. 3(1), 35–56 (2013)Google Scholar
  5. 5.
    Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach. Prentice Hall Press, Upper Saddle River (2009)zbMATHGoogle Scholar
  6. 6.
    Rogers, E.M.: Diffusion of Innovations. Free Press, New York (1995)Google Scholar
  7. 7.
    Peters, H., Vogel, M.: Machine Learning Research Progress. Nova Science, New York (2010)Google Scholar
  8. 8.
    Kashyap, P.: Machine Learning for Decision Makers: Cognitive Computing. Apress, Bangalore (2017)CrossRefGoogle Scholar
  9. 9.
    Aube, T., TechCruch: No UI is the new UI (2015). Accessed 23 Aug 2016
  10. 10.
    Peirson-Smith, T.: In the age of Artificial Intelligence, can we rise to the challenge of mass unemployment? (2017). Accessed 09 May 2018
  11. 11.
    Radziwill, N.M., Benton, M.C.: Evaluating quality of chatbots and intelligent conversational agents. arXiv preprint arXiv:1704.04579 (2017)
  12. 12.
    Rahman, A.M., Al Mamun, A., Islam, A.: Programming challenges of chatbot: current and future prospective. In: 2017 IEEE Region 10 Humanitarian Technology Conference (Project assistant0-HTC), pp. 75–78 (2017)Google Scholar
  13. 13.
    Ray, G., Muhanna, W.A., Barney, J.B.: Information technology and the performance of the customer service process: a resource-based analysis. MIS Q. 29(4), 625–652 (2005)CrossRefGoogle Scholar
  14. 14.
    Shah, K.B., Shetty, M.S., Shah, D.P., Pamnani, R.: Approaches towards building a banking assistant. Int. J. Comput. Appl. 166(11), 1–6 (2017)Google Scholar
  15. 15.
    Shawar, B.A., Atwell, E.: Using corpora in machine-learning chatbot systems. Int. J. Corpus Linguist. 10(4), 489–516 (2005)CrossRefGoogle Scholar
  16. 16.
    Shawar, B.A., Atwell, E.: Chatbots: are they really useful? In: Ldv Forum, vol. 22, no. 1, pp. 29–49 (2007)Google Scholar
  17. 17.
    Singh, A., Shree, R.: Recognition of natural language processing to manage digital electronic applications. Int. J. Adv. Res. Comput. Sci. 8(5), 1641–1643 (2017)Google Scholar
  18. 18.
    BAM & Conant, P.: Chatbots 101: Practical insights on business of Bots (2016). Accessed 20 Aug 2016
  19. 19.
    Beer, A.: Let’s Chat! Designing methods for designing Chatbots (2015). Accessed 9 Aug 2016
  20. 20.
    BI Intelligence and Business Insider UK. Messaging apps are now bigger than social networks (2016).
  21. 21.
    Charlier, M.: The O’Reilly Design Podcast: designing for the IoT, design’s responsibility, and the importance of team dynamics. Interview with Mary Tressler for Accessed 23 June 2016
  22. 22.
    Connolly, E., Intercom: Principles of Bot design: Inside Intercom. Accessed 29 Aug 2016
  23. 23.
    Debecker, A., VenturaBeat: Three stats that show Chatbots are here to stay. Accessed 29 Aug 2016
  24. 24.
    Engadet and Low, C.: Google’s ‘assistant’ is at the core of its new hardware. Accessed 5 Sept 2016
  25. 25.
    Huet, E., Bloomberg: The humans hiding behind the Chatbots. Accessed 29 Aug 2016
  26. 26.
    Huge Inc., van Hoof, P.: The designer’s AI Study guide. Accessed 18 Aug 2016
  27. 27.
    McCarth, J.J., Stanford University: What is Artificial Intelligence? Accessed 5 Oct 2016
  28. 28.
    Kojouharov, S., VentureBeat: Five Tips for building the next great Chatbot. Accessed 15 Aug 2016
  29. 29.
    Livingston, T.: Bots are better without conversation. Accessed 20 Aug 2016Google Scholar
  30. 30.
    Livingston, T.: The future of Chat isn’t AI. Accessed 20 Aug 2016Google Scholar
  31. 31.
    Mielke, C., Smashing Magazine: Conversational interfaces: Where are we heading? Accessed 20 Aug 2016
  32. 32.
    Mose, P., Deloitte: Technical trends 2016: Innovating in the digital era. Accessed 8 Sept 2016
  33. 33.
  34. 34.
    Mortensen, D.: How we are building our AIs with Machine Learning. Accessed 5 Sept 2016
  35. 35.
    Mortensen, D., Understanding the Facebook and Microsoft Chatbot revolution. Accessed 5 Sept 2016
  36. 36.
    Nicholson, C., Gibson, A.: Artificial Intelligence, machine learning and deep learning Deeplearning4j: Open-source, distributed deep learning for the JVM. Accessed 5 Oct 2016
  37. 37.
    Neilson, J.: 10 Heuristics for user interface design: Article by Neilson. Accessed 20 Aug 2016
  38. 38.
    Aishwarya, B., Vijay, G., Sachin, D.: Chatbot for business organization: utility, processes and significance. Int. J. Sci. Res. Comput. Sci. Eng. Inf. Technol. 2(6), 872–881 (2017)Google Scholar
  39. 39.
    Collumn, A., Costea, C., Joyeux, D., Hasan, O., Brunie, L.: A study and comparison of sentiment analysis methods for reputation evaluation. Rapport de recherche RR-LIRIS-2014-002 (2014)Google Scholar
  40. 40.
    Hill, J., Ford, W.R., Farreras, I.G.: Real conversations with artificial intelligence: a comparison between human-human online conversations and human-chatbot conversations. Computer. Hum. Behav. 49, 245–250 (2015)CrossRefGoogle Scholar
  41. 41.
    Hippner, H., Rentzmann, R.: Text mining. Informatik Spektrum 29, 287–290 (2006)CrossRefGoogle Scholar
  42. 42.
    Kowcika, A., Gupta, A., Sondhi, K., Shivhre, N., Kumar, R.: Sentiment analysis for social media. Int. J. Adv. Res. Comput. Sci. Softw. Eng. (2013)Google Scholar
  43. 43.
    Maruti TechLab: How to develop a chatbot from scratch. Accessed 9 May 2017
  44. 44.
    MyTourismData: (2019)
  45. 45.
    Nica, I., Tazl, O.A., Wotawa, F.: Chatbot-based tourist recommendations using model-based reasoning. In: Proceedings of the 20th International Configuration Workshop, pp. 27–28, September 2018Google Scholar
  46. 46.
    Poliakova, A.: Application of the customer satisfaction index (CSI) to transport services. Perner’s Contact 5(4), 208–215 (2010)Google Scholar
  47. 47.
    Xu, A., Liu, Z., Guo, Y., Sinha, V., Akkiraju, R.: A new chatbot for customer service on social media. In: Proceeding of the ACM Conference on Human Factors in Computing Systems (2017)Google Scholar
  48. 48.
    Zaman, H.B.: Simbiosis Seni, Sains dan Teknologi Berasingan ke Multimedia-fusion. Penerbit Universiti Kebengsaan Malaysia (UKM), Bangi (2009)Google Scholar
  49. 49.
    Radziwill, N., Benton, M.C.: Neurodiversity secrets for innovation and design. SXSW Interactive, Austin TX (2017)Google Scholar
  50. 50.
    Neff, G., Nagy, P.: Automation, algorithms, and politics—talking to bots: symbiotic agency and the case of tay. Int. J. Commun. 10, 17 (2016)Google Scholar
  51. 51.
    Goh, O.S., Ardil, C., Wong, W., Fung, C.C.: A black-box approach for response quality evaluation of conversational agent systems. Int. J. Comput. Intell. 3(3), 195–203 (2007)Google Scholar
  52. 52.
    Vetter, M.: Quality aspects of bots. In: Meyerhoff, D., Laibarra, B., van der Pouw Kraan, R., Wallet, A. (eds.) Software Quality and Software Testing in Internet Times, pp. 165–184. Springer, Berlin (2002). Scholar
  53. 53.
    Košir, D.: Implementacija in testiranje klepetalnika. Doctoral dissertation, Univerza v Ljubljani (2013)Google Scholar
  54. 54.
    Coniam, D.: The linguistic accuracy of chatbots: usability from an ESL perspective. Text Talk 34(5), 545–567 (2014)CrossRefGoogle Scholar
  55. 55.
    Meira, M.O., Canuto, A.M.P.: Evaluation of emotional agents’ architectures: an approach based on quality metrics and the influence of emotions on users. In: Proceedings of the World Congress on Engineering, vol. 1 (2015)Google Scholar
  56. 56.
    Kaleem, M., Alobadi, O., O’Shea, J., Crockett, K.: Framework for the formulation of metrics for conversational agent evaluation. In: RE-WOCHAT: Workshop on Collecting and Generating Resources for Chatbots and Conversational Agents-Development and Evaluation Workshop Programme, 28 May 2016, p. 20 (2016)Google Scholar
  57. 57.
    Abran, A., Khelifi, A., Suryn, W., Seffah, A.: Consolidating the ISO usability models. In: Proceedings of 11th International Software Quality Management Conference, vol. 2003, pp. 23–25 (2003)Google Scholar
  58. 58.
    Saaty, T.L.: Decision Making for Leaders: The Analytic Hierarchy Process for Decisions in a Complex World. RWS Publications, Pittsburgh (1990)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Nurul Muizzah Johari
    • 1
  • Halimah Badioze Zaman
    • 1
    Email author
  • Puteri N. E. Nohuddin
    • 1
  1. 1.Institute of Visual InformaticsUniversiti Kebangsaan MalaysiaBangiMalaysia

Personalised recommendations