Improving response capability of chatbot using twitter

  • Seong-Soo Jeong
  • Yeong-Seok SeoEmail author
Original Research


One of the most familiar services to smartphone users is mobile messenger. At present, such messenger services are rapidly developing into chatbots, including links with commercial platforms to pursue customer service. A chatbot is not a person but a virtual conversation partner or a conversational robot that can respond to questions through a chatting interface according to predefined response rules in a widely used messaging platform. A chatbot usually has a knowledge database to have conversations with users and provides smart responses by applying various algorithms. However, because it mostly only uses limited dialogue information pre-stored in the knowledge database, the chatbot cannot provide flexible answers when it receives an input question that does not exist in the knowledge database. Therefore, this study proposes a novel technique that can provide accurate and flexible answers to the users of chatbot services by expanding the knowledge database through the use of Twitter, a popular social networking service. To this end, a keyword matching-based answer retrieval technique was studied to provide appropriate answers in terms of context based on a method that can collect the data of dialogues between users from Twitter. To verify the effectiveness of the proposed technique, a comparative analysis with three commercial chatbots, currently used by many people, was performed. Ten evaluators were recruited and the response appropriateness of each chatbot was evaluated with a variety of question sets. As a result, it was concluded that the proposed technique provided the most appropriate answers in terms of context.


Chatbot Twitter Social network Social networking service Human computer interaction JSON Apache Lucene 



This work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (No. NRF-2017R1C1B5018295). Also, this work was supported by the 2017 Yeungnam University Research Grant.


  1. Abdul-Kader SA, Woods J (2015) Survey on chatbot design techniques in speech conversation systems. Int J Adv Comput Sci Appl 6(7):72–80Google Scholar
  2. AbuShawar B, Atwell E (2015) ALICE chatbot trials and outputs. Comput y Sist 19(4):625–632Google Scholar
  3. Alicebot (2019) Free AIML chat bot content. Accessed 25 Feb 2019
  4. Al-Zubaide H, Issa AA (2011) Ontbot: ontology based chatbot. In: Proc. International symposium on innovations in information and communications technology, 7–12Google Scholar
  5. Amazon Alexa (2019) Ways to build with alexa. Accessed 25 Feb 2019
  6. Antoniou G, Harmelen F (2008) A semantic web primer. The MIT Press, CambridgeGoogle Scholar
  7. Apple (2019) Siri does more than ever. Even before you ask. Accessed 25 Feb 2019
  8. Bellegarda JR (2014) Spoken language understanding for natural interaction: the Siri experience. Natural Interaction with Robots, Knowbots and Smartphones. Springer-Verlag, New York, pp 3–14CrossRefGoogle Scholar
  9. Berners-Lee T, Hendler J, Lassila O (2001) The semantic web. Sci Am 284(5):28–37CrossRefGoogle Scholar
  10. Carpenter R, Freeman J (2005) Computing machinery and the individual: the personal turing test. Accessed 25 Feb 2019
  11. Chu S, Zhu K (2018) Designing a vibrotactile reading system for mobile phones. J Inform Process Syst 14(5):1102–1113Google Scholar
  12. D’silva GM, Thakare S, More S, Kuriakose J (2017) Real world smart chatbot for customer care using a software as a service (SaaS) architecture. In: Proc. 2017 international conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud). 658–664Google Scholar
  13. Facebook for developers (2019) Messenger platform. Accessed 25 Feb 2019
  14. Google Assistant (2019) Discover what your assistant can do. Accessed 25 Feb 2019
  15. Goyal P, Pandey S, Jain K (2018) Deep learning for natural language processing. Apress, BerkeleyCrossRefGoogle Scholar
  16. Guynn J (2016) Zuckerberg’s facebook messenger launches ‘chat bots’ platform. Accessed 25 Feb 2019
  17. Hardeniya N (2015) NLTK essentials. Packt Publishing Ltd, BirminghamGoogle Scholar
  18. Huang J, Efthimiadis EN (2009) Analyzing and evaluating query reformulation strategies in web search logs. In: Proc. 18th ACM Conf. Information and knowledge management, 77–86Google Scholar
  19. Huh JH (2018) Server operation and virtualization to save energy and cost in future sustainable computing. Sustainability 10(6):1919CrossRefGoogle Scholar
  20. Huh JH, Kim TJ (2019) A location-based mobile health care facility search system for senior citizens. J Supercomput 75(4):1831–1848CrossRefGoogle Scholar
  21. Huh JH, Seo K (2017) An indoor location-based control system using bluetooth beacons for IoT systems. Sensors 17(12):2917CrossRefGoogle Scholar
  22. Huh JH, Seo K (2019) Blockchain-based mobile fingerprint verification and automatic log-in platform for future computing. J Supercomput 75(6):3123–3139CrossRefGoogle Scholar
  23. Joseph W (1966) ELIZA-a computer program for the study of natural language communication between man and machine. Commun ACM 9(1):36–45CrossRefGoogle Scholar
  24. JSON (2019) Introducing JSON. Accessed 25 Feb 2019
  25. Kang J (2018) Mobile payment in Fintech environment: trends, security challenges, and services. Hum-Cent Comput Inform Sci 8(32):1–16Google Scholar
  26. Kim SK, Kim UM, Huh JH (2019) A study on improvement of blockchain application to overcome vulnerability of IoT multiplatform security. Energies 12(3):402CrossRefGoogle Scholar
  27. Lamont I (2016) Twitter In 30 minutes: How to connect with interesting people, write great tweets, and find information that's relevant to you. 3rd edn. i30 Media Corporation, MA, USAGoogle Scholar
  28. Lee SH, Roh JH, Kim SH, Jin SH (2018) Feature subset for improving accuracy of keystroke dynamics on mobile environment. J Inform Process Syst 14(2):523–538Google Scholar
  29. Li J, Galley M, Brockett C, Spithourakis GP, Gao J, Dolan B (2016) A persona-based neural conversation model. In: Proc. the 54th annual meeting of the association for computational linguistics. 994–1003Google Scholar
  30. Lokman AS, Zain JM (2010) Extension and prerequisite: an algorithm to enable relations between responses in chatbot technology. J Comput Sci 6(10):1212–1218CrossRefGoogle Scholar
  31. Lopez G, Quesada L, Guerrero LA (2017) Alexa vs. Siri vs. Cortana vs. Google Assistant: a comparison of speech-based natural user interfaces. In: Proc. International conference on applied human factors and ergonomics. 241–250Google Scholar
  32. Lu Z, Li H (2013) A deep architecture for matching short texts. In: Proceedings of the 26th international conference on neural information processing systems, pp 1367–1375Google Scholar
  33. Lucene (2019) Welcome to Apache Lucene. Accessed 25 Feb 2019
  34. Lundqvist KO, Pursey G, Williams S (2013) Design and implementation of conversational agents for harvesting feedback in eLearning systems. In: Proc. European conference on technology enhanced learning, 617–618Google Scholar
  35. McNeal ML, Newyear D (2013) Introducing chatbots in libraries. Accessed 25 Feb 2019
  36. McTear M, Callejas Z, Griol D (2016) The conversational interface: talking to smart devices. Springer International Publishing AG, Cham, SwitzerlandCrossRefGoogle Scholar
  37. Microsoft Cortana (2019) Cortana. Your intelligent assistant across your life. Accessed 25 Feb 2019
  38. Nair G, Johnson S, Sathya V (2018) Chatbot as a personal assistant. Int J Appl Eng Res 13(20):14644–14649Google Scholar
  39. Ng E, Mohan V (2015) Lucene 4 cookbook. Packt Publishing Ltd, BirminghamGoogle Scholar
  40. Pestanes P, Gautier B (2017) The rise of intelligent voice assistants: new gadget for your living room or window of opportunity to reshuffle the cards in the web economy?. Accessed 25 Feb 2019
  41. Pilato G, Augello A, Vassallo G, Gaglio S (2007) Sub-symbolic semantic layer in cyc for intuitive chat-bots. In: Proc. International conference on semantic computing, 121–128Google Scholar
  42. Robertson SE, Zaragoza H, Taylor MJ (2004) Simple BM25 extension to multiple weighted fields. In: Proc. the thirteenth ACM international conference on information and knowledge management, 42–49Google Scholar
  43. Sarikaya R (2017) The technology behind personal digital assistants—an overview of the system architecture and key components. IEEE Signal Process Mag 34(1):67–81CrossRefGoogle Scholar
  44. Setiaji B, Wibowo FW (2016) Chatbot using a knowledge in database. In: Proc. 2016 7th international conference on intelligent systems, modelling and simulation, 72–77Google Scholar
  45. Sharma PK, Ryu JH, Park KY, Park JH, Park JH (2018) Li-Fi based on security cloud framework for future IT environment. Hum-Cent Comput Inform Sci 8(23):1–13Google Scholar
  46. Sheth B (2016) Once-in-a-decade paradigm shift: messaging. Accessed 25 Feb 2019
  47. Shim Y, Lee H, Fomin V (2019) What benefits couldn’t ‘Joyn’ enjoy?: the changing role of standards in the competition in mobile instant messengers in Korea. Technol Forecast Soc Chang 139:125–134CrossRefGoogle Scholar
  48. Statista (2019) Number of Twitter users worldwide from 2014 to 2020 (in millions). Accessed 25 Feb 2019
  49. Strzalkowski T, Harabagiu S (2008) Advances in open-domain question-answering. Springer-Verlag, BerlinCrossRefGoogle Scholar
  50. Sundstrom S (2017) Should you pay attention to Chatbots? Probably. Accessed 25 Feb 2019
  51. Szabo G, Polatkan G, Boykin PO, Chalkiopoulos A (2018) Social media data mining and analytics. Wiley, IN, USAGoogle Scholar
  52. Tweepy (2019) An easy-to-use Python library for accessing the twitter API. Accessed 25 Feb 2019
  53. Twitter (2019) It’s what’s happening. Accessed 25 Feb 2019
  54. Wallace RS (2009) The Anatomy of A.L.I.C.E., Parsing the turing test. Springer, Dordrecht, NetherlandsGoogle Scholar
  55. Yamaguchi H, Mozgovoy M, Danielewicz-Betz A (2018) A chatbot based on AIML rules extracted from twitter dialogues. In: Proc. 2018 federated conference on computer science and information systems, 1–6Google Scholar
  56. Yan R, Song Y, Wu H (2016a) Learning to respond with deep neural networks for retrieval-based human-computer conversation system. In: Proc. the 39th international ACM SIGIR conference on research and development in information retrieval, 55–64Google Scholar
  57. Yan Z, Duan N, Bao J, Chen P, Zhou M, Li Z, Zhou J (2016b) DocChat: an information retrieval approach for chatbot engines using unstructured documents. In: Proc. the 54th annual meeting of the association for computational linguistics, 516–525Google Scholar
  58. Yih WT, He X, Meek C (2014) Semantic parsing for single-relation question answering. In: Proceedings of the 52nd annual meeting of the association for computational linguistics, pp 643–648Google Scholar
  59. Yorke-Smith N, Saadati S, Myers KL, Morley DN (2012) The design of a proactive personal agent for task management. Int J Artif Intell Tools 21(01):1250004CrossRefGoogle Scholar
  60. Yu Z, Xu Z, Black AW, Rudnicky AI (2016) Chatbot evaluation and database expansion via crowdsourcing. In: Proc. the RE-WOCHAT workshop of LREC, 1–5Google Scholar
  61. Zhou A, Jia M, Yao M (2017) Business of bots: How to grow your company through conversation. Topbots Inc., NY, USAGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer EngineeringYeungnam UniversityGyeongsanRepublic of Korea

Personalised recommendations