Effectiveness of Social Media Sentiment Analysis Tools with the Support of Emoticon/Emoji

  • Duncan C. Peacock
  • Habib Ullah KhanEmail author
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 800)


Organizations are increasingly interested in using microblogging platforms, such as Twitter, to get rapid feedback in several domains using sentiment analysis algorithms to rate, for example, whether a target audience is happy or unhappy. However, posts on microblogging platforms can differ from the source material used to train the sentiment analysis tools. For example, emojis and emoticons are increasingly employed in social media to clarify, enhance, or sometimes reverse the sentiment of a post but can be stripped out of a piece of text before it is processed. Responding to this interest, many sentiment analysis algorithms are being made available as web services, but as details of the algorithms used are not always published on the website, comparisons between web services and how well they deal with the peculiarities of microblogging posts can be difficult. To address this, a prototype web application was developed to compare the performance of nine tweet-related sentiment analysis web services and, through targeted hypotheses, to study the effect of emojis and emoticons on polarity classification. Twelve specific research test sets were created with the application, labelled by volunteers, and tested against the analysis web services with evaluation provided by two- and three-class accuracy measures. Distinct differences were found in how the web services used emoticons and emojis in assigning a positive or negative sentiment value to a tweet, with some services seeming to ignore their presence. It was found in general that web services classified polarity sensitive tweets significantly less accurately than tweets where the sentiment of the emoji/emoticon supported the sentiment of the text.


Social media Sentiment analysis Emoticon/emoji 


  1. 1.
    Liu, B.: Sentiment Analysis and Opinion Mining. Morgan & Claypool (Synthesis lectures on human language technologies, 16), San Rafael (2012)Google Scholar
  2. 2.
    Khan, H.U., Gadhoum, Y.: Measuring internet addiction in Arab based knowledge societies: a case study of Saudi Arabia. J. Theor. Appl. Inf. Technol. 96, (2018)Google Scholar
  3. 3.
    Abbasi, A., Hassan, A., Dhar, M.: Benchmarking twitter sentiment analysis tools. In: ResearchGate. 9th Language Resources and Evaluation Conference. Available at: (2014). Accessed 15 Dec 2018
  4. 4.
    Go, A., Bhayani, R., Huang, L.: Twitter sentiment classification using distant supervision. CS224N Proj. Rep. Stanford. 1, 12 (2009)Google Scholar
  5. 5.
    Hogenboom, A., Bal, D., Frasincar, F., Bal, M., de Jong, F., Kaymak, U.: Exploiting emoticons in sentiment analysis. In: Proceedings of the 28th Annual ACM Symposium on Applied Computing, pp. 703–710. ACM (2013)Google Scholar
  6. 6.
    Teh, P.L., Rayson, P.E., Pak, I., Piao, S.S., Yeng, S.M.: Reversing the polarity with emoticons. In 21st International Conference on Applications of Natural Language to Information Systems. Available at: (2016). Accessed 20 Dec 2018
  7. 7.
    Khan, H.U., Awan, M.A.: Possible factors affecting internet addiction: a case study of higher education students of Qatar. Int. J. Bus. Inf. Syst. 26(2), 261–276 (2017)Google Scholar
  8. 8.
    Brock, V.F., Khan, H.U.: Big data analytics: does organizational factor matters impact technology acceptance? J. Big Data. 4(1), 1–28 (2017)CrossRefGoogle Scholar
  9. 9.
    Feldman, R.: Techniques and applications for sentiment analysis. Commun. ACM. 56(4), 82–89 (2013). CrossRefGoogle Scholar
  10. 10.
    Urabe, Y., Rzepka, R., Araki, K.: Comparison of emoticon recommendation methods to improve computer-mediated communication. In: Ulusoy, Ö., Tansel, A.U., Arkun, E. (eds.) Recommendation and Search in Social Networks, pp. 23–39. Springer International Publishing (Lecture Notes in Social Networks) (2015). Google Scholar
  11. 11.
    Heang, J.F., Khan, H.U.: The role of internet marketing in the development of agricultural industry: a case study of China. J. Internet Commer. 14(1), 1–49 (2015)CrossRefGoogle Scholar
  12. 12.
    Bashir, G.M., Khan, H.U.: Factors affecting learning capacity of information technology concepts in a classroom environment of adult learner. 15th International Conference on Information Technology Based Higher Education and Training (IEEE Conference), Istanbul, Turkey, September 8th – September 10, 2016. (Conference Proceeding) (2016)Google Scholar
  13. 13.
    Statista: Twitter MAU worldwide 2016/Statistic, Statista. Available at: (2017). Accessed 10 Jan 2019
  14. 14.
    Najmi, E., Hashmi, K., Malik, Z., Rezgui, A., Khan, H.U.: ConceptOnto: an upper ontology based on Conceptnet. 11th ACS/IEEE International Conference on Computer Systems and Applications (AICCSA’ 2014), November 10–13, 2014, Doha, Qatar, pp. 366–372. (Conference Proceeding) (2014)Google Scholar
  15. 15.
    Nakov, P., Ritter, A., Rosenthal, S., Sebastiani, F., Stoyanov, V.: SemEval-2016 task 4: sentiment analysis in Twitter. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016), San Diego, USA. Available at: (2016). Accessed 20 Dec 2018
  16. 16.
    Mohammad, S.M., Kiritchenko, S., Zhu, X.: NRC-Canada: building the state-of-the-art in sentiment analysis of tweets. arXiv preprint arXiv:1308.6242. Available at: (2013). Accessed 27 Dec 2018
  17. 17.
    Dresner, E., Herring, S.C.: Functions of the nonverbal in CMC: emoticons and illocutionary force. Commun. Theory. 20(3), 249–268 (2010). CrossRefGoogle Scholar
  18. 18. Emoji Versions, v3.0. Available at: (2016). Accessed 4 Jan 2019
  19. 19.
    Yamamoto, Y., Kumamoto, T., Nadamoto, A.: Role of emoticons for multidimensional sentiment analysis of Twitter. In: Proceedings of the 16th International Conference on Information Integration and Web-based Applications & Services. ACM, pp. 107–115. Available at: (2014). Accessed 27 Dec 2018
  20. 20.
    Zhang, L., Pei, S., Deng, L., Han, Y., Zhao, J., Hong, F.: Microblog sentiment analysis based on emoticon networks model. In: Proceedings of the Fifth International Conference on Internet Multimedia Computing and Service. ACM, pp. 134–138. Available at: (2013). Accessed 27 Dec 2018
  21. 21.
    Ribeiro, F.N., Araújo, M., Gonçalves, P., André Gonçalves, M., Benevenuto, F.: SentiBench – a benchmark comparison of state-of-the-practice sentiment analysis methods. EPJ Data Sci. 5(1), (2016).
  22. 22.
    Bankole, O.A., Lalitha, M., Khan, H.U., Jinugu, A.: Information technology in the maritime industry past, present and future: focus on Lng carriers. 7th IEEE International Advance Computing Conference, Hyderabad, India, 5–7 Jan 2017. (Conference Proceeding) (2017)Google Scholar
  23. 23.
    Smuts, R.G., Lalitha, M., Khan, H.U.: Change management guidelines that adress barriers to technology adoption in an HEI context. 7th IEEE International Advance Computing Conference, Hyderabad, India, 5–7 Jan 2017. (Conference Proceeding) (2017)Google Scholar
  24. 24.
    Rajadesingan, A., Zafarani, R., Liu, H. Sarcasm detection on Twitter: a behavioral modeling approach. In: Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, pp. 97–106. ACM (WSDM ‘15), New York, NY (2015).
  25. 25.
    Hu, X., Tang, J., Gao, H., Liu, H.: Unsupervised sentiment analysis with emotional signals. In: Proceedings of the 22nd International Conference on World Wide Web, pp. 607–618. ACM (WWW ‘13), New York, NY (2013).
  26. 26.
    Beasley, A., Mason, W.: Emotional states vs. emotional words in social media. In: Proceedings of the ACM Web Science Conference, pp. 31:1–31:10. ACM (WebSci ‘15), New York, NY (2015).
  27. 27.
    Giachanou, A., Crestani, F.: Like it or not: a survey of twitter sentiment analysis methods. ACM Comput. Surv. 49(2), 28:1–28:41 (2016). CrossRefGoogle Scholar
  28. 28.
    Khan, H.U., Uwemi, S.: Possible impact of E-commerce strategies on the utilization of E-commerce in Nigeria. Int. J. Bus. Innov. Res. 15(2), 231–246 (2018)CrossRefGoogle Scholar
  29. 29.
    Khan, H.U., Ejike, A.C.: An assessment of the impact of mobile banking on traditional banking in Nigeria. Int. J. Bus. Excell. 11(4), 446:463 (2017)Google Scholar
  30. 30.
    Khan, H.U., Alhusseini, A.: Optimized web design in the Saudi culture. IEEE Science and Information Conference 2015, pp. 906–915. London, UK, 28–30 July 2015 (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.University of LiverpoolLiverpoolUK
  2. 2.College of Business and EconomicsQatar UniversityDohaQatar

Personalised recommendations