Abstract
In the digital world, social media has become one of the most popular communication mediums that allow users to share their views on various topics in their social network. For example, Twitter users are allowed to share their thoughts on various topic by sending tweets with a maximum length of 140 characters. Hence, social media driven information contains opinions and sentiments on various topics of interest which are extremely useful for companies to design marketing strategies. Sentiment Analysis is widely used to assist people to understand the massive amount of data available online and identify the polarity of the topical based social media opinions. However, social media platforms’ users come from all over the world and have variation in terms of informal language and short notation used on social media platforms. Therefore, the identification on the polarity of topical social media has become more challenging and the accuracy on the polarity of topical social media opinions might be influenced. This paper investigates the effectiveness of applying different spelling correction algorithms, such as Levenshtein distance and Peter Norvig’s algorithm for spelling correction of unknown words found in social media such as Twitter, before carrying out sentiment analysis. The evaluation of spelling correction algorithms on sentiment analysis is carried out by comparing the polarities of manually annotated tweets with the polarities obtained from the sentiment analysis algorithm. Based on the results obtained, there are slight improvements in term of percentage of matched polarity, where 1.6% improvement by using the Levenshtein distance-based algorithm and 2.0% improvement by using the Peter Norvig’s algorithm.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Fazal, M.K., Aurangzeb, K., Muhammad, Z.A., Shakeel, A.: Context-aware spelling corrector for sentiment analysis. MAGNT Res. Rep. 2(5), 1–10 (2014). ISSN 1444-8939
Ayushi, D., Thish, G., Vasudeva, V.: Manwitter sentiment analysis: the good, the bad, and the neutral. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), Denver, Colorado, 4–5 June 2015, pp. 520–526. Association for Computational Linguistics (2015)
Read, J.: Using emoticons to reduce dependency in machine learning techniques for sentiment classification. In: Proceedings of the ACL Student Research Workshop (2005), pp 43–48 (2005)
Baccianella, S.A., Esuli, F.S.: SENTIWORDNET 3.0: an enhanced lexical resource for sentiment analysis and opinion mining. In: Proceedings of LREC (2010)
Liu, B., Li, S., Lee, W.S., Yu, P.S.: Text classification by labeling words. In: Proceedings of the National Conference on Artificial Intelligence, pp 425–430. AAAI Press/MIT Press, Menlo Park/Cambridge/London (2004)
Zubair, A.M., Aurangzeb, K., Shakeel, A., Fazal, M.K.: A review of feature extraction in sentiment analysis. J. Basic Appl. Sci. Res. 4(3), 181–186 (2014)
Jaccard, P.: Distribution de la flore alpine dans le bassin des Dranses et dans quelques régions voisines. Bulletin de la Société Vaudoise des Sciences Naturelles 37, 241–272 (1901)
Nerbonne, J., Heeringa, W., Kleiweg, P.: Edit distance and dialect proximity. In: Time Warps, String Edits and Macromolecules: The Theory and Practice of Sequence Comparison, 2nd edn, p. 15 (1999)
Gilleland, M.: Levenshtein distance in three flavors (2009). https://people.cs.pitt.edu/~kirk/cs1501/Pruhs/Spring2006/assignments/editdistance/Levenshtein%20Distance.htm. Accessed 10 Aug 2018
Yannakoudakis, E.J., Fawthrop, D.: The rules of spelling errors. Inf. Process. Manage. 19(2), 87–99 (1983)
Bilal, A.: Lexical normalisation of Twitter data. http://www.aclweb.org/anthology/P11–1038. Accessed 10 Aug 2018
Abinash, T., Ankit, A., Santanu, K.R.: Classification of sentiment reviews using n-gram machine learning approach. Expert Syst. Appl. 57, 117–126 (2016)
Bo, P., Lillian, L.: Opinion Mining and Sentiment Analysis. Found. Trends Inf. Retrieval 2(1–2), 1–135 (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Alfred, R., Teoh, R.W. (2019). Improving Topical Social Media Sentiment Analysis by Correcting Unknown Words Automatically. In: Yap, B., Mohamed, A., Berry, M. (eds) Soft Computing in Data Science. SCDS 2018. Communications in Computer and Information Science, vol 937. Springer, Singapore. https://doi.org/10.1007/978-981-13-3441-2_23
Download citation
DOI: https://doi.org/10.1007/978-981-13-3441-2_23
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-3440-5
Online ISBN: 978-981-13-3441-2
eBook Packages: Computer ScienceComputer Science (R0)