A Novel Ensemble Approach for Feature Selection to Improve and Simplify the Sentimental Analysis

  • Muhammad Latif
  • Usman QamarEmail author
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 997)


Text Classification is a renowned machine learning approach to simplify the domain-specific investigation. Consequently, it is frequently utilized in the field of sentimental analysis. The demanding business requirements urge to devise new techniques and approaches to improve the performance of sentimental analysis. In this context, ensemble of classifiers is one of the promising approach to improve classification accuracy. However, classifier ensemble is usually done for classification while ignoring the significance of feature selection. In the presence of right feature selection methodology, the classification accuracy can be significantly improved even when the classification is performed through a single classifier. This article presents a novel feature selection ensemble approach for sentimental classification. Firstly, the combination of three well-known features (i.e. lexicon, phrases and unigram) is introduced. Secondly, two level ensemble is proposed for feature selection by exploiting Gini Index (GI), Information Gain (IG), Support Vector Machine (SVM) and Logistic Regression (LR). Subsequently, the classification is performed through SVM classifier. The implementation of proposed approach is carried out in GATE and RapidMiner tools. Furthermore, two benchmark datasets, frequently utilized in the domain of sentimental classification, are used for experimental evaluation. The experimental results prove that our proposed ensemble approach significantly improve the performance of sentimental classification with respect to well-known state-of-the-art approaches. Furthermore, it is also analyzed that the ensemble of classifiers for the improvement of classification accuracy is not necessarily important in the presence of right feature selection methodology.


Feature selection ensemble Classifiers ensemble Sentimental classification SVM 


  1. 1.
    Wang, G., et al.: Sentiment classification: the contribution of ensemble learning. J. Decis. Support Syst. 57, 77–93 (2013)Google Scholar
  2. 2.
    Onan, A., Korukoğlu, S., Bulut, H.: A multiobjective weighted voting ensemble classifier based on differential evolution algorithm for text sentiment classification. JESA 62, 1–16 (2016)Google Scholar
  3. 3.
    da Silva, N.F.F., et al.: Tweet sentiment analysis with classifier ensembles. J. Decis. Support Syst. 66, 170–179 (2014)Google Scholar
  4. 4.
    Athar, A., Butt, W.H., Anwar, M.W., Latif, M., Azam, F.: Exploring the ensemble of classifiers for sentimental analysis—a systematic literature review. In: 9th International Conference on Machine Learning and Computing, Singapore 2017Google Scholar
  5. 5.
    Fersini, E., Messina, E., Pozzi, F.A.: Sentiment analysis: Bayesian ensemble learning. DSS 68, 26–38 (2014)Google Scholar
  6. 6.
    Xia, R., Zong, C., Li, S.: Ensemble of feature sets and classification algorithms for sentiment classification. Inf. Sci. 181(6), 1138–1152 (2011)CrossRefGoogle Scholar
  7. 7.
    Vinodhini, G., Chandrasekaran, R.M.: A comparative performance evaluation of neural network based approach for sentiment classification of online reviews. J. King Saud Univ. – Comput. Inf. Sci. 28(1), 2–12 (2016)Google Scholar
  8. 8.
    Catal, C., Nangir, M.: A sentiment classification model based on multiple classifiers. Appl. Soft Comput. 50, 135–141 (2017)CrossRefGoogle Scholar
  9. 9.
    King, M.A., Abrahams, A.S., Ragsdale, C.T.: Ensemble learning methods for pay-per-click campaign management. ESA 42(10), 4818–4829 (2015)Google Scholar
  10. 10.
    Lochter, J.V., Zanetti, R.F., Reller, D., Almeida, T.A.: Short text opinion detection using ensemble of classifiers and semantic indexing. ESA 62, 243–249 (2016)Google Scholar
  11. 11.
    Ekbal, A., Saha, S.: Combining feature selection and classifier ensemble using a multi objective simulated annealing approach: application to named entity recognition. Soft. Comput. 17(1), 1–16 (2013)Google Scholar
  12. 12.
    Wang, Y., Rao, Y., Zhan, X., Chen, H., Luo, M., Yin, J.: Sentiment and emotion classification over noisy labels. KBS 111, 207–216 (2016)Google Scholar
  13. 13.
    Vinodhini, G.: A sampling based sentiment mining approach for e-commerce applications. JIPM 53(1), 223–236 (2017)Google Scholar
  14. 14.
    Prabowo, R., Thelwall, M.: Sentiment analysis: a combined approach. Informatics 3(2), 143–157 (2009)Google Scholar
  15. 15.
    Saha, S., Ekbal, A.: Combining multiple classifiers using vote based classifier ensemble technique for named entity recognition. JD&KE 85, 15–39 (2013)Google Scholar
  16. 16.
    Abbasi, A., Chen, H., Thoms, S., Fu, T.: Affect analysis of web forums and blogs using correlation ensembles. IEEE Trans. Knowl. Data Eng. 20(9), 1168–1180 (2008)Google Scholar
  17. 17.
    Vinodhini, G., Chandrasekaran, R.M.: Sentiment mining using SVM-based hybrid classification model. In: Computational Intelligence, Cyber Security and Computational Models, vol. 246, pp. 155–162. Springer, New Delhi (2013)Google Scholar
  18. 18.
    Lin, Y., Wang, X., Zhang, J., Zhou, A.: Assembling the optimal sentiment classifiers. In: Proceedings of 13th International Conference, Paphos, Cyprus, November 28–30, 2012, vol. 7651, pp. 271–283 (2012)Google Scholar
  19. 19.
    Wan, Y., Gao, Q.: An ensemble sentiment classification system of Twitter data for airline services analysis. In: IEEE 15th Data Mining Workshops (2015)Google Scholar
  20. 20.
    Su, Y., Zhang, Y., Ji, D., Wang, Y., Wu, H.: Ensemble learning for sentiment classification. In: Workshop on Chinese Lexical Semantics, vol. 7717, pp 84–93. Springer, Berlin, Heidelberg (2013)Google Scholar
  21. 21.
    Whitehead, M., Yaeger, L.: Sentiment mining using Ensemble classification model. In: Innovations and advances in computer sciences and engineering, pp. 509–514. Springer, Dordrecht (2010)Google Scholar
  22. 22.
    Chalothom, T., Ellman, J.: Simple approaches of sentiment analysis via ensemble learning. In: Information science and applications, vol. 339, pp 631–639. Springer-Verlag, Berlin, Heidelberg (2015)Google Scholar
  23. 23.
    Prusa, J., Khoshgoftaar, T.M., Dittman, D.J.: Using ensemble learners to improve classifier performance on tweet sentiment data. In: IEEE 16th ICIRI 2015Google Scholar
  24. 24.
    Hagen, M., Potthast, M., Büchner, M., Stein, B.: Twitter sentiment detection via ensemble classification using averaged confidence scores. In: European Conference on Information Retrieval, pp. 741–754. Springer, Cham (2015)Google Scholar
  25. 25.
    Dai, L., Chen, H., Li, X.: Improving sentiment classification using feature highlighting and feature bagging. In: 11th IEEE ICDMW 2011, pp. 61–66Google Scholar
  26. 26.
    Wang, Z., Li, S., Zhou, G., Li, P., Zhu, Q.: Imbalanced sentiment classification with multi-strategy ensemble learning. In: Proceedings Asian Language Processing (2011)Google Scholar
  27. 27.
    Wang, W.: Heterogeneous Bayesian ensembles for classifying spam emails. In: Proceedings on Neural Networks, 2010Google Scholar
  28. 28.
    Kumar, V., Minz, S.: Multi-view ensemble learning for poem data classification using SentiWordNet. In: Advanced Computing and Informatics Proceedings of ICACNI 2014, vol. 27, pp. 57–66Google Scholar
  29. 29.
    Hassan, A., Abbasi, A., Zeng, D.: Twitter sentiment analysis: a bootstrap ensemble framework. In: International Conference on Social Computing (2013)Google Scholar
  30. 30.
    Vinodhini, G., Chandrasekaran, R.M.: Sentiment mining using SVM-based hybrid classification model. Comput. Intell. Cyber Secur. Comput. Models 246, 155–162 (2013)Google Scholar
  31. 31.
    Wang, F., Zhang, Y., Rao, Q., Li, K., Zhang, H.: Exploring mutual information-based sentimental analysis with kernel-based extreme learning machine for stock prediction. Soft Comput. 2016, PP 1–13Google Scholar
  32. 32.
    Lu, B., Tsou, B.K.: Combining a large sentiment lexicon and machine learning for subjectivity classification. In: Proceedings of the Ninth International Conference on Machine Learning and Cybernetics, 11–14 July 2010Google Scholar
  33. 33.
    RapidMiner Tool.
  34. 34.
  35. 35.
  36. 36.
  37. 37.
    Blitzer, J., Dredze, M., Pereira, F.: Biographies, bollywood, boom-boxes and blenders: domain adaptation for sentiment classification. In: ACL, vol. 7, pp. 440–447, Jun 2007Google Scholar
  38. 38.
    Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up? sentiment classification using machine learning techniques. In: Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing, vol. 10, pp. 79–86 (2002)Google Scholar
  39. 39.
    GATE2CVS Tool. Accessed Apr 2017
  40. 40.
    Xia, R., Xu, F., Yu, J., Qi, Y., Cambria, E.: Polarity shift detection, elimination and ensemble: A three-stage model for document-level sentiment analysis. J. Inf. Process. Manage. 52, 36–45, 2016Google Scholar
  41. 41.
    Fattah, M.A.: New term weighting schemes with combination of multiple classifiers. J. Neurocomput. 167, 434–442 (2015)Google Scholar
  42. 42.
    Dang, Y., Zhang, Y., Chen, H.: A lexicon-enhanced method for sentiment classification: an experiment on online product reviews. IEEE Intell. Syst. 25(4), 46–53 (2010)Google Scholar
  43. 43.
    Agarwal, B., Poria, S., Mittal, N., Gelbukh, A., Hussain, A.: Concept-level sentiment analysis with dependency-based semantic parsing: a novel approach. J. Cogn. Comput. 7(4), 487–499 (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Computer Engineering, College of Electrical & Mechanical Engineering (E&ME)National University of Sciences and Technology (NUST)IslamabadPakistan
  2. 2.National Centre for Big Data and Cloud Computing (NCBC)LahorePakistan

Personalised recommendations