Abstract
Most studies about sentiment analysis on microblogging usually focus on the features mining from the text. This paper presents a new sentiment analysis method by combing features from text with features from image. Bigram model is applied in text feature extraction while color and texture information are extracted from images. Considering the sentiment classification, we propose a new neighborhood classier based on the similarity of two instances described by the fusion of text and features. Experimental results show that our proposed method can improve the performance significantly on Sina Weibo data (we collect and label the data). We find that our method can not only increasingly improve the F values of the classification comparing with only used text or images features, but also outperforms the NaiveBayes and SVM classifiers using all features with text and images.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bianchi-Berthouze, N.: K-DIME: an affective image filtering system. IEEE on Multimedia 10(3), 103–106 (2003)
Diakopoulos, N.A., Shamma, D.A.: Characterizing debate performance via aggregated twitter sentiment. In: Conference on Human Factors in Computing Systems (CHI 2010) (2010)
Tamura, H., Mori, S., Yamawaki, T.: Textural features corresponding to visual perception. IEEE Transactions on Systems, Man and Cybernetics 8(6), 460–473 (1978)
Itten, J.: The art of color: the subjective experience and objective rationale of color. Van Nostrand Reinhold, New York (1973)
Wei-ning, W., Ying-lin, Y., Sheng-ming, J.: Image retrieval by emotional semantics: a study of emotional space and feature extraction. In: IEEE International Conference on Systems, Man and Cybernetics, SMC 2006, vol. 4, pp. 3534–3539. IEEE (2006)
Jansen, B.J., Zhang, M., Sobel, K., Chowdury, A.: Twitter power: Tweets as electronic word of mouth. Journal of the American Society for Information Science and Technology (2009)
Mardia, K.V., Jupp, P.E.: Directional statistics. Wiley (2009)
Tumasjan, A., Sprenger, T.O., Sandner, P.G., et al.: Predicting elections with twitter: what 140 characters reveal about political sentiment. In: Proceedings of the fourth international AAAI conference on weblogs and social media, pp. 178–185 (2010)
Haralock, R.M., Shapiro, L.G.: Computer and robot vision. Addison-Wesley Longman Publishing Co., Inc (1991)
Bollen, J., Mao, H., Zeng, X.: Twitter mood predicts the stock market. Journal of Computational Science 2(1), 1–8 (2011)
Datta, R., Joshi, D., Li, J., Wang, J.Z.: Studying aesthetics in photographic images using a computational approach. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 288–301. Springer, Heidelberg (2006)
OConnor, B., Balasubramanyan, R., Routledge, B.R., et al.: From tweets to polls: linking text sentiment to public opinion time series. In: Proceedings of the International AAAI Conference on Weblogs and Social Media, pp. 122–129 (2010)
Yang, J., et al.: Feature fusion: parallel strategy vs. serial strategy. Pattern Recognition 36(6), 1369–1381 (2003)
Wang, H.L., Cheong, L.F.: Affective understanding in film. IEEE Transactions on Circuits and Systems for Video Technology 16(6), 689–704 (2006)
Jones, K.S.: A statistical interpretation of term specificity and its application in retrieval. Journal of documentation 28(1), 11–21 (1972)
Colombo, C., Del Bimbo, A., Pala, P.: Semantics in visual information retrieval. IEEE on Multimedia 6(3), 38–53 (1999)
Pang, B., Lee, L.: Opinion mining and sentiment analysis. Found. Trends Inf. Retr. 2(1–2), 1–135 (2008)
Osgood, C.E., Suci, G.J., Tannenbaum, P.H.: The measurement of meaning. University of Illinois Press, Urbana (1957)
Tan, S., Zhang, J.: An empirical study of sentiment analysis for chinese documents. Expert Systems with Applications 34(4), 2622–2629 (2008)
Galavotti, L., Sebastiani, F., Simi, M.: Experiments on the use of feature selection and negative evidence in automated text categorization. In: Borbinha, J.L., Baker, T. (eds.) ECDL 2000. LNCS, vol. 1923, pp. 59–68. Springer, Heidelberg (2000)
Kuroda, K., Hagiwara, M.: An image retrieval system by impression words and specific object names–IRIS. Neurocomputing 43(1), 259–276 (2002)
Yanulevskaya, V., Van Gemert, J.C., Roth, K., et al.: Emotional valence categorization using holistic image features. In: 15th IEEE International Conference on Image Processing, ICIP 2008, pp. 101–104. IEEE (2008)
Read, J.: Using emoticons to reduce dependency in machine learning techniques for sentiment classification. In: ACL. The Association for Computer Linguistics (2005)
Zagibalov, T.: Unsupervised and knowledge-poor approaches to sentiment analysis. Diss. University of Sussex (2010)
Ponomareva, N., Thelwall, M.: Do neighbours help? An exploration of graph-based algorithms for cross-domain sentiment classification. The 2012 Conference on Empirical Methods on Natural Language Processing and Computational Natural Language Learning (EMNLPCoNLL 2012), pp. 655–665 (2012)
Hayashi, T., Hagiwara, M.: Image query by impression words-the IQI system. IEEE Transactions on Consumer Electronics 44(2), 347–352 (1998)
Hanjalic, A.: Extracting moods from pictures and sounds: Towards truly personalized TV. IEEE on Signal Processing Magazine 23(2), 90–100 (2006)
Wu, Q., Zhou, C.-L., Wang, C.: Content-based affective image classification and retrieval using support vector machines. In: Tao, J., Tan, T., Picard, R.W. (eds.) ACII 2005. LNCS, vol. 3784, pp. 239–247. Springer, Heidelberg (2005)
Altman, N.S.: An introduction to kernel and nearest-neighbor nonparametric regression. The American Statistician 46(3), 175–185 (1992)
Valdez, P., Mehrabian, A.: Effects of color on emotions. Journal of Experimental Psychology: General 123(4), 394 (1994)
Zhang, H.P., Liu, Q., Cheng, X.Q., et al.: Chinese lexical analysis using hierarchical hidden markov model. In: Proceedings of the second SIGHAN workshop on Chinese language processing, vol. 17, pp. 63–70. Association for Computational Linguistics (2003)
Go, A., Huang, L., Bhayani, R.: Twitter sentiment analysis. Final Projects from CS224N for Spring 2008/2009 at The Stanford Natural Language Processing Group (2009)
Muralidharan, S., Rasmussen, L., Patterson, D., et al.: Hope for Haiti: An analysis of Facebook and Twitter usage during the earthquake relief efforts. Public Relations Review 37(2), 175–177 (2011)
Mojsilovic, A., Gomes, J., Rogowitz, B.: Semantic-friendly indexing and quering of images based on the extraction of the objective semantic cues. International Journal of Computer Vision 56(1–2), 79–107 (2004)
Asur, S., Huberman, B.A.: Predicting the future with social media. In: 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT), vol. 1. IEEE (2010)
Wang, W., He, Q.: A survey on emotional semantic image retrieval. In: 15th IEEE Int. Conf. on Image Processing, pp. 117–120 (2008)
Zhai, Z., et al.: Exploiting effective features for chinese sentiment classification. Expert Systems with Applications 38(8), 9139–9146 (2011)
Mejova, Y., Srinivasan, P.: Exploring feature definition and selection for sentiment classifiers. ICWSM (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Zhang, Y., Shang, L., Jia, X. (2015). Sentiment Analysis on Microblogging by Integrating Text and Image Features. In: Cao, T., Lim, EP., Zhou, ZH., Ho, TB., Cheung, D., Motoda, H. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2015. Lecture Notes in Computer Science(), vol 9078. Springer, Cham. https://doi.org/10.1007/978-3-319-18032-8_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-18032-8_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-18031-1
Online ISBN: 978-3-319-18032-8
eBook Packages: Computer ScienceComputer Science (R0)