Abstract
With the popularization of social media, inferring user profiles from the user-generated content has aroused wide attention for its applications in marketing, advertising, recruiting, etc. Most existing works focus on using data from single modality (such as texts and profile photos) and fail to notice that the combination of multi-modal data can supplement with each other and can therefore improve the prediction accuracy. In this paper, we propose AMUP model, namely the Attention-based Multi-modal User Profiling model, which uses different tailored neural networks to extract and fuse semantic information from three modalities, i.e., texts, avatar, and relation network. We propose a dual attention mechanism. The word-level attention network selects informative words from the noisy and prolix texts and the modality-level attention network addresses the problem of imbalanced contribution among different modalities. Experimental results on more than 1.5K users’ real-world data extracted from a popular Q&A social platform show that our proposed model outperforms the single-modality methods and achieves better accuracy when compared with existing approaches that utilize multi-modal data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Farnadi, G., et al.: Computational personality recognition in social media. User Model. User-Adap. Inter. 26(2–3), 109–142 (2016)
Segalin, C., Cheng, D.S., Cristani, M.: Social profiling through image understanding: personality inference using convolutional neural networks. Comput. Vis. Image Underst. 156, 34–50 (2017)
Liu, Y., et al.: CrowdOS: a ubiquitous operating system for crowdsourcing and mobile crowd sensing. arXiv preprint arXiv:1909.00805 (2019)
Guo, B., et al.: Mobile crowd sensing and computing: the review of an emerging human-powered sensing paradigm. ACM Comput. Surv. (CSUR) 48(1), 7 (2015)
Nowson, S., Oberlander, J.: The identity of bloggers: openness and gender in personal weblogs. In Proceedings of AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs, pp. 163–167 (2006)
Gou, L., Zhou, M.X., Yang, H.: Knowmeandshareme: understanding automatically discovered personality traits from social media and user sharing preferences. In: Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems, pp. 955–964. ACM (2014)
Fong, K., Mar, R.A.: What does my avatar say about me? Inferring personality from avatars. Pers. Soc. Psychol. Bull. 41(2), 237–249 (2015)
Staiano, J., et al.: Friends don’t lie: inferring personality traits from social network structure. In: Proceedings of the 2012 ACM Conference on Ubiquitous Computing. ACM (2012)
Wei, H., et al.: Beyond the words: predicting user personality from heterogeneous information. In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining. ACM (2017)
Farnadi, G., et al.: User profiling through deep multimodal fusion. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. ACM (2018)
Mischel, W., Shoda, Y., Smith, R.E., Mischel, F.W.: Introduction to Personality. University of Phoenix: A John Wiley & Sons, Ltd., Publication (2004)
Tausczik, Y.R., Pennebaker, J.W.: The psychological meaning of words: LIWC and computerized text analysis methods. J. Lang. Soc. Psychol. 29(1), 24–54 (2010)
Yarkoni, T.: Personality in 100,000 words: a large-scale analysis of personality and word use among bloggers. J. Res. Pers. 44(3), 363–373 (2010)
Schwartz, H.A., et al.: Personality, gender, and age in the language of social media: the open-vocabulary approach. PLoS ONE 8(9), e73791 (2013)
Garcia, D., Sikström, S.: The dark side of Facebook: Semantic representations of status updates predict the Dark Triad of personality. Personality Individ. Differ. 67, 92–96 (2014)
Zhang, D., et al.: User classification with multiple textual perspectives. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers (2016)
Qiu, L., et al.: You are what you tweet: Personality expression and perception on Twitter. J. Res. Personality 46(6), 710–718 (2012)
Ferwerda, B., Schedl, M., Tkalcic, M.: Predicting personality traits with Instagram pictures. In: Proceedings of the 3rd Workshop on Emotions and Personality in Personalized Systems 2015. ACM (2015)
Liu, L., et al.: Analyzing personality through social media profile picture choice. In: Tenth International AAAI Conference on Web and Social Media (2016)
Youyou, W., Kosinski, M., Stillwell, D.: Computer-based personality judgments are more accurate than those made by humans. Proc. Natl. Acad. Sci. 112(4), 1036–1040 (2015)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. Comput. Sci. (2014)
Grover, A., Leskovec, J.: node2vec: scalable feature learning for networks. In: ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (2016)
Perozzi, B., Al-Rfou, R., Skiena, S.: DeepWalk: online learning of social representations. In: ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (2014)
Mikolov, T., et al.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
Akiba, T., Suzuki, S., Fukuda, K.: Extremely large minibatchsgd: training resnet-50 on ImageNet in 15 minutes. arXiv preprint arXiv:1711.04325 (2017)
Nguyen, D., et al.: How old do you think I am? A study of language and age in Twitter. In: Seventh International AAAI Conference on Weblogs and Social Media (2013)
Bay, H., Tuytelaars, T., Van Gool, L.: SURF: speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006). https://doi.org/10.1007/11744023_32
Kim, Y.: Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014)
Acknowledgement
This work was partially supported by the National Key R&D Program of China (2017YFB1001800), the National Science Foundation of China (No. 61772428, 61725205), and the University-Enterprise Cooperation of Northwestern Polytechnical University (No. G2019KY04302).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Li, Z., Guo, B., Sun, Y., Wang, Z., Wang, L., Yu, Z. (2019). An Attention-Based User Profiling Model by Leveraging Multi-modal Social Media Contents. In: Ning, H. (eds) Cyberspace Data and Intelligence, and Cyber-Living, Syndrome, and Health. CyberDI CyberLife 2019 2019. Communications in Computer and Information Science, vol 1138. Springer, Singapore. https://doi.org/10.1007/978-981-15-1925-3_20
Download citation
DOI: https://doi.org/10.1007/978-981-15-1925-3_20
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-1924-6
Online ISBN: 978-981-15-1925-3
eBook Packages: Computer ScienceComputer Science (R0)