Abstract
Image is an important part of microblog, and its visual information can offer additional semantics besides the textual information. To overcome short text’s semantic sparsity problem and fully utilize the semantics of text and image, we propose a cross-modal short text expansion method for microblog search in this paper. First, we expand short texts using the distributed representations of words, and then based on deep neural network, we extract related information of images and append them to the original short text. The expanded pseudo-documents contain richer semantics, and by turning pseudo-documents into vectors, we can achieve accurate microblog search. Experiments on real-world datasets show that the proposed cross-modal short text expansion method can effectively extract the semantics of microblogs and improve search performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
X. Zhao, F. Zhu, W. Qian et al., Impact of multimedia in Sina Weibo: Popularity and life span, Semantic Web and Web Science (Springer, New York, NY, 2013), pp. 55–65
T. Chen, D. Lu, M.Y. Kan et al., Understanding and classifying image tweets, in Proceedings of the 21st ACM International Conference on Multimedia (ACM, 2013), pp. 781–784
T. Chen, H.M. SalahEldeen, X. He et al., VELDA: relating an image Tweet’s text and images, in AAAI (2015), pp. 30–36
Z. Chen, A. Mukherjee, B. Liu et al., Discovering coherent topics using general knowledge, in Proceedings of the 22nd ACM International Conference on Information & Knowledge Management (ACM, 2013), pp. 209–218
Y. Zuo, J. Zhao, K. Xu, Word network topic model: a simple but general solution for short and imbalanced texts. Knowl. Inf. Syst. 48(2), 379–398 (2016)
N. Aletras, M. Stevenson, Evaluating topic coherence using distributional semantics, in Proceedings of the 10th International Conference on Computational Semantics (IWCS 2013)–Long Papers (2013), pp. 13–22
P. Bicalho, M. Pita, G. Pedrosa et al., A general framework to expand short text for topic modeling. Inf. Sci. 393, 66–81 (2017)
T. Mikolov, K. Chen, G. Corrado et al., Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
M. Mitchell, X. Han, J. Dodge et al., Midge: generating image descriptions from computer vision detections, in Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics (Association for Computational Linguistics, 2012), pp. 747–756
J. Mao, W. Xu, Y. Yang et al., Explain images with multimodal recurrent neural networks. arXiv preprint arXiv:1410.1090 (2014)
O. Vinyals, A. Toshev, S. Bengio et al., Show and tell: a neural image caption generator, in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, 2015), pp. 3156–3164
K He, X. Zhang, S. Ren et al., Deep residual learning for image recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 770–778
S. Arora, Y. Liang, T. Ma, A simple but tough-to-beat baseline for sentence embeddings (2016)
Acknowledgements
This work is supported by the National Natural Science Foundation of China (No. 61320106006, No. 61532006, No. 61772083).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Shi, Y., Du, J., Kou, F., Chen, C. (2019). A Cross-Modal Short Text Semantic Expansion Method for Microblog Search. In: Jia, Y., Du, J., Zhang, W. (eds) Proceedings of 2018 Chinese Intelligent Systems Conference. Lecture Notes in Electrical Engineering, vol 529. Springer, Singapore. https://doi.org/10.1007/978-981-13-2291-4_4
Download citation
DOI: https://doi.org/10.1007/978-981-13-2291-4_4
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-2290-7
Online ISBN: 978-981-13-2291-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)