Abstract
The relations among heterogeneous data objects (e.g., image, tag, user and geographical point-of-interest (POI)) on interactive Online Social Media (OSM) play an important information source in describing complicated connections among Web entities (users and POIs) and items (images). Jointly predicting multiple relations instead of single relation completion in separate tasks facilitates sufficient knowledge sharing among heterogeneous relations and mitigate the information imbalance among different tasks. In this paper, we propose JEREMIE, a Joint SEmantic FeatuRe LEarning model via Multi-relational MatrIx ComplEtion, which jointly complements the semantic features of different entities from heterogeneous domains. Specifically, to perform appropriate information averaging, we first divide the social image collection into data blocks according to the affiliated user and POI information, where POIs are detected by mean shift from the GPS information. Then we develop a block-wise batch learning method which jointly learns the semantic features (e.g., image-tag, POI-tag and user-tag relations) by optimizing a transductive matrix completion framework with structure preservation and appropriate information averaging functionality. Experimental results on automatic image annotation, image-based user retrieval and image-based POI retrieval demonstrate that our approach achieves promising performance in various relation prediction tasks on six city-scale OSM datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cartis, C., Gould, N.I., Toint, P.L.: On the evaluation complexity of composite function minimization with applications to nonconvex nonlinear programming. SIAM J. Optim. 21(4), 1721–1739 (2011)
Chen, M., Zheng, A., Weinberger, K.: Fast image tagging. In: ICML, pp. 1274–1282 (2013)
Cho, E., Myers, S.A., Leskovec, J.: Friendship and mobility: user movement in location-based social networks. In: KDD, pp. 1082–1090. ACM (2011)
Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. TPAMI 24(5), 603–619 (2002)
Crandall, D.J., Backstrom, L., Huttenlocher, D., Kleinberg, J.: Mapping the world’s photos. In: WWW 2009, pp. 761–770. ACM (2009)
Guillaumin, M., Mensink, T., Verbeek, J., Schmid, C.: TagProp: discriminative metric learning in nearest neighbor models for image auto-annotation. In: ICCV, pp. 309–316. IEEE (2009)
Halpin, H., Robu, V., Shepherd, H.: The complex dynamics of collaborative tagging. In: WWW, pp. 211–220. ACM (2007)
Hariharan, B., Zelnik-Manor, L., Varma, M., Vishwanathan, S.: Large scale max-margin multi-label classification with priors. In: ICML, pp. 423–430 (2010)
Jenatton, R., Roux, N.L., Bordes, A., Obozinski, G.R.: A latent factor model for highly multi-relational data. In: NIPS, pp. 3167–3175 (2012)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS, pp. 1097–1105 (2012)
Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401(6755), 788–791 (1999)
Li, X., Snoek, C.G., Worring, M.: Learning social tag relevance by neighbor voting. TMM 11(7), 1310–1322 (2009)
Lin, Z., Ding, G., Hu, M., Wang, J., Ye, X.: Image tag completion via image-specific and tag-specific linear sparse reconstructions. In: CVPR, pp. 1618–1625. IEEE (2013)
Liu, D., Yan, S., Hua, X.S., Zhang, H.J.: Image retagging using collaborative tag propagation. TMM 13(4), 702–712 (2011)
Liu, D., Ye, G., Chen, C.T., Yan, S., Chang, S.F.: Hybrid social media network. In: ACMMM, pp. 659–668. ACM (2012)
Liu, J., Li, Z., Tang, J., Jiang, Y., Lu, H.: Personalized geo-specific tag recommendation for photos on social websites. TMM 16(3), 588–600 (2014)
Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online learning for matrix factorization and sparse coding. JMLR 11, 19–60 (2010)
Moxley, E., Kleban, J., Manjunath, B.: Spirittagger: a geo-aware tag suggestion tool mined from flickr. In: ACM MIR, pp. 24–30. ACM (2008)
Qian, X., Feng, H., Zhao, G., Mei, T.: Personalized recommendation combining user interest and social circle. TKDE 26(7), 1763–1777 (2014)
Russell, B.C., Torralba, A., Murphy, K.P., Freeman, W.T.: LabelMe: a database and web-based tool for image annotation. IJCV 77(1–3), 157–173 (2008)
Singh, A.P., Gordon, G.J.: Relational learning via collective matrix factorization. In: KDD, pp. 650–658. ACM (2008)
Sutskever, I., Tenenbaum, J.B., Salakhutdinov, R.R.: Modelling relational data using bayesian clustered tensor factorization. In: NIPS, pp. 1821–1828 (2009)
Thomee, B., Shamma, D.A., Friedland, G., Elizalde, B., Ni, K., Poland, D., Borth, D., Li, L.J.: YFCC100M: the new data in multimedia research. Commun. ACM 59(2), 64–73 (2016)
Wang, Z., Zhu, W., Cui, P., Sun, L., Yang, S.: Social media recommendation. In: Ramzan, N., van Zwol, R., Lee, J.S., Clüver, K., Hua, X.S. (eds.) Social Media Retrieval. CCN, pp. 23–42. Springer, London (2013). https://doi.org/10.1007/978-1-4471-4555-4_2
Wu, L., Jin, R., Jain, A.K.: Tag completion for image retrieval. TPAMI 35(3), 716–727 (2013)
Wu, L., Yang, L., Yu, N., Hua, X.S.: Learning to tag. In: WWW 2009, pp. 361–370. ACM (2009)
Ye, M., Yin, P., Lee, W.C., Lee, D.L.: Exploiting geographical influence for collaborative point-of-interest recommendation. In: SIGIR, pp. 325–334. ACM (2011)
Yin, H., Sun, Y., Cui, B., Hu, Z., Chen, L.: LCARS: a location-content-aware recommender system. In: KDD, pp. 221–229. ACM (2013)
Zhang, J., Wang, S., Huang, Q.: Location-based parallel tag completion for geo-tagged social image retrieval. In: ICMR, pp. 355–362. ACM (2015)
Zhu, G., Yan, S., Ma, Y.: Image tag refinement towards low-rank, content-tag prior and error sparsity. In: ACMMM, pp. 461–470. ACM (2010)
Zhu, J.: Max-margin nonparametric latent feature models for link prediction. In: ICML, pp. 719–726 (2012)
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China under Grant 61672497, Grant 61332016, Grant 61620106009, Grant 61650202 and Grant U1636214, in part by the National Basic Research Program of China (973 Program) under Grant 2015CB351802, and in part by the Key Research Program of Frontier Sciences of CAS under Grant QYZDJ-SSW-SYS013. This work was also partially supported by CAS Pioneer Hundred Talents Program by Dr. Qiang Qu.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Zhang, J., Wang, S., Qu, Q., Huang, Q. (2018). JEREMIE: Joint Semantic Feature Learning via Multi-relational Matrix Completion. In: Doulkeridis, C., Vouros, G., Qu, Q., Wang, S. (eds) Mobility Analytics for Spatio-Temporal and Social Data. MATES 2017. Lecture Notes in Computer Science(), vol 10731. Springer, Cham. https://doi.org/10.1007/978-3-319-73521-4_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-73521-4_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-73520-7
Online ISBN: 978-3-319-73521-4
eBook Packages: Computer ScienceComputer Science (R0)