Abstract
With the recent penetration and proliferation of social networks into our lives, human choices and preferences have become more socially accessible. This easy accessibility of private data in different formats has opened many new initiatives. The big explosion of multimedia data on the web has enabled social networks to gauge user likes, dislikes, and needs. This has imposed high demands on multimedia information retrieval (MIR) techniques. This manuscript illustrates the MIR concept in terms of its application to social media. It further positions the current research in the field of 3D MIR. Further it highlights the challenges in 3-D MIR on social media and finally translates them into significant research directions.
Similar content being viewed by others
References
Kou, F., Du, J., He, Y., Ye, L.: Social network search based on semantic analysis and learning. CAAI Trans. Intell. Technol. 1(4), 293–302 (2016)
Tandera, T., Hendro, Suhartono, D., Wongso, R., Prasetio, Y.L.: Personality prediction system from Facebook users. Proc. Comput. Sci. 116, 604–611 (2017)
Araque, O., Corcuera-Platas, I., Sánchez-Rada, J.F., Iglesias, C.A.: Enhancing deep learning sentiment analysis with ensemble techniques in social applications. Expert Syst. Appl. 77, 236–246 (2017)
Guimaraes, R.G., Rosa, R.L., De Gaetano, D., Rodriguez, D.Z., Bressan, G.: Age groups classification in social network using deep learning. IEEE Access 5(c), 10805–10816 (2017)
Najafabadi, M.M., Villanustre, F., Khoshgoftaar, T.M., Seliya, N., Wald, R., Muharemagic, E.: Deep learning applications and challenges in big data analytics. J. Big Data 2(1), 1–21 (2015)
Phan, N., Dou, D., Wang, H., Kil, D., Piniewski, B.: Ontology-based deep learning for human behavior prediction with explanations in health social networks. Inf. Sci. (NY) 384, 298–313 (2017)
Zheng, X., Zeng, Z., Chen, Z., Yu, Y., Rong, C.: Detecting spammers on social networks. Neurocomputing 159(1), 27–34 (2015)
Vinay, A., Shekhar, V.S., Rituparna, J., Aggrawal, T., Murthy, K.N.B., Natarajan, S.: Cloud based big data analytics framework for face recognition in social networks using machine learning. Proc. Comput. Sci. 50, 623–630 (2015)
Arel, I., Rose, D.C., Karnowski, T.P.: Deep machine learning—a new frontier in artificial intelligence research. IEEE Comput. Intell. Mag. 5(4), 13–18 (2010)
Bengio, Y.: Learning deep architectures for AI. Found. Trends® Mach. Learn. 2(1), 1–127 (2009)
Seraj, R.M.: Multi-task learning (2014)
Chen, X.-W., Lin, X.: Big data deep learning: challenges and perspectives. IEEE Access 2, 514–525 (2014)
Mesnil, G., Dauphin, Y., Glorot, X., Rifai, S., Bengio, Y., Goodfellow, I.J., Lavoie, E., Muller, X., Desjardins, G., Warde-Farley, D., Vincent, P., Courville, A., Bergstra, J.: Unsupervised and transfer learning challenge: a deep learning approach. JMLR W& CP Proc. Unsuperv. Transf. Learn. Chall. Work. 27, 97–110 (2012)
Phan, N., Dou, D., Wang, H., Kil, D., Piniewski, B.: Ontology-based deep learning for human behavior prediction in health social networks. In: Proceedings of the 6th ACM Conference on Bioinformatics, Computational Biology Health Informatics, pp. 433–442 (2015)
Ghodrati, H., Ben Hamza, A.: Deep shape-aware descriptor for nonrigid 3D object retrieval. Int. J. Multimed. Inf. Retr. 5(3), 151–164 (2016)
Ain, Q.T., Ali, M., Riaz, A., Noureen, A., Kamran, M., Hayat, B., Rehman, A.: Sentiment analysis using deep learning techniques: a review, IJACSA). Int. J. Adv. Comput. Sci. Appl. 8(6), 424–433 (2017)
Chen, T., Xu, R., He, Y., Wang, X.: Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN. Expert Syst. Appl. 72, 221–230 (2017)
Vateekul, P., Koomsubha, T.: A study of sentiment analysis using deep learning techniques on Thai Twitter data. In: 2016 13th International Joint Conference on Computer Science Software Engineering, pp. 1–6 (2016)
Abbes, M., Kechaou, Z., Alimi, A.M.: Enhanced deep learning models for sentiment analysis in arab social media. In: Lecture Notes in Computer Science (including subseries Lecture Notes Artificial Intelligence, Lecture Notes Bioinformatics), vol. 10638, pp. 667–676 (2017)
Liao, L., He, X., Zhang, H., Chua, T.-S.: Attributed Social Network Embedding, vol. 14(8), pp. 1–12 (2017)
Pal, S., Dong, Y., Thapa, B., Chawla, N.V., Swami, A., Ramanathan, R.: Deep learning for network analysis: problems, approaches and challenges. In: MILCOM 2016—2016 IEEE Military Communications Conference, pp. 588–593 (2016)
Jia, Y., Song, X., Zhou, J., Liu, L., Nie, L., Rosenblum, D.S.: Fusing social networks with deep learning for volunteerism tendency prediction. In: Proceedings of the 13th AAAI Conference on Artificial Intelligence, pp. 165–171 (2016)
Nguyen, D.T., Joty, S., Imran, M., Sajjad, H., Mitra, P.: Applications of Online Deep Learning for Crisis Response Using Social Media Information (2016)
Yao, L., Wang, L., Pan, L., Yao, K.: Link prediction based on common-neighbors for dynamic social network. Proc. Comput. Sci. 83, 82–89 (2016)
Zhou Zhonghua, X.J., Huiran, Z.: Data crawler for Sina Weibo based on Python. J. Comput. Appl. 34(11), 3131–3134 (2014)
Yang, C., Wang, Y.: Online social network image classification and application based o n deep learning. ICETA 2016, 41–46 (2016)
Hanafiah, N., Kevin, A., Sutanto Fiona, C., Arifin, Y., Hartanto, J.: Text normalization algorithm on Twitter in complaint category. Proc. Comput. Sci. 116, 20–26 (2017)
Lu, Y., Sakamoto, K., Shibuki, H., Mori, T.: Are deep learning methods better for Twitter sentiment analysis ? 言語処理学会 第23回年次大会 発表論文集 C, 787–790 (2017)
Atrey, P.K., Kankanhalli, M.S., Jain, R.: Information assimilation framework for event detection in multimedia surveillance systems. Multimed. Syst. 12(3), 239–253 (2006)
Chen, Y., Rui, Y.: Real-time speaker tracking using particle filter sensor fusion. Proc. IEEE 92(3), 485–494 (2004)
Datta, R., Li, J., Wang, J.Z.: Content-based image retrieval—approaches and trends of the new age. In: Proceedings of the 7th ACM SIGMM International Workshop on Multimedia Information Retrieval—MIR’05, p. 253 (2005)
Schreck, T.: 3D object. IEEE Comput. Graph. Appl. 27(4), 22–27 (2007)
Murnane, K.: Thirteen Companies that Use Deep Learning to Produce Actionable Results (online). https://www.forbes.com/sites/kevinmurnane/2016/04/01/thirteen-companies-that-use-deep-learning-to-produce-actionable-results/#127cea8033b8. Accessed: 16 Jan 2018 (2016)
Trovati, M., Hill, R., Anjum, A., Zhu, S.Y., Liu, L.: Big-data analytics and cloud computing: theory, algorithms and applications. Big-Data Anal. Cloud Comput. Theory, Algorithms Appl. 2016, i–xvi (2016)
Hanjalic, A.: New grand challenge for multimedia information retrieval: bridging the utility gap. Int. J. Multimed. Inf. Retr. 1(3), 139–152 (2012)
Rüger, S.: Multimedia Information Retrieval, vol. 1(1) (2010)
Raieli, R.: Multimedia Information Retrieval : Theory and Techniques. Chandos Publishing, Oxford (2013)
Zhu, W.: Visions and views multimedia big data computing. IEEE Multimed. 22(3), 96–105 (2015)
Abraham, A.: Computational social networks: mining and visualization. Comput. Soc. Netw. Min. Vis. 9781447140, 1–385 (2012)
Lee, H., Largman, Y., Pham, P., Ng, A.: Unsupervised feature learning for audio classification using convolutional deep belief networks. Adv. Neural Inf. Process. Syst. 22, 1096–1104 (2009)
Kereliuk, C., Sturm, B.L., Larsen, J.: Deep learning and music adversaries. IEEE Trans. Multimed. 17(11), 2059–2071 (2015)
Lao, J., Chen, Y., Li, Z.C., Li, Q., Zhang, J., Liu, J., Zhai, G.: A deep learning-based radiomics model for prediction of survival in glioblastoma multiforme. Sci. Rep. 7(1), 1–8 (2017)
Li, Z., Zou, D., Xu, S., Ou, X., Jin, H., Wang, S., Deng, Z., Zhong, Y.: VulDeePecker: A Deep Learning-Based System for Vulnerability Detection. NDSS (2018)
Korzeniowski, F., Widmer, G.: Feature Learning for Chord Recognition: The Deep Chroma Extractor (2016)
Sun, J.-Q., Lee, S.-P.: Query by singing/humming system based on deep learning. Int. J. Appl. Eng. Res. 12(13), 973–4562 (2017)
Isin, A., Ozdalili, S.: ScienceDirect cardiac arrhythmia detection using deep learning. Proc. Comput. Sci. 120, 268–275 (2017)
Lee, C.S., Tyring, A.J., Deruyter, N.P., Wu, Y., Rokem, A., Lee, A.Y.: Deep-learning based, automated segmentation of macular edema in optical coherence tomography. Biomed. Opt. Express 8(7), 3440 (2017)
Zhang, S., Yao, L., Sun, A.: Deep learning based recommender system: a survey and new perspectives. ACM Comput. Surv. 1(1), 1–35 (2017)
Paltz, N.: Metric learning based data augmentation for environmental sound classification. Department of Electronics, State Key Lab of Intelligent Technologies and Systems Tsinghua National Laboratory for Information Science and Technology (TNList), Beijing, pp. 1–5 (2017)
Monti, F., Boscaini, D., Masci, J., Rodolà, E., Svoboda, J., Bronstein, M.M.: Geometric deep learning on graphs and manifolds using mixture model CNNs (2016)
Khasanova, R., Frossard, P.: Graph-Based Classification of Omnidirectional Images 1 Introduction 2 Related work, pp. 869–878
Raffel, C.: Learning-Based Methods for Comparing Sequences, with Applications to Audio-to-MIDI Alignment and Matching, p. 222 (2016)
Luo, S., Li, X., Li, J.: Automatic Alzheimer’s disease recognition from MRI data using deep learning method. J. Appl. Math. Phys. 5(9), 1892–1898 (2017)
Saiyeda, A.: Cloud computing for deep learning analytics : a survey of current trends and challenges. Int. J. Adv. Res. Comput. Sci. 8(2), 68–72 (2017)
Nakamura, N., Takano, S., Okada, Y.: 3D multimedia data search system based on stochastic ARG matching method. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence, Lecture Notes in Bioinformatics), vol. 5371, pp. 379–389 (2009)
Tangelder, J.W.H., Veltkamp, R.C.: A survey of content based 3D shape retrieval methods. Multimed. Tools Appl. 39(3), 441–471 (2008)
Iyer, N., Jayanti, S., Lou, K., Kalyanaraman, Y., Ramani, K.: Three-dimensional shape searching: state-of-the-art review and future trends. Comput. Des. 37(5), 509–530 (2005)
Havemann, S., Fellner, D.W.: Seven research challenges of generalized 3D documents. IEEE Comput. Graph. Appl. 27(3), 70–76 (2007)
Bustos, B., Keim, D., Saupe, D., Schreck, T., Vrani, D.: An experimental effectiveness comparison of methods for 3D similarity search. Int. J. Digit. Libr. 6(1), 39–54 (2006)
Böhm, C., Berchtold, S., Keim, D.A.: Searching in high-dimensional spaces: index structures for improving the performance of multimedia databases. ACM Comput. Surv. 33(3), 322–373 (2001)
Bustos, B., Keim, D.A., Saupe, D., Schreck, T., Vranić, D.V.: Feature-based similarity search in 3D object databases. ACM Comput. Surv. 37(4), 345–387 (2005)
Rea, H.J., Corney, J.R., Clark, D.E.R., Taylor, N.K.: A surface partitioning spectrum (SPS) for retrieval and indexing of 3D CAD models. In: Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004, pp. 167–174
Saito, S., Li, T., Li, H.: Real-time facial segmentation and performance capture from RGB input. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence, Lecture Notes in Bioinformatics), vol. 9912(1), pp. 244–261 (2016)
Santana, J.M., Wendel, J., Trujillo, A., Suárez, J.P., Simons, A., Koch, A.: Progress in Location-Based Services 2016, pp. 329–353 (2017)
Castellani, U., Cortelazzo, G.M., Cristani, M., Delponte, E., Fusiello, A., Giachetti, A., Mizzaro, S., Odone, F., Puppo, E., Scateni, R., Zanuttigh, P.: 3-SHIRT : three-dimensional shape indexing and retrieval techniques. In: Eurographics Italian Chapter Conference, pp. 113–120 (2008)
Chatfield, K., Arandjelović, R., Parkhi, O., Zisserman, A.: On-the-fly learning for visual search of large-scale image and video datasets. Int. J. Multimed. Inf. Retr. 4(2), 75–93 (2015)
Funkhouser, T., Min, P., Kazhdan, M., Chen, J., Halderman, A., Dobkin, D., Jacobs, D.: A search engine for 3D models. ACM Trans. Graph. 22(1), 83–105 (2003)
Min, P., Funkhouser, T.: A 3D model search engine. Comput. Sci. 2004, 139 (2004)
Shilane, P., Min, P., Kazhdan, M., Funkhouser, T.: The Princeton shape benchmark. Proc. SMI 8540, 167–178 (2004)
Biasotti, S., Giorgi, D., Marini, S., Spagnuolo, M., Falcidieno, B.: A Comparison Framework for 3D Object Classification Methods
Nehab, D., Shilane, P.: Stratified point sampling of 3D models. In: Proceedings of the First Eurographics Conference on Point-Based Graphics, pp. 49–56 (2004)
Besl, P.J., Jain, R.C.: Three-dimensional object recognition. ACM Comput. Surv. 17(1), 75–145 (1985)
Guo, Y., Liu, Y., Georgiou, T., Lew, M.S.: A review of semantic segmentation using deep neural networks. Int. J. Multimed. Inf. Retr. 7, 1–7 (2017)
Kankanhalli, M.S., Rui, Y.: Application potential of multimedia information retrieval. Proc. IEEE 96(4), 712–720 (2008)
Lew, M.S., Sebe, N., Djeraba, C., Jain, R.: Content-based multimedia information retrieval: state of the art and challenges. ACM Trans. Multimed. Comput. Commun. Appl. 2(1), 1–19 (2006)
Moise, D., Shestakov, D., Gudmundsson, G., Amsaleg, L.: Indexing and searching 100 M images with map-reduce. In: Proceedings of the 3rd ACM International Conference on Multimedia Retrieval—ICMR’13, p. 17 (2013)
Krishna, M., Kannan, B., Ramani, A., CloudCom, 2010 IEEE, and undefined 2010: Implementation and performance evaluation of a hybrid distributed system for storing and processing images from the web. ieeexplore.ieee.org
He, B., Fang, W., Luo, Q., Govindaraju, N.K., Wang, T.: Mars: a MapReduce framework on graphics processors. In: Proceedings of the 17th International Conference on Parallel Architecture and Compilation Techniques—PACT’08, p. 260 (2008)
Chen, S.Y., Lai, C.F., Hwang, R.H., Chao, H.C., Huang, Y.M.: A multimedia parallel processing approach on GPU MapReduce framework. In: Proceedings of the 2014 7th International Conference on Ubi-Media Computing and Workshops, U-MEDIA 2014, pp. 154–159 (2014)
Wang, H., Shen, Y., Wang, L., Zhufeng, K., Wang, W., Cheng, C.: /12/$31.00 ©2012 IEEE large-scale multimedia data mining using MapReduce framework. In: 2012 IEEE 4th International Conference on Cloud Computing Technology Science, pp. 978–1 (2012)
Mera, D., Batko, M., Zezula, P.: Towards fast multimedia feature extraction: Hadoop or storm. In: Proceedings of the 2014 IEEE International Symposium Multimedia, ISM 2014, pp. 106–109 (2015)
Zhang, C., Chen, T.: Indexing and retrieval of 3D models aided by active learning. In: Proceedings of the Ninth ACM International Conference on Multimedia—MULTIMEDIA’01, p. 615 (2001)
Gao, Y., Dai, Q.: View-Based 3-D Object Retrieval. Elsevier, New York (2014)
Vranic, D.V., Saupe, D., Richter, J.: Tools for 3D-object retrieval: Karhunen–Loeve transform and spherical harmonics. In: 2001 IEEE Fourth Workshop on Multimedia Signal Processing (Cat. No. 01TH8564), pp. 293–298 (2001)
Hilaga, M., Shinagawa, Y., Kohmura, T., Kunii, T.L.: Topology matching for fully automatic similarity estimation of 3D shapes. In: Proceedings of the 28th Annual Conference on Computing Graphics Interactive Techniques—SIGGRAPH’01, pp. 203–212 (2001)
Laga, H., Takahashi, H., Nakajima, M.: Geometry image based similarity estimation for 3D model retrieval. In: Nicograph International 2004, Taiwan, pp. 133–138 (2004)
Choi, S.-M., Kim, Y.-G.: Similarity Estimation of 3D Shapes Using Modal Strain Energy, pp. 206–212. Springer, Berlin (2005)
Sánchez-Cruz, H., Bribiesca, E.: A method of optimum transformation of 3D objects used as a measure of shape dissimilarity. Image Vis. Comput. 21(12), 1027–1036 (2003)
Shum, H.-Y., Hebert, M., Ikeuchi, K.: On 3D Shape Similarity (1995)
Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)
Cass, T.A.: Robust affine structure matching for 3D object recognition. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1265–1274 (1998)
Flynn, P.J., Jain, A.K.: BONSAI: 3D object recognition using constrained search. IEEE Trans. Pattern Anal. Mach. Intell. 13(10), 1066–1075 (1991)
Wang, W., Iyengar, S.S.: Efficient data structures for model-based 3-D object recognition and localization from range images. IEEE Trans. Pattern Anal. Mach. Intell. 14(10), 1035–1045 (1992)
Zhou, Y., Kaufman, A., Toga, A.W.: Three-dimensional skeleton and centerline generation based on an approximate minimum distance field. Vis. Comput. 14(7), 303–314 (1998)
Modemlis, A., Daras, P., Tzovaras, D., Strintzis, M.G. :On 3D partial matching of meaningful parts. In: Proceedings of the International Conference on Image Processing ICIP, vol. 2, pp. 1–4 (2007)
Funkhouser, T., Shilane, P.: Partial matching of 3D shapes with priority-driven search. In: Proceedings of the fourth Eurographics Symposium on Geometry Processing, pp. 131–142 (2006)
Ioannides, M., Quak, E.: 3D research challenges in cultural heritage : a roadmap in digital heritage preservation
Münster, S., Pfarr-Harfst, M., Kuroczyński, P., Ioannides, M.: 3D research challenges in cultural heritage II : how to manage data and knowledge related to interpretative digital 3D reconstructions of cultural heritage
Dugelay, J.-L., Baskurt, A., Daoudi, M.: 3D Object Processing : Compression, Indexing and Watermarking. Wiley, New York (2008)
Kuroczyński, P., Hauck, O., Dworak, D.: 3D Models on Triple Paths—New Pathways for Documenting and Visualizing Virtual Reconstructions, pp. 149–172 (2016)
von Schwerin, J., Lyons, M., Loos, L., Billen, N., Auer, M., Zipf, A.: Show Me the Data!: Structuring Archaeological Data to Deliver Interactive, Transparent 3D Reconstructions in a 3D WebGIS, pp. 198–230 (2016)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wason, R., Jain, V., Narula, G.S. et al. Deep understanding of 3-D multimedia information retrieval on social media: implications and challenges. Iran J Comput Sci 2, 101–111 (2019). https://doi.org/10.1007/s42044-019-00030-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42044-019-00030-5