Abstract
Automatic image annotation is a technique that can be used to quickly generate tags for a massive dataset based on the content of the images. Nearest-neighbor-based methods such as TagProp are successful methods which have been used for image annotation. However, these methods focus more on weights based on the distances between the images and their neighbors, and ignore the weights of the different labels which can co-occur in the same image. In this paper, an improved method is proposed for automatic semantic annotation of images, which tags rare labels more effectively by processing the label matrix of the training set. In addition, image segmentation and data-driven methods are adopted to provide differential weights to the tags in one image, to improve the accuracy of the predicted tags. Experimental results show that the proposed method outperforms many classical baseline methods and can generate better annotation results than state-of-the-art nearest-neighbor based methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Mensink, T., Verbeek, J., Schmid, C., Guillaumin, M.: TagProp: discriminative metric learning in nearest neighbor models for image auto-annotation. In: IEEE International Conference on Computer Vision, pp. 309–316 (2010)
Yang, J.Y., Liu, G.H.: Content-based image retrieval using color difference histogram. Pattern Recogn. 46(1), 188–198 (2013)
Chang, S.F., Kozintsev, I.V., Kennedy, L.S.: To search or to label?: predicting the performance of search-based automatic image classifiers. In: ACM International Workshop on Multimedia, Information Retrieval, pp. 249–258 (2006)
Shimada, A., Nagahara, H., Taniguchi, R.I., Xu, X.: Learning multi-task local metrics for image annotation. Multimed. Tools Appl. 75(4), 1–29 (2014)
Lavrenko, V., Manmatha, R., Jeon, J.: Automatic image annotation and retrieval using cross-media relevance models. In: International ACM SIGIR Conference on Research and Development in Informaion Retrieval, pp. 119–126 (2003)
Manmatha, R., Jeon, J., Lavrenko, V.: A model for learning the semantics of pictures. In: NIPS, pp. 553–560 (2003)
Manmatha, R., Lavrenko, V., Feng, S.L.: Multiple Bernoulli relevance models for image and video annotation. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 2, pp. II-1002–II-1009 (2004)
Chan, A.B., Moreno, P.J., Vasconcelos, N., Carneiro, G.: Supervised learning of semantic classes for image annotation and retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 29(3), 394–410 (2007)
Jawahar, C.V., Verma, Y.: Exploring SVM for image annotation in presence of confusing labels. In: British Machine Vision Conference, pp. 25.1–25.11 (2013)
Pavlovic, V., Kumar, S., Makadia, A.: Baselines for image annotation. Int. J. Comput. Vis. 90(1), 88–105 (2010)
Maji, S., Manmatha, R., Murthy, V.N.: Automatic image annotation using deep learning representations. In: ACM on International Conference on Multimedia Retrieval, pp. 603–606 (2015)
Wang, Z., et al.: Weakly semi-supervised deep learning for multi-label image annotation. IEEE Trans. Big Data 1(3), 109–122 (2015)
Chandra Sekhar, C., Sarangi, N.: Automatic image annotation using convex deep learning models. In: International Conference on Pattern Recognition Applications and Methods, pp. 92–99 (2015)
Wang, X.J., Zhang, L., Jing, F., Ma, W.Y.: AnnoSearch: image auto-annotation by search. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1483–1490 (2006)
Zhang, L., Wang, X.-J., Ma, W.-Y., Li, X.: Annotating images by mining image search results. IEEE Trans. Pattern Anal. Mach. Intell. 30(11), 1919 (2008)
Hemami, S., Estrada, F., Susstrunk, S., Achanta, R.: Frequency-tuned salient region detection. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 1597–1604 (2009)
Irani, M., Faktor, A.: Co-segmentation by composition. In: IEEE International Conference on Computer Vision, pp. 1297–1304 (2014)
Metzler, D., Manmatha, R.: An inference network approach to image retrieval. In: Enser, P., Kompatsiaris, Y., O’Connor, N.E., Smeaton, A.F., Smeulders, A.W.M. (eds.) CIVR 2004. LNCS, vol. 3115, pp. 42–50. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-27814-6_9
Huang, J., Li, H., Metaxas, D.N., Zhang, S.: Automatic image annotation and retrieval using group sparsity. IEEE Trans. Syst. Man Cybern. Part B Cybern. Publ. IEEE Syst. Man Cybern. Soc. 42(3), 838 (2012)
Fu, H., Zhang, Q., Qiu, G.: Random forest for image annotation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 86–99. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33783-3_7
Acknowledgement
This work was supported by the National Natural Science Foundation of China (61572060, 61772060) and CERNET Innovation Project (NGII20151004, NGII20160316).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Niu, J., Li, S., Mo, S., Ma, J. (2018). LWTP: An Improved Automatic Image Annotation Method Based on Image Segmentation. In: Romdhani, I., Shu, L., Takahiro, H., Zhou, Z., Gordon, T., Zeng, D. (eds) Collaborative Computing: Networking, Applications and Worksharing. CollaborateCom 2017. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 252. Springer, Cham. https://doi.org/10.1007/978-3-030-00916-8_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-00916-8_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-00915-1
Online ISBN: 978-3-030-00916-8
eBook Packages: Computer ScienceComputer Science (R0)