Abstract
Extended from the traditional pure statistical learning methods, we propose to augment the statistical learning methods with ontology and apply this idea for image attribute learning. In order to capture structural information among attributes, the graph-guided fused lasso model is adopted and improved by a new distance metric based on WordNet. The novelty of our method is that we find the semantic correlation with the ontology-guided attribute space and integrate inter-attribute similarity information into the learning model. The hierarchy of ImageNet is exploited to define the image attributes and a dataset from ImageNet including over 30,000 images is collected. The experimental results show that this method can both improve the accuracy and accelerate the algorithm convergency. Moreover, the learned semantic correlation owns transfer ability to related applications.
Similar content being viewed by others
References
Benitez AB, Chang SF (2003) Image classification using multimedia knowledge networks. In: Image Processing, 2003. ICIP 2003. Proceedings. 2003 International Conference on, vol 3, pp III–613–16 vol 2. IEEE
Breen C, Khan L, Ponnusamy A (2002) Image classification using neural networks and ontologies. In: Database and Expert Systems Applications, 2002. Proceedings. 13th International Workshop on, pp 98–102. IEEE
Chen X, Lin Q, Kim S, Carbonell JG, Xing EP (2012) Smoothing proximal gradient method for general structured sparse regression. Ann Appl Stat 6(2):719–752
Clancey WJ (1993) The knowledge level reinterpreted: Modeling sociotechnical systems. Int J Intell Syst 8(1):33–49
Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database. In: Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pp 248–255. IEEE
Eric Maillot N, Thonnat M (2008) Ontology based complex object recognition. Image Vis Comput 26(1):102–113
Fan J, Gao Y, Luo H (2008) Integrating concept ontology and multitask learning to achieve more effective classifier training for multilevel image annotation. IEEE Trans Image Process 17(3):407–426
Farhadi A, Endres I, Hoiem D, Forsyth D (2009) Describing objects by their attributes. In: Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pp 1778–1785. IEEE
Fawcett T (2006) An introduction to roc analysis. Pattern Recogn Lett 27(8):861–874
Ferrari V, Zisserman A (2007) Learning visual attributes. In: Advances in Neural Information Processing Systems, pp 433–440
Gruber TR (1993) A translation approach to portable ontology specifications. Knowl Acquis 5(2):199–220
Guarino N (1995) Formal ontology, conceptual analysis and knowledge representation. Int J Hum Comput Stud 43(5):625–640
Han Y, Wu F, Lu X, Tian Q, Zhuang Y, Luo J (2012) Correlated attribute transfer with multi-task graph-guided fusion. In: Proceedings of the 20th ACM international conference on Multimedia, pp 529–538. ACM
Hwang SJ, Sha F, Grauman K (2011) Sharing features between objects and their attributes. In: Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pp 1761–1768. IEEE
Jolicoeur P, Gluck MA, Kosslyn SM (1984) Pictures and names: Making the connection. Cogn Psychol 16(2):243–275
Kankuekul P, Kawewong A, Tangruamsub S, Hasegawa O (2012) Online incremental attribute-based zero-shot learning. In: Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pp 3657–3664. IEEE
Kim S, Sohn KA, Xing EP (2009) A multivariate regression approach to association analysis of a quantitative trait network. Bioinformatics 25(12):i204—i212
Kumar N, Berg AC, Belhumeur PN, Nayar SK (2009) Attribute and simile classifiers for face verification. In: Computer Vision, 2009 IEEE 12th International Conference on, pp 365–372. IEEE
Lampert CH, Nickisch H, Harmeling S (2009) Learning to detect unseen object classes by between-class attribute transfer. In: Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pp 951–958. IEEE
Li LJ, Socher R, Fei-Fei L (2009) Towards total scene understanding: Classification, annotation and segmentation in an automatic framework. In: Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pp 2036–2043. IEEE
Liu J, Kuipers B, Savarese S (2011) Recognizing human actions by attributes. In: Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pp 3337–3344. IEEE
Mahajan D, Sellamanickam S, Nair V (2011) A joint learning framework for attribute models and object descriptions. In: Computer Vision (ICCV), 2011 IEEE International Conference on, pp 1227–1234. IEEE
Marszalek M, Schmid C (2007) Semantic hierarchies for visual object recognition. In: Computer Vision and Pattern Recognition, 2007. CVPR’07. IEEE Conference on, pp 1–7. IEEE
Mezaris V, Kompatsiaris I, Strintzis MG (2003) An ontology approach to object-based image retrieval. In: Image Processing, 2003. ICIP 2003. Proceedings. 2003 International Conference on, vol 2, pp II–511–14 vol 3. IEEE
Russakovsky O, Fei-Fei L (2012) Attribute learning in large-scale datasets. In: Trends and Topics in Computer Vision, pp 1–14. Springer
Sharmanska V, Quadrianto N, Lampert CH (2012) Augmented attribute representations. In: Computer VisionECCV 2012, vol 7576, pp 242–255. Springer
Shi R, Lee CH, Chua TS (2007) Enhancing image annotation by integrating concept ontology and text-based bayesian learning model. In: Proceedings of the 15th international conference on Multimedia, pp 341–344. ACM
Siddiquie B, Feris RS, Davis LS (2011) Image ranking and retrieval based on multi-attribute queries
Srikanth M, Varner J, Bowden M, Moldovan D (2005) Exploiting ontologies for automatic image annotation. In: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, pp 552–558. ACM
Srikanth M, Varner J, Bowden M, Moldovan D (2005) Exploiting ontologies for automatic image annotation. In: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, pp 552–558. ACM
Tibshirani R (2011) Regression shrinkage and selection via the lasso: a retrospective. J R Stat Soc Ser B (Statistical Methodology) 73(3):273–282
Tousch AM, Herbin S, Audibert JY (2012) Semantic hierarchies for image annotation: A survey. Pattern Recogn Lett 45(1):333–345
Wang C, Yan S, Zhang HJ (2009) Large scale natural image classification by sparsity exploration. In: Acoustics, Speech and Signal Processing, 2009. ICASSP 2009. IEEE International Conference on, pp 3709–3712. IEEE
Wang M, Yang K, Hua XS, Zhang HJ (2010) Towards a relevant and diverse search of social images. IEEE Trans Multimedia 12(8):829–842
Wang Y, Mori G (2010) A discriminative latent model of object classes and attributes. In: Computer Vision ECCV 2010, vol 6315, pp 155–168. Springer
Yu FX, Cao L, Feris RS, Smith JR, Chang SF (2013) Designing category-level attributes for discriminative visual recognition. In: Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, pp 771–778. IEEE
Yu FX, Ji R, Tsai MH, Ye G, Chang SF (2012) Weak attributes for large-scale image retrieval. In: Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pp 2949–2956. IEEE
Yu X, Aloimonos Y (2010) Attribute-based transfer learning for object categorization with zero/one training example. In: Computer VisionECCV 2010, pp 127–140. Springer
Zhang H, Zha ZJ, Yang Y, Yan S, Gao Y, Chua TS (2013) Attribute-augmented semantic hierarchy: towards bridging semantic gap and intention gap in image retrieval. In: Proceedings of the 21st ACM international conference on Multimedia, pp 33–42. ACM
Acknowledgement
This work was partly supported by the NSFC (under Grant 61202166, 61472276) and Doctoral Fund of Ministry of Education of China (under Grant 20120032120042).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Li, C., Feng, Z. & Han, Y. Image attribute learning with ontology guided fused lasso. Multimed Tools Appl 75, 7029–7043 (2016). https://doi.org/10.1007/s11042-015-2630-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-015-2630-5