Skip to main content
Log in

Image attribute learning with ontology guided fused lasso

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Extended from the traditional pure statistical learning methods, we propose to augment the statistical learning methods with ontology and apply this idea for image attribute learning. In order to capture structural information among attributes, the graph-guided fused lasso model is adopted and improved by a new distance metric based on WordNet. The novelty of our method is that we find the semantic correlation with the ontology-guided attribute space and integrate inter-attribute similarity information into the learning model. The hierarchy of ImageNet is exploited to define the image attributes and a dataset from ImageNet including over 30,000 images is collected. The experimental results show that this method can both improve the accuracy and accelerate the algorithm convergency. Moreover, the learned semantic correlation owns transfer ability to related applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Benitez AB, Chang SF (2003) Image classification using multimedia knowledge networks. In: Image Processing, 2003. ICIP 2003. Proceedings. 2003 International Conference on, vol 3, pp III–613–16 vol 2. IEEE

  2. Breen C, Khan L, Ponnusamy A (2002) Image classification using neural networks and ontologies. In: Database and Expert Systems Applications, 2002. Proceedings. 13th International Workshop on, pp 98–102. IEEE

  3. Chen X, Lin Q, Kim S, Carbonell JG, Xing EP (2012) Smoothing proximal gradient method for general structured sparse regression. Ann Appl Stat 6(2):719–752

    Article  MathSciNet  MATH  Google Scholar 

  4. Clancey WJ (1993) The knowledge level reinterpreted: Modeling sociotechnical systems. Int J Intell Syst 8(1):33–49

    Article  Google Scholar 

  5. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database. In: Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pp 248–255. IEEE

  6. Eric Maillot N, Thonnat M (2008) Ontology based complex object recognition. Image Vis Comput 26(1):102–113

    Article  Google Scholar 

  7. Fan J, Gao Y, Luo H (2008) Integrating concept ontology and multitask learning to achieve more effective classifier training for multilevel image annotation. IEEE Trans Image Process 17(3):407–426

    Article  MathSciNet  Google Scholar 

  8. Farhadi A, Endres I, Hoiem D, Forsyth D (2009) Describing objects by their attributes. In: Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pp 1778–1785. IEEE

  9. Fawcett T (2006) An introduction to roc analysis. Pattern Recogn Lett 27(8):861–874

    Article  MathSciNet  Google Scholar 

  10. Ferrari V, Zisserman A (2007) Learning visual attributes. In: Advances in Neural Information Processing Systems, pp 433–440

  11. Gruber TR (1993) A translation approach to portable ontology specifications. Knowl Acquis 5(2):199–220

    Article  Google Scholar 

  12. Guarino N (1995) Formal ontology, conceptual analysis and knowledge representation. Int J Hum Comput Stud 43(5):625–640

    Article  Google Scholar 

  13. Han Y, Wu F, Lu X, Tian Q, Zhuang Y, Luo J (2012) Correlated attribute transfer with multi-task graph-guided fusion. In: Proceedings of the 20th ACM international conference on Multimedia, pp 529–538. ACM

  14. Hwang SJ, Sha F, Grauman K (2011) Sharing features between objects and their attributes. In: Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pp 1761–1768. IEEE

  15. Jolicoeur P, Gluck MA, Kosslyn SM (1984) Pictures and names: Making the connection. Cogn Psychol 16(2):243–275

    Article  Google Scholar 

  16. Kankuekul P, Kawewong A, Tangruamsub S, Hasegawa O (2012) Online incremental attribute-based zero-shot learning. In: Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pp 3657–3664. IEEE

  17. Kim S, Sohn KA, Xing EP (2009) A multivariate regression approach to association analysis of a quantitative trait network. Bioinformatics 25(12):i204—i212

    Article  Google Scholar 

  18. Kumar N, Berg AC, Belhumeur PN, Nayar SK (2009) Attribute and simile classifiers for face verification. In: Computer Vision, 2009 IEEE 12th International Conference on, pp 365–372. IEEE

  19. Lampert CH, Nickisch H, Harmeling S (2009) Learning to detect unseen object classes by between-class attribute transfer. In: Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pp 951–958. IEEE

  20. Li LJ, Socher R, Fei-Fei L (2009) Towards total scene understanding: Classification, annotation and segmentation in an automatic framework. In: Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pp 2036–2043. IEEE

  21. Liu J, Kuipers B, Savarese S (2011) Recognizing human actions by attributes. In: Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pp 3337–3344. IEEE

  22. Mahajan D, Sellamanickam S, Nair V (2011) A joint learning framework for attribute models and object descriptions. In: Computer Vision (ICCV), 2011 IEEE International Conference on, pp 1227–1234. IEEE

  23. Marszalek M, Schmid C (2007) Semantic hierarchies for visual object recognition. In: Computer Vision and Pattern Recognition, 2007. CVPR’07. IEEE Conference on, pp 1–7. IEEE

  24. Mezaris V, Kompatsiaris I, Strintzis MG (2003) An ontology approach to object-based image retrieval. In: Image Processing, 2003. ICIP 2003. Proceedings. 2003 International Conference on, vol 2, pp II–511–14 vol 3. IEEE

  25. Russakovsky O, Fei-Fei L (2012) Attribute learning in large-scale datasets. In: Trends and Topics in Computer Vision, pp 1–14. Springer

  26. Sharmanska V, Quadrianto N, Lampert CH (2012) Augmented attribute representations. In: Computer VisionECCV 2012, vol 7576, pp 242–255. Springer

  27. Shi R, Lee CH, Chua TS (2007) Enhancing image annotation by integrating concept ontology and text-based bayesian learning model. In: Proceedings of the 15th international conference on Multimedia, pp 341–344. ACM

  28. Siddiquie B, Feris RS, Davis LS (2011) Image ranking and retrieval based on multi-attribute queries

  29. Srikanth M, Varner J, Bowden M, Moldovan D (2005) Exploiting ontologies for automatic image annotation. In: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, pp 552–558. ACM

  30. Srikanth M, Varner J, Bowden M, Moldovan D (2005) Exploiting ontologies for automatic image annotation. In: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, pp 552–558. ACM

  31. Tibshirani R (2011) Regression shrinkage and selection via the lasso: a retrospective. J R Stat Soc Ser B (Statistical Methodology) 73(3):273–282

    Article  MathSciNet  Google Scholar 

  32. Tousch AM, Herbin S, Audibert JY (2012) Semantic hierarchies for image annotation: A survey. Pattern Recogn Lett 45(1):333–345

    Article  Google Scholar 

  33. Wang C, Yan S, Zhang HJ (2009) Large scale natural image classification by sparsity exploration. In: Acoustics, Speech and Signal Processing, 2009. ICASSP 2009. IEEE International Conference on, pp 3709–3712. IEEE

  34. Wang M, Yang K, Hua XS, Zhang HJ (2010) Towards a relevant and diverse search of social images. IEEE Trans Multimedia 12(8):829–842

    Article  Google Scholar 

  35. Wang Y, Mori G (2010) A discriminative latent model of object classes and attributes. In: Computer Vision ECCV 2010, vol 6315, pp 155–168. Springer

  36. Yu FX, Cao L, Feris RS, Smith JR, Chang SF (2013) Designing category-level attributes for discriminative visual recognition. In: Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, pp 771–778. IEEE

  37. Yu FX, Ji R, Tsai MH, Ye G, Chang SF (2012) Weak attributes for large-scale image retrieval. In: Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pp 2949–2956. IEEE

  38. Yu X, Aloimonos Y (2010) Attribute-based transfer learning for object categorization with zero/one training example. In: Computer VisionECCV 2010, pp 127–140. Springer

  39. Zhang H, Zha ZJ, Yang Y, Yan S, Gao Y, Chua TS (2013) Attribute-augmented semantic hierarchy: towards bridging semantic gap and intention gap in image retrieval. In: Proceedings of the 21st ACM international conference on Multimedia, pp 33–42. ACM

Download references

Acknowledgement

This work was partly supported by the NSFC (under Grant 61202166, 61472276) and Doctoral Fund of Ministry of Education of China (under Grant 20120032120042).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yahong Han.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, C., Feng, Z. & Han, Y. Image attribute learning with ontology guided fused lasso. Multimed Tools Appl 75, 7029–7043 (2016). https://doi.org/10.1007/s11042-015-2630-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-2630-5

Keywords

Navigation