PERIA-Framework: A Prediction Extension Revision Image Annotation Framework

  • Umer RashidEmail author
  • Bakhtawar Arif
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 738)


Image annotation is a procedure to interpret the semantic concepts associated with image objects and represent them as their textual descriptions. Automatic and manual techniques have been extensively discussed in recent years to annotate the image objects, but are not without limitations. Automatic image annotation techniques mainly consider a single classifier and a descriptor type to annotate the image objects. Furthermore, thesaurus based extensions and human-centered revisions of the annotations are usually not possible. The fine-tuning of classifiers is generally not supported. In contrast to this, manual image annotation improves the accuracy, but tedious to annotate huge collections of image objects. Alternatively, semi-automatic image annotation techniques are human-centered, enhances the efficiency, and also speed-up the annotation process by machine intervention. In this research, a semi-automatic image annotation framework is proposed to address limitations in automatic and manual image annotation techniques. Our image annotation framework considers multiple descriptors and artificial neural networks to annotate the image objects. Along with that, a voting mechanism is provided to recommend the suitable annotations extendible by thesaurus and human revisions. Revised and extended annotations employed further to fine-tune the classifiers. Image annotation framework is instantiated and tested on a real dataset by implementing an image annotation tool.


Annotation Descriptors Extension Framework Image Prediction Revision Semi-automatic Tool 



The authors would like to acknowledge the provision of the research facilities provided by Department of Computer Sciences, Quaid-i-Azam University, Islamabad, Pakistan to carry out this research work. Authors would also acknowledge the finical support offered by Higher Education Commission, Pakistan to present this research work in the conference.


  1. 1.
    M.G. Van Doorn, A.P. de Vries, The psychology of multimedia databases, in Proceedings of the Fifth ACM Conference on Digital Libraries (ACM, New York, 2000), pp. 1–9Google Scholar
  2. 2.
    M. Kolodnytsky, N.O. Bernsen, L. Dybkjær, A visual interface for a multimodal interactivity annotation tool: design issues and implementation solutions, in Proceedings of the Working Conference on Advanced Visual Interfaces (ACM, New York, 2004), pp. 407–410Google Scholar
  3. 3.
    R. Baeza-Yates, B. Ribeiro-Neto, et al., Modern Information Retrieval, vol. 463 (ACM, New York, 1999)Google Scholar
  4. 4.
    A. Hanbury, A survey of methods for image annotation. J. Vis. Lang. Comput. 19(5), 617–627 (2008)Google Scholar
  5. 5.
    Y. Jin, L. Khan, L. Wang, M. Awad, Image annotations by combining multiple evidence & wordnet, in Proceedings of the 13th Annual ACM International Conference on Multimedia (ACM, New York, 2005), pp. 706–715Google Scholar
  6. 6.
    P.G. Enser, C.J. Sandom, P.H. Lewis, Automatic annotation of images from the practitioner perspective, in Proceedings of International Conference on Image and Video Retrieval (Springer, Cham, 2005), pp. 497–506Google Scholar
  7. 7.
    J. Li, J.Z. Wang, Real-time computerized annotation of pictures. IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 985–1002 (2008)Google Scholar
  8. 8.
    P. Rogelj, S. Kovacic, Local similarity measures for multimodal image matching, in Proceedings of the First International Workshop on Image and Signal Processing and Analysis (IEEE, Piscataway, 2000), pp. 81–86Google Scholar
  9. 9.
    K. Zagoris, S.A. Chatzichristofis, N. Papamarkos, Y.S. Boutalis, Automatic image annotation and retrieval using the joint composite descriptor, in Proceedings of 14th Panhellenic Conference on Informatics (PCI) (IEEE, Piscataway, 2010), pp. 143–147Google Scholar
  10. 10.
    O.A. Penatti, E. Valle, R.d.S. Torres, Comparative study of global color and texture descriptors for web image retrieval. J. Vis. Commun. Image Represent. 23(2), 359–380 (2012)Google Scholar
  11. 11.
    A. Yavlinsky, E. Schofield, S.M. Rüger, Automated image annotation using global features and robust nonparametric density estimation, in Proceedings of International Conference on Image and Video Retrieval, vol. 3568 (Springer, Cham, 2005), pp. 507–517Google Scholar
  12. 12.
    Y. Zhao, Y. Zhao, Z. Zhu, J.-S. Pan, A novel image annotation scheme based on neural network, in Proceedings of Eighth International Conference on Intelligent Systems Design and Applications, vol. 3 (IEEE, Piscataway, 2008), pp. 644–647Google Scholar
  13. 13.
    L. Jiang, J. Hou, Z. Chen, D. Zhang, Automatic image annotation based on decision tree machine learning, in Proceedings of International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (IEEE, Piscataway, 2009), pp. 170–175Google Scholar
  14. 14.
    S. Rui, W. Jin, T.-S. Chua, A novel approach to auto image annotation based on pairwise constrained clustering and semi-naive bayesian model, in Proceedings of 11th International Conference on Multimedia Modeling (IEEE, Piscataway, 2005), pp. 322–327Google Scholar
  15. 15.
    M. Dong, C. Yang, F. Fotouhi, I2a: an interactive image annotation system, in Proceedings of IEEE International Conference on Multimedia and Expo (IEEE, Piscataway, 2005), pp. 1–4Google Scholar
  16. 16.
    B. Broda, H. Kwasnicka, M. Paradowski, M. Stanek, Magma—efficient method for image annotation in low dimensional feature space based on multivariate Gaussian models, in Proceeding of International Multiconference on Computer Science and Information Technology (IEEE, Piscataway, 2009), pp. 131–138Google Scholar
  17. 17.
    O.O. Karadag, F.T.Y. Vural, Hanolistic: a hierarchical automatic image annotation system using holistic approach, in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (IEEE, Piscataway, 2009), pp. 16–21Google Scholar
  18. 18.
    P. Koletsis, E.G. Petrakis, Sia: semantic image annotation using ontologies and image content analysis, in Proceedings of International Conference Image Analysis and Recognition (Springer, Cham, 2010), pp. 374–383Google Scholar
  19. 19.
    A. Kerne, E. Koh, B. Dworaczyk, J.M. Mistrot, H. Choi, S.M. Smith, R. Graeber, D. Caruso, A. Webb, R. Hill, et al., Combinformation: a mixed-initiative system for representing collections as compositions of image and text surrogates, in Proceedings of the 6th ACM and IEEE-CS Joint Conference on Digital Libraries (ACM, New York, 2006), pp. 11–20Google Scholar
  20. 20.
    C. Halaschek-Wiener, J. Golbeck, A. Schain, M. Grove, B. Parsia, J. Hendler, Photostuff-an image annotation tool for the semantic web, in Proceedings of the 4th International Semantic Web Conference (2005)Google Scholar
  21. 21.
    A. Torralba, B.C. Russell, J. Yuen, Labelme: online image annotation and applications. Proc. IEEE 98(8), 1467–1484 (2010)CrossRefGoogle Scholar
  22. 22.
    E. Skounakis, V. Sakkalis, K. Marias, K. Banitsas, N. Graf, Doctoreye: a multifunctional open platform for fast annotation and visualization of tumors in medical images, in Proceedings of Annual International Conference of the IEEE Engineering in Medicine and Biology Society International Conference of the IEEE (IEEE, Piscataway, 2009), pp. 3759–3762Google Scholar
  23. 23.
    H. Peng, F. Long, E.W. Myers, Vano: a volume-object image annotation system. Bioinformatics 25(5), 695–697 (2009)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Quaid-i-Azam UniversityIslamabadPakistan
  2. 2.iENGINEERING Pakistan (Private) LimitedIslamabadPakistan

Personalised recommendations