Abstract
This paper presents a unified multimedia classification approach that integrates effectively visual and textual features. It combines the Bag of Visual Words model (BoVW) together with a generalized Bag of Colors (BoC) model and textual information in an early stage for modality detection of images in the medical domain. Our contribution is twofold: First we generalize the BoC model incorporating spatial information derived from a quad-tree decomposition of the images. Second we propose a weighted linear combination of word embeddings for the textual representation of the images. Experimental results conducted on the data of the ImageCLEF contest for the years 2011, 2012, 2013 and 2016 demonstrate the effectiveness and robustness of our framework in terms of classification accuracy outperforming all the published results so far on the aforementioned datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bosch, A., Zisserman, A., Muñoz, X.: Image classification using random forests and ferns (2007)
De Natale, F., Granelli, F.: Structured-based image retrieval using a structured color descriptor. In: International Workshop on Content-Based Multimedia Indexing (CBMI 2001), pp. 109–115 (2001)
Fan, R.E., Chang, K.W., Hsieh, C.J., Wang, X.R., Lin, C.J.: Liblinear: a library for large linear classification. J. Mach. Learn. Res. 9, 1871–1874 (2008)
Furuya, T., Ohbuchi, R.: Dense sampling and fast encoding for 3d model retrieval using bag-of-visual features. In: Proceedings of the ACM International Conference on image and video retrieval, p. 26. ACM (2009)
de Herrera, A.G.S., Kalpathy-Cramer, J., Demner-Fushman, D., Antani, S.K., Müller, H.: Overview of the imageCLEF 2013 medical tasks. In: Working Notes for CLEF 2013 Conference (2013)
de Herrera, A.G.S., Markonis, D., Müller, H.: Bag–of–colors for biomedical document image classification. In: Greenspan, H., Müller, H., Syeda-Mahmood, T. (eds.) MCBR-CDS 2012. LNCS, vol. 7723, pp. 110–121. Springer, Heidelberg (2013). doi:10.1007/978-3-642-36678-9_11
de Herrera, A.G.S., Schaer, R., Bromuri, S., Müller, H.: Overview of the imageCLEF 2016 medical task. In: Working Notes of CLEF 2016 Conference, pp. 219–232 (2016)
Jégou, H., Douze, M., Schmid, C.: Improving bag-of-features for large scale image search. Int. J. Comput. Vis. 87(3), 316–336 (2010)
Kalpathy-Cramer, J., Müller, H., Bedrick, S., Eggel, I., de Herrera, A.G.S., Tsikrika, T.: Overview of the CLEF 2011 medical image classification and retrieval tasks. In: CLEF 2011 Labs and Workshop, Notebook Papers, 19–22 (2011)
Khan, M., Ohno, Y.: A hybrid image compression technique using quadtree decomposition and parametric line fitting for synthetic images. Adv. Comput. Sci. Eng. 1(3), 263–283 (2007)
Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 2169–2178. IEEE (2006)
Li, F.F., Perona, P.: A Bayesian hierarchical model for learning natural scene categories. In: CVPR, vol. 2, pp. 524–531 (2005)
Lowe, D.G.: Object recognition from local scale-invariant features. In: Proceedings of the International Conference on Computer Vision, ICCV 1999, vol. 2, p. 1150. IEEE Computer Society (1999)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. CoRR abs/1301.3781 (2013)
Morvan, Y., Farin, D., De With, P.H.: Depth-image compression based on an RD optimized quadtree decomposition for the transmission of multiview images. In: IEEE International Conference on Image Processing ICIP 2007, vol. 5, pp. V-105. IEEE (2007)
Müller, H., de Herrera, A.G.S., Kalpathy-Cramer, J., Demner-Fushman, D., Antani, S.K., Eggel, I.: Overview of the imageCLEF 2012 medical image retrieval and classification tasks. In: Working Notes for CLEF 2012 Conference (2012)
Müller, H., Michoux, N., Bandon, D., Geissbühler, A.: A review of content-based image retrieval systems in medical applications - clinical benefits and future directions. I. J. Med. Inform. 73(1), 1–23 (2004)
Pass, G., Zabih, R., Miller, J.: Comparing images using color coherence vectors. In: Proceedings of the Fourth ACM International Conference on Multimedia, MULTIMEDIA 1996, NY, USA, pp. 65–73. ACM, New York (1996)
Ramanathan, V., Mishra, S., Mitra, P.: Quadtree decomposition based extended vector space model for image retrieval. In: IEEE Workshop on Applications of Computer Vision (WACV 2011), 5–7 January 2011, Kona, HI, USA, pp. 139–144 (2011)
Van de Sande, K.E., Gevers, T., Snoek, C.G.: A comparison of color features for visual concept classification. In: Proceedings of the 2008 International Conference on Content-Based Image and Video Retrieval, pp. 141–150. ACM (2008)
Shusterman, E., Feder, M.: Image compression via improved quadtree decomposition algorithms. IEEE Trans. Image Process. 3(2), 207–215 (1994)
Sivic, J., Russell, B.C., Efros, A.A., Zisserman, A., Freeman, W.T.: Discovering object categories in image collections. In: Proceedings of the International Conference on Computer Vision (2005)
Smith, J.M., Chang, S.F.: Quad-tree segmentation for texture-based image query. In: Blattner, M., Limb, J.O. (eds.) ACM Multimedia, pp. 279–286. ACM Press, New York (1994)
Vedaldi, A., Zisserman, A.: Efficient additive kernels via explicit feature maps. IEEE Trans. Pattern Anal. Mach. Intell. 34(3), 480–492 (2012)
Wengert, C., Douze, M., Jégou, H.: Bag-of-colors for improved image search. In: Proceedings of the 19th International Conference on Multimedia 2011, pp. 1437–1440 (2011)
Yang, J., Jiang, Y.G., Hauptmann, A.G., Ngo, C.W.: Evaluating bag-of-visual-words representations in scene classification. In: Proceedings of the Internationla Workshop on Multimedia Information Retrieval, pp. 197–206. ACM (2007)
Yin, X., Düntsch, I., Gediga, G.: Quadtree representation and compression of spatial data. In: Peters, J.F., Skowron, A., Chan, C.-C., Grzymala-Busse, J.W., Ziarko, W.P. (eds.) Transactions on Rough Sets XIII. LNCS, vol. 6499, pp. 207–239. Springer, Heidelberg (2011). doi:10.1007/978-3-642-18302-7_12
Zhou, X., Depeursinge, A., Müller, H.: Information fusion for combining visual and textual image retrieval in imageCLEF@ICPR. In: Ünay, D., Çataltepe, Z., Aksoy, S. (eds.) ICPR 2010. LNCS, vol. 6388, pp. 129–137. Springer, Heidelberg (2010). doi:10.1007/978-3-642-17711-8_14
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Valavanis, L., Stathopoulos, S., Kalamboukis, T. (2017). Fusion of Bag-of-Words Models for Image Classification in the Medical Domain. In: Jose, J., et al. Advances in Information Retrieval. ECIR 2017. Lecture Notes in Computer Science(), vol 10193. Springer, Cham. https://doi.org/10.1007/978-3-319-56608-5_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-56608-5_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-56607-8
Online ISBN: 978-3-319-56608-5
eBook Packages: Computer ScienceComputer Science (R0)