Abstract
This paper presents a cross-modal approach of image retrieval from a medical image collection which integrates visual information based on purely low-level image contents and case related textual information from the annotated XML files. The advantages of both the modalities are exploited by involving the users in the retrieval loop. For content-based search, low-level visual features are extracted in vector form at different image representations. For text-based search, keywords from the annotated files are extracted and indexed by employing the vector space model of information retrieval. Based on the relevance feedback, textual and visual query refinements are performed and user’s perceived semantics are propagated from one modality to another. Finally, the most similar images are obtained by a linear combination of similarity matching and re-ordering in a pre-filtered image set. The experiments are performed on a collection of diverse medical images with case-based annotation of each image by experts. It demonstrates the flexibility and the effectiveness of the proposed approach compared to using only a single modality or without any feedback information.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Müller, H., Michoux, N., Bandon, D., Geissbuhler, A.: A review of content-based image retrieval applications–clinical benefits and future directions. International Journal of Medical Informatics 73, 1–23 (2004)
Tagare, H.D., Jafe, C., Duncan, J.: Medical image databases: A content-based retrieval approach. Journal of the American Medical Informatics Association 4 (3), 184–198 (1997)
Smeulder, A., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-Based Image Retrieval at the End of the Early Years. IEEE Trans. on Pattern Anal. and Machine Intell. 22, 1349–1380 (2000)
Lu, Y., Zhang, H., Wenyin, L., Hu, C.: Joint semantics and feature based image retrieval using relevance feedback. IEEE transactions on multimedia 5(3), 339–347 (2003)
Sclaroff, S., Cascia, M.L., Sethi, S., Taycher, L.: Unifying Textual and Visual Cues for Content-Based Image Retrieval on the World Wide Web. Computer Vision and Image Understanding 75, 86–98 (1999)
Rong, Z., Grosky, W.I.: Narrowing the semantic gap – improved text-based web document retrieval using visual features. IEEE Transactions on Multimedia 4(2), 189–200 (2002)
Rocchio, J.J.: Relevance Feedback in Information Retrieval. In: The Smart Retrieval System, pp. 313–323. Prentice-Hall, Englewood Cliffs (1979)
Manjunath, B.S., Salembier, P., Sikora, T. (eds.): Introduction to MPEG-7 –Multimedia Content Description Interface, pp. 187–212. John Wiley Sons Ltd., Chichester (2002)
Haralick, R.M., Shanmugam, D.I.: Textural features for image classification. IEEE Trans System, Man, Cybernetics SMC-3, 610–621 (1973)
Baeza-Yates, R., Ribiero-Neto, B.: Modern Information Retrieval. Addison-Wesley, Reading (1999)
Rui, Y., Huang, T.S.: Relevance Feedback: A Power Tool for Interactive Content-Based Image Retrieval. IEEE Circuits Syst. Video Technol. 8 (1999)
Salton, G., Buckley, C.: Improving retrieval performance by relevance feedback. JASIS 41(4), 288–297 (1990)
Friedman, J.: Regularized Discriminant Analysis. Journal of American Statistical Association 84, 165–175 (2002)
Fukunaga, K.: Introduction to Statistical Pattern Recognition, 2nd edn. Academic Press Professional, Inc., San Diego (1990)
Müller, H., Deselaers, T., Lehmann, T., Clough, P., Hersh, W.: Overview of the ImageCLEFmed 2006 medical retrieval and annotation tasks. In: CLEF working notes, Alicante, Spain (September 2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Rahman, M.M., Sood, V., Desai, B.C., Bhattacharya, P. (2006). Cross-Modal Interaction and Integration with Relevance Feedback for Medical Image Retrieval. In: Cham, TJ., Cai, J., Dorai, C., Rajan, D., Chua, TS., Chia, LT. (eds) Advances in Multimedia Modeling. MMM 2007. Lecture Notes in Computer Science, vol 4351. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69423-6_43
Download citation
DOI: https://doi.org/10.1007/978-3-540-69423-6_43
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69421-2
Online ISBN: 978-3-540-69423-6
eBook Packages: Computer ScienceComputer Science (R0)