Multimodal Medical Image Retrieval OHSU at ImageCLEF 2008

  • Jayashree Kalpathy-Cramer
  • Steven Bedrick
  • William Hatt
  • William Hersh
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5706)


We present results from the Oregon Health & Science University’s participation in the medical retrieval task of ImageCLEF 2008. Our web-based retrieval system was built using a Ruby on Rails framework. Ferret, a Ruby port of Lucene was used to create the full-text based index and search engine. In addition to the textual index of annotations, supervised machine learning techniques using visual features were used to classify the images based on image acquisition modality. Our system provides the user with a number of search options including the ability to limit their search by modality, UMLS-based query expansion, and Natural Language Processing-based techniques. Purely textual runs as well as mixed runs using the purported modality were submitted. We also submitted interactive runs using user specified search options. Although the use of the UMLS metathesaurus increased our recall, our system is geared towards early precision. Consequently, many of our multimodal automatic runs using the custom parser as well as interactive runs had high early precision including the highest P10 and P30 among the official runs. Our runs also performed well using the bpref metric, a measure that is more robust in the case of incomplete judgments.


Image Retrieval Image Retrieval System Medical Image Retrieval Supervise Machine Learning Technique Image Acquisition Modality 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hersh, W.R., Müller, H., Jensen, J.R., Yang, J., Gorman, P.N., Ruch, P.: Advancing biomedical image retrieval: Development and analysis of a test collection. J. Am. Med. Inform. Assoc. (June 2006); M2082Google Scholar
  2. 2.
    Smeulders, A., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(12), 1349–1380 (2000)CrossRefGoogle Scholar
  3. 3.
    Tagare, H.D., Jaffe, C.C., Duncan, J.: Medical image databases: A content-based retrieval approach. J. Am. Med. Inform. Assoc. 4(3), 184–198 (1997)CrossRefGoogle Scholar
  4. 4.
    Aisen, A.M., Broderick, L.S., Winer-Muram, H., Brodley, C.E., Kak, A.C., Pavlopoulou, C., Dy, J., Shyu, C.R., Marchiori, A.: Automated storage and retrieval of thin-section ct images to assist diagnosis: System description and preliminary assessment. Radiology 228(1), 265–270 (2003)CrossRefGoogle Scholar
  5. 5.
    Schmid-Saugeona, P., Guillodb, J., Thirana, J.P.: Towards a computer-aided diagnosis system for pigmented skin lesions. Computerized Medical Imaging and Graphics: The Official Journal of the Computerized Medical Imaging Society 27(1), 65–78 (2003); PMID: 12573891CrossRefGoogle Scholar
  6. 6.
    Müller, H., Michoux, N., Bandon, D., Geissbuhler, A.: A review of content-based image retrieval systems in medical applications–clinical benefits and future directions. International Journal of Medical Informatics 73(1), 1–23 (2004)CrossRefGoogle Scholar
  7. 7.
    Hersh, W.R., Kalpathy-Cramer, J., Jensen, J.: Medical image retrieval and automated annotation: OHSU at imageCLEF 2006. In: Peters, C., Clough, P., Gey, F.C., Karlgren, J., Magnini, B., Oard, D.W., de Rijke, M., Stempfhuber, M. (eds.) CLEF 2006. LNCS, vol. 4730, pp. 660–669. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  8. 8.
    Kalpathy-Cramer, J., Hersh, W.: Automatic image modality based classification and annotation to improve medical image retrieval. Studies in Health Technology and Informatics 129(Pt 2), 1334–1338 (2007); PMID: 17911931Google Scholar
  9. 9.
    Braschler, M., Peters, C.: Cross-language evaluation forum: Objectives, results, achievements. Information Retrieval 7(1), 7–31 (2004)CrossRefGoogle Scholar
  10. 10.
    Müller, H., Clough, P., Hersh, W., Deselaers, T., Lehmann, T., Geissbuhler, A.: Evaluation axes for medical image retrieval systems: the imageclef experience. In: Proceedings of the 13th annual ACM international conference on Multimedia, Hilton, Singapore, pp. 1014–1022. ACM, New York (2005)CrossRefGoogle Scholar
  11. 11.
    Müller, H., Kalpathy-Cramer, J., Kahn Jr., C.E., Hatt, W., Bedrick, S., Hersh, W.: Overview of the ImageCLEFmed 2008 medical image retrieval task. In: Peters, C., et al. (eds.) CLEF 2008. LNCS, vol. 5706, pp. 512–522. Springer, Heidelberg (2009)Google Scholar
  12. 12.
    Furnas, G.W., Deerwester, S., Dumais, S.T., Landauer, T.K., Harshman, R.A., Streeter, L.A., Lochbaum, K.E.: Information retrieval using a singular value decomposition model of latent semantic structure. In: Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval, Grenoble, France, pp. 465–480. ACM, New York (1988)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Jayashree Kalpathy-Cramer
    • 1
  • Steven Bedrick
    • 1
  • William Hatt
    • 1
  • William Hersh
    • 1
  1. 1.Department of Medical Informatics & Clinical EpidemiologyOregon Health and Science University (OHSU)PortlandUSA

Personalised recommendations