Skip to main content
SpringerLink
Log in

The Role of Content-Based Image Retrieval in Mammography CAD

  • Chapter
  • First Online:
Computational Intelligence in Biomedical Imaging

Abstract

There has been a tremendous increase in the amount of stored medical images, making manual search infeasible for a busy radiology clinic. Content-based image retrieval (CBIR) offers a computerized solution that aims to query images for diagnostic information based on the content or extracted features of the images rather than their textual annotation. Potentially, this approach would provide the radiologist with archived examples that are relevant to the case being evaluated. In this chapter, we review recent advances in CBIR technology and discuss its expanding role in medical imaging and its particular application to mammography. We provide two examples based on our experience using CBIR in mammography; one example is to model perceptual similarity in CBIR and the other example is to apply CBIR to achieve case-adaptive classification in computer-aided diagnosis (CAD). We also highlight the potential opportunities in this field for CAD research and clinical decision-making.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. American Cancer Society (2012) Cancer facts and figures. American Cancer Society, Atlanta

    Google Scholar 

  2. Mushlin AI, Kouides RW, Shapiro DE (1998) Estimating the accuracy of screening mammography: a meta-analysis. Am J Prev Med 14:143–153

    Article  Google Scholar 

  3. Urbain JL (2005) Breast cancer screening, diagnostic accuracy and health care policies. CMAJ 172:210–211

    Article  Google Scholar 

  4. Kolb TM, Lichy J, Newhouse JH (2002) Comparison of the performance of screening mammography, physical examination, and breast US and evaluation of factors that influence them: an analysis of 27,825 patient evaluations. Radiology 225:165–175

    Article  Google Scholar 

  5. Elmore JG, Barton MB, Moceri VM, Polk S, Arena PJ, Fletcher SW (1998) Ten-year risk of false positive screening mammograms and clinical breast examinations. N Engl J Med 338:1089–1096

    Article  Google Scholar 

  6. Tan A, Freeman DH Jr, Goodwin JS, Freeman JL (2006) Variation in false-positive rates of mammography reading among 1067 radiologists: a population-based assessment. Breast Cancer Res Treat 100:309–318

    Article  Google Scholar 

  7. Bazzocchi M, Mazzarella F, Del Frate C, Girometti F, Zuiani C (2007) CAD systems for mammography: a real opportunity? A review of the literature. Radiol Med 112:329–353

    Article  Google Scholar 

  8. Collins MJ, Hoffmeister J, Worrell SW (2006) Computer-aided detection and diagnosis of breast cancer. Semin Ultrasound CT MR 27:351–355

    Article  Google Scholar 

  9. Hadjiiski L, Sahiner B, Helvie MA, Chan HP, Roubidoux MA, Paramagul C, Blane C, Petrick N, Bailey J, Klein K, Foster M, Patterson SK, Adler D, Nees AV, Shen J (2006) Breast masses: computer-aided diagnosis with serial mammograms. Radiology 240:343–356

    Article  Google Scholar 

  10. Malich A, Fischer DR, Bottcher J (2006) CAD for mammography: the technique, results, current role and further developments. Eur Radiol 16:1449–1460

    Article  Google Scholar 

  11. Dean JC, Ilvento CC (2006) Improved cancer detection using computer-aided detection with diagnostic and screening mammography: prospective study of 104 cancers. AJR Am J Roentgenol 187:20–28

    Article  Google Scholar 

  12. Georgian-Smith D, Moore RH, Halpern E, Yeh ED, Rafferty EA, D’Alessandro HA, Staffa M, Hall DA, McCarthy KA, Kopans DB (2007) Blinded comparison of computer-aided detection with human second reading in screening mammography. AJR Am J Roentgenol 189:1135–1141

    Article  Google Scholar 

  13. Yang SK, Moon WK, Cho N, Park JS, Cha JH, Kim SM, Kim SJ, Im JG (2007) Screening mammography-detected cancers: sensitivity of a computer-aided detection system applied to full-field digital mammograms. Radiology 244:104–111

    Article  Google Scholar 

  14. Fenton JJ, Taplin SH, Carney PA, Abraham L, Sickles EA, D’Orsi C, Berns EA, Cutter G, Hendrick RE, Barlow WE, Elmore JG (2007) Influence of computer-aided detection on performance of screening mammography. N Engl J Med 356:1399–1409

    Article  Google Scholar 

  15. Smeulders AWM, Worring M (2000) Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22:32 p

    Article  Google Scholar 

  16. Guo GD, Jain AK, Ma WY, Zhang HJ (2002) Learning similarity measure for natural image retrieval with relevance feedback. IEEE Trans Neural Netw 13:811–820

    Article  Google Scholar 

  17. Chen Y, Wang JZ, Krovetz R (2005) CLUE: cluster-based retrieval of images by unsupervised learning. IEEE Trans Image Process 14:1187–1201

    Article  Google Scholar 

  18. Müller H, Michoux N, Bandon D, Geissbuhler A (2004) A review of content-based image retrieval systems in medical applications – clinical benefits and future directions. Int J Med Inform 73:1–23

    Article  Google Scholar 

  19. Bustos B, Keim D, Saupe D, Schreck T (2007) Content-based 3D object retrieval. IEEE Comput Graph Appl 27:22–27

    Article  Google Scholar 

  20. Bhanu B, Peng J, Qing S (1998) Learning feature relevance and similarity metrics in image database. In: IEEE workshop proceedings on content-based access of image and video libraries. IEEE Computer Society, Washington, DC, USA, pp 14–18

    Google Scholar 

  21. El Naqa I, Yang Y, Galatsanos NP, Nishikawa RM, Wernick MN (2004) A similarity learning approach to content-based image retrieval: application to digital mammography. IEEE Trans Med Imaging 23:1233–1244

    Article  Google Scholar 

  22. Swett HA, Mutalik PG, Neklesa VP, Horvath L, Lee C, Richter J, Tocino I, Fisher PR (1998) Voice-activated retrieval of mammography reference images. J Digit Imaging 11:65–73

    Article  Google Scholar 

  23. Shyu CR, Brodley CE, Kak AC, Kosaka A, Aisen A, Broderick LS (1999) ASSERT: a physician-in-the-loop content-based retrieval system for HRCT image database. Comput Vis Image Underst 75(1–2):111–132

    Article  Google Scholar 

  24. Dy JG, Brodley CE, Kak A, Broderick LS, Aisen AM (2003) Unsupervised feature selection applied to content-based retrieval of lung images. IEEE Trans Pattern Anal Mach Intell 25:373–378

    Article  Google Scholar 

  25. Antani S, Long LR, Thoma GR (2004) Content-based image retrieval for large biomedical image archives. Stud Health Technol Inform 107:829–833

    Google Scholar 

  26. Guimond A, Subsol G (1997) Automatic MRI database exploration and applications. Pattern Recognit Artif Intell 11:1345–1365

    Article  Google Scholar 

  27. Liu Y, Lazar N, Rothfus W (2002) Semantic-based biomedical image indexing and retrieval. In: Proceedings of the international conference on diagnostic imaging and analysis, pp 18–20

    Google Scholar 

  28. Cai W, Feng DD, Fulton R (2000) Content-based retrieval of dynamic PET functional images. IEEE Trans Inf Technol Biomed 4:152–158

    Article  Google Scholar 

  29. Tobin KW, Abramoff MD, Chaum E, Giancardo L, Govindasamy VP, Karnowski TP, Tennant MTS, Swainson S (2008) Using a patient image archive to diagnose retinopathy. In: 30th annual international conference of the IEEE. Engineering in Medicine and Biology Society (EMBS), Vancouver, Canada, pp 5441–5444

    Google Scholar 

  30. Dorileo EAG, Frade MAC, Roselino AMF, Rangayyan RM, Azevedo-Marques PM (2008) Color image processing and content-based image retrieval techniques for the analysis of dermatological lesions. In: 30th annual international conference of the IEEE. Engineering in Medicine and Biology Society (EMBS), Vancouver, Canada, pp 1230–1233

    Google Scholar 

  31. Xue Z, Antani S, Long LR, Jeronimo J, Thoma GR (2007) Investigating CBIR techniques for cervicographic images. In: Proceedings of the 2007 annual symposium of the American Medical Information Association (AMIA), Chicago, pp 826–830

    Google Scholar 

  32. Zheng L, Wetzel AW, Gilbertson J, Becich MJ (2003) Design and analysis of a content-based pathology image retrieval system. IEEE Trans Inf Technol Biomed 7:249–255

    Article  Google Scholar 

  33. Muller H, Rosset A, Garcia A, Vallee JP, Geissbuhler A (2005) Informatics in radiology (infoRAD): benefits of content-based visual data access in radiology. Radiographics 25:849–858

    Article  Google Scholar 

  34. Vannier MW, Summers RM (2003) Sharing images. Radiology 228:23–25

    Article  Google Scholar 

  35. Lehmann TM, Gould MO, Thies C, Fischer B, Keysers M, Kohnen D, Schubert H, Wein BB (2003) Content-based image retrieval in medical applications for picture archiving and communication systems. In: Proceedings SPIE medical imaging, vol 5033, San Diego, pp109–117

    Google Scholar 

  36. Muller H, Rosset A, Vallee J, Geissbuhler A (2004) Comparing features sets for content-based image retrieval in a medical-case database. In: PACS and Imaging Informatics IS&T/SPIE Medical Imaging, San Diego, CA, pp 99–109

    Google Scholar 

  37. Lehmann T, Guld M, Thies C, Plodowski B, Keysers D, Ott B (2004) IRMA – content-based image retrieval in medical applications. In: 14th world congress on medical informatics. IOS, Amsterdam, pp 842–848

    Google Scholar 

  38. Antani S, Lee DJ, Long LR, Thoma GR (2004) Evaluation of shape similarity measurement methods for spine X-ray images. J Vis Commun Image Represent 15:285–302

    Article  Google Scholar 

  39. American College of Radiology (2003) American College of Radiology (ACR) Breast Imaging Reporting and Data System Atlas (BI-RADS® Atlas). American College of Radiology, Reston

    Google Scholar 

  40. Sklansky J, Tao EY, Bazargan M, Ornes CJ, Murchison RC, Teklehaimanot S (2000) Computer-aided, case-based diagnosis of mammographic regions of interest containing microcalcifications. Acad Radiol 7:395–405

    Article  Google Scholar 

  41. Qi H, Snyder WE (1999) Content-based image retrieval in picture archiving and communications systems. J Digit Imaging 12:81–83

    Article  Google Scholar 

  42. El Naqa I (2002) Content-based image retrieval by similarity learning for digital mammography in electrical engineering. Illinois Institute of Technology, Chicago

    Google Scholar 

  43. El Naqa I, Wernick M, Yang Y, Galatsanos N (2000) Image retrieval based on similarity learning. In: IEEE international conference on image processing, Vancouver, pp 722–725

    Google Scholar 

  44. El Naqa I, Wernick M, Yang Y, Galatsanos N (2002) Content-based image retrieval for digital mammography. In: IEEE international conference on image processing, Rochester, pp 141–144

    Google Scholar 

  45. Giger M, Huo Z, Vyborny C, Lan L, Bonta I, Horsch K (2002) Intelligent CAD workstation for breast imaging using similarity to known lesions and multiple visual prompt aids. In: Proceedings of the SPIE, vol 4684, San Diego, pp 768–773

    Google Scholar 

  46. Tourassi GD, Harrawood B, Singh S, Lo JY, Floyd CE (2007) Evaluation of information-theoretic similarity measures for content-based retrieval and detection of masses in mammograms. Med Phys 34:140–150

    Article  Google Scholar 

  47. Tourassi GD, Ike R 3rd, Singh S, Harrawood B (2008) Evaluating the effect of image preprocessing on an information-theoretic CAD system in mammography. Acad Radiol 15:626–634

    Article  Google Scholar 

  48. Tourassi GD, Vargas-Voracek R, Catarious DM Jr, Floyd CE Jr (2003) Computer-assisted detection of mammographic masses: a template matching scheme based on mutual information. Med Phys 30:2123–2130

    Article  Google Scholar 

  49. Zheng B, Lu A, Hardesty LA, Sumkin JH, Hakim CM, Ganott MA, Gur D (2006) A method to improve visual similarity of breast masses for an interactive computer-aided diagnosis environment. Med Phys 33:111–117

    Article  Google Scholar 

  50. Zheng B, Mello-Thoms C, Wang X, Gur D (2007) Improvement of visual similarity of similar breast masses selected by computer-aided diagnosis schemes. In: 4th IEEE international symposium on biomedical imaging: from nano to macro, Washington, DC, pp 516–519

    Google Scholar 

  51. Kinoshita SK, de Azevedo-Marques PM, Pereira RR Jr, Rodrigues JA, Rangayyan RM (2007) Content-based retrieval of mammograms using visual features related to breast density patterns. J Digit Imaging 20:172–190

    Article  Google Scholar 

  52. Holt A, Bichindaritz I, Schmidt R, Perner P (2005) Medical applications in case-based reasoning. Knowl Eng Rev 20:289–292

    Article  Google Scholar 

  53. Wei L, Yang Y, Nishikawa R, Wernick M (2006b) Learning of perceptual similarity from expert readers for mammogram retrieval. In: IEEE international symposium medical imaging: macro to nano, Arlington, pp 1356–1359

    Google Scholar 

  54. Bilska-Wolak AO, Floyd CE Jr (2002) Development and evaluation of a case-based reasoning classifier for prediction of breast biopsy outcome with BI-RADS lexicon. Med Phys 29:2090–2100

    Article  Google Scholar 

  55. Floyd CE, Lo JY, Tourassi GD (2000) Case-based reasoning computer algorithm that uses mammographic findings for breast biopsy decisions. Am J Roentgenol 175:1347–1352

    Article  Google Scholar 

  56. Mazurowski MA, Habas PA, Zurada JM, Tourassi GD (2008) Decision optimization of case-based computer-aided decision systems using genetic algorithms with application to mammography. Phys Med Biol 53:895–908

    Article  Google Scholar 

  57. Nakayama R, Abe H, Shiraishi J, Doi K (2009) Potential usefulness of similar images in the differential diagnosis of clustered microcalcifications on mammograms. Radiology 253:625–631

    Article  Google Scholar 

  58. Del Bimbo A (1999) Visual information retrieval. Morgan Kaufmann Publishers, San Francisco

    Google Scholar 

  59. Wei L, Yang Y, Wernick MN, Nishikawa RM (2009) Learning of perceptual similarity from expert readers for mammogram retrieval. IEEE J Sel Top Signal Process 3:53–61

    Article  MATH  Google Scholar 

  60. Efron B, Tibshirani R (1993) An introduction to the bootstrap. Chapman & Hall, New York

    Book  MATH  Google Scholar 

  61. Vapnik V (1998) Statistical learning theory. Wiley, New York

    MATH  Google Scholar 

  62. Wei L, Yang Y, Nishikawa RM, Jiang Y (2005) A study on several machine-learning methods for classification of malignant and benign clustered microcalcifications. IEEE Trans Med Imaging 24:371–380

    Article  Google Scholar 

  63. El Naqa I, Yang Y, Galatsanos NP, Wernick MN (2004b) Mammogram retrieval based on incremental learning. In: IEEE international symposium on biomedical imaging: nano to macro, vol 2, Arlington, pp 1163–1166

    Google Scholar 

  64. Oh JH, Yang Y, El Naqa I (2010) Adaptive learning for relevance feedback: application to digital mammography. Med Phys 37:4432–4444

    Article  Google Scholar 

  65. EI Naqa I, Oh J, Yang Y (2012) Machine learning in computer-aided diagnosis. In: Suzuki K (ed) Medical imaging intelligence and analysis. IGI Global, Hershey

    Google Scholar 

  66. Scholkopf B, Smola A, Muller KR (1998) Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput 10:1299–1319

    Article  Google Scholar 

  67. Bishop CM (2006) Pattern recognition and machine learning. Springer, New York

    MATH  Google Scholar 

  68. Heath M, Bowyer K, Kopans D, Moore R, Kegelmeyer P (2001) The digital database for screening mammography. In: 5th international workshop on digital mammography (IWDM). Medical Physics Publishing Corp, Madison, pp 212–218

    Google Scholar 

  69. Guyon I, Elissee A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182

    MATH  Google Scholar 

  70. Samuelson FW, Petrick N (2006) Comparing image detection algorithms using resampling. In: International symposium on biomedical imaging, Arlington, pp 1312–1315

    Google Scholar 

  71. Bandos AI, Rockette HE, Gur D (2005) A permutation test sensitive to differences in areas for comparing ROC curves from a paired design. Stat Med 24:2873–2893

    Article  MathSciNet  Google Scholar 

  72. Metz CE, Herman BA, Shen JH (1998) Maximum likelihood estimation of receiver operating characteristic (ROC) curves from continuously-distributed data. Stat Med 17:1033–1053

    Article  Google Scholar 

  73. El Naqa I, Wei L, Yang Y (2010) Ubiquitous health and medical informatics. In: Mohammed S, Fiaidhi J (eds) The ubiquity 2.0 trend and beyond. IGI Global, Hershey

    Google Scholar 

  74. Li Q, Li F, Shiraishi J, Katsuragawa S, Sone S, Doi K (2003) Investigation of new psychophysical measures for evaluation of similar images on thoracic CT for distinction between benign and malignant nodules. Med Phys 30:2584–2593

    Article  Google Scholar 

  75. Muramatsu C, Li Q, Schmidt R, Suzuki K, Shiraishi J, Newstead G, Doi K (2006) Experimental determination of subjective similarity for pairs of clustered microcalcifications on mammograms: observer study results. Med Phys 33:3460–3468

    Article  Google Scholar 

  76. Muramatsu C, Li Q, Suzuki K, Schmidt RA, Shiraishi J, Newstead GM, Doi K (2005) Investigation of psychophysical measure for evaluation of similar images for mammographic masses: preliminary results. Med Phys 32:2295–2304

    Article  Google Scholar 

  77. Aggarwal P, Sardana HK, Jindal G (2009) Content based medical image retrieval: theory gaps and future directions. ICGST-GVIP J 9:27–37

    Google Scholar 

  78. Wei L, Yang Y, Nishikawa RM, Wernick MN, Edwards A (2005) Relevance vector machine for automatic detection of clustered microcalcifications. IEEE Trans Med Imaging 24:1278–1285

    Article  Google Scholar 

  79. Wei C, Li C, Wilson R (2006) Database modeling for industrial data management: emerging technologies and applications. Idea Group Publishing, Hershey, pp 258–291

    Google Scholar 

Download references

Acknowledgments

This work was supported in part by NIH/NIBIB Grant R01EB009905 and the Natural Sciences and Engineering Research Council of Canada under grant NSERC-RGPIN 397711-11.

Author information

Authors and Affiliations

  1. Montreal General Hospital, Room #L5-112, 1650 Cedar Avenue, Montreal, QC, Canada, H3G 1A4

    Issam El Naqa

  2. Department of Electrical and Computer Engineering, Illinois Institute of Technology, 3301 S. Dearborn Street, Chicago, IL, 60616, USA

    Yongyi Yang

Authors
  1. Issam El Naqa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yongyi Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yongyi Yang .

Editor information

Editors and Affiliations

  1. Department of Radiology, University of Chicago, Chicago, Illinois, USA

    Kenji Suzuki

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer Science+Business Media New York

About this chapter

Cite this chapter

Naqa, I.E., Yang, Y. (2014). The Role of Content-Based Image Retrieval in Mammography CAD. In: Suzuki, K. (eds) Computational Intelligence in Biomedical Imaging. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-7245-2_2

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-7245-2_2

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4614-7244-5

  • Online ISBN: 978-1-4614-7245-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation