Skip to main content
SpringerLink
Log in

Interobserver and intermodality agreement of standardized algorithms for non-invasive diagnosis of hepatocellular carcinoma in high-risk patients: CEUS-LI-RADS versus MRI-LI-RADS

  • Hepatobiliary-Pancreas
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

We compared the interobserver agreement for the recently introduced contrast-enhanced ultrasound (CEUS)-based algorithm CEUS-LI-RADS (Liver Imaging Reporting and Data System) versus the well-established magnetic resonance imaging (MRI)-LI-RADS for non-invasive diagnosis of hepatocellular carcinoma (HCC) in high-risk patients.

Methods

Focal liver lesions in 50 high-risk patients (mean age 66.2 ± 11.8 years; 39 male) were assessed retrospectively with CEUS and MRI. Two independent observers reviewed CEUS and MRI examinations, separately, classifying observations according to CEUS-LI-RADSv.2016 and MRI-LI-RADSv.2014. Interobserver agreement was assessed with Cohen’s kappa.

Results

Forty-three lesions were HCCs; two were intrahepatic cholangiocarcinomas; five were benign lesions. Arterial phase hyperenhancement was perceived less frequently with CEUS than with MRI (37/50 / 38/50 lesions = 74%/78% [CEUS; observer 1/observer 2] versus 46/50 / 44/50 lesions = 92%/88% [MRI; observer 1/observer 2]). Washout appearance was observed in 34/50 / 20/50 lesions = 68%/40% with CEUS and 31/50 / 31/50 lesions = 62%/62%) with MRI. Interobserver agreement was moderate for arterial hyperenhancement (ĸ = 0.511/0.565 [CEUS/MRI]) and “washout” (ĸ = 0.490/0.582 [CEUS/MRI]), fair for CEUS-LI-RADS category (ĸ = 0.309) and substantial for MRI-LI-RADS category (ĸ = 0.609). Intermodality agreement was fair for arterial hyperenhancement (ĸ = 0.329), slight to fair for “washout” (ĸ = 0.202) and LI-RADS category (ĸ = 0.218)

Conclusion

Interobserver agreement is substantial for MRI-LI-RADS and only fair for CEUS-LI-RADS. This is mostly because interobserver agreement in the perception of washout appearance is better in MRI than in CEUS. Further refinement of the LI-RADS algorithms and increasing education and practice may be necessary to improve the concordance between CEUS and MRI for the final LI-RADS categorization.

Key Points

• CEUS-LI-RADS and MRI-LIRADS enable standardized non-invasive diagnosis of HCC in high-risk patients.

• With CEUS, interobserver agreement is better for arterial hyperenhancement than for “washout”.

• Interobserver agreement for major features is moderate for both CEUS and MRI.

• Interobserver agreement for LI-RADS category is substantial for MRI, and fair for CEUS.

• Interobserver-agreement for CEUS-LI-RADS will presumably improve with ongoing use of the algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Abbreviations

ACR:

American College of Radiology

bh:

Breath-hold

BMI:

Body mass index

CE-CT:

Contrast-enhanced computed tomography

CE-MRI:

Contrast-enhanced magnetic resonance imaging

CEUS:

Contrast-enhanced Ultrasound

cor:

Coronal

CT:

Computed tomography

DWI:

Diffusion-weighted image

fs:

Fat saturation

GRE:

Gradient echo

HASTE:

Half-Fourier acquisition single-shot turbo spin echo

HCC:

Hepatocellular carcinoma

ICC:

Intrahepatic cholangiocellular carcinoma

LI-RADS:

Liver Imaging Reporting and Data System

LR:

LI-RADS category

MRI:

Magnetic resonance imaging

T:

Tesla

T1w:

T1-weighted

T2w:

T2-weighted

TE:

Echo time

TR:

Repetition time

tra:

Transversal

TSE:

Turbo spin echo

VIBE:

Volumetric-interpolated breath-hold examination

References

  1. Westwood M, Joore M, Grutters J et al (2013) Contrast-enhanced ultrasound using SonoVue(R) (sulphur hexafluoride microbubbles) compared with contrast-enhanced computed tomography and contrast-enhanced magnetic resonance imaging for the characterisation of focal liver lesions and detection of liver metastases: a systematic review and cost-effectiveness analysis. Health Technol Assess 17:1–243

    Article  Google Scholar 

  2. Claudon M, Dietrich CF, Choi BI et al (2013) Guidelines and good clinical practice recommendations for contrast enhanced ultrasound (CEUS) in the liver - update 2012: a WFUMB-EFSUMB initiative in cooperation with representatives of AFSUMB, AIUM, ASUM, FLAUS and ICUS. Ultrasound Med Biol 39:187–210

    Article  Google Scholar 

  3. Bota S, Piscaglia F, Marinelli S, Pecorelli A, Terzi E, Bolondi L (2012) Comparison of international guidelines for noninvasive diagnosis of hepatocellular carcinoma. Liver Cancer 1:190–200

    Article  Google Scholar 

  4. Bruix J, Sherman M (2011) Management of hepatocellular carcinoma: an update. Hepatology 53:1020–1022

    Article  Google Scholar 

  5. Kokudo N, Hasegawa K, Akahane M et al (2015) Evidence-based clinical practice guidelines for hepatocellular carcinoma: The Japan Society of Hepatology 2013 update (3rd JSH-HCC guidelines). Hepatol Res 45. https://doi.org/10.1111/hepr.12464

    Article  Google Scholar 

  6. Friedrich-Rust M, Klopffleisch T, Nierhoff J et al (2013) Contrast-enhanced ultrasound for the differentiation of benign and malignant focal liver lesions: a meta-analysis. Liver Int 33:739–755

    Article  CAS  Google Scholar 

  7. Shah S, Shukla A, Paunipagar B (2014) Radiological features of hepatocellular carcinoma. J Clin Exp Hepatol 4:S63–S66

    Article  Google Scholar 

  8. Leoni S, Piscaglia F, Granito A et al (2013) Characterization of primary and recurrent nodules in liver cirrhosis using contrast-enhanced ultrasound: which vascular criteria should be adopted? Ultraschall Med 34:280–287

    Article  CAS  Google Scholar 

  9. Mitchell DG, Bruix J, Sherman M, Sirlin CB (2015) LI-RADS (Liver Imaging Reporting and Data System): summary, discussion, and consensus of the LI-RADS Management Working Group and future directions. Hepatology 61:1056–1065

    Article  Google Scholar 

  10. Elsayes KM, Kielar AZ, Agrons MM et al (2017) Liver Imaging Reporting and Data System: an expert consensus statement. J Hepatocell Carcinoma 4:29–39

    Article  Google Scholar 

  11. AWMF-Leitlinie S3 - Leitlinie Deutschland: “Hepatozelluläres Karzinom”, Registernummer 032/053OL Stand: 01.05.2013 , gültig bis 30.04.2018; http://www.awmf.org/uploads/tx_szleitlinien/032-053OLl_S3_Hepatozellul%C3%A4res_Karzinom_Diagnostik_Therapie_2013-verlaengert.pdf accessed: May 19 2017. Available via http://www.awmf.org/uploads/tx_szleitlinien/032-053OLl_S3_Hepatozellul%C3%A4res_Karzinom_Diagnostik_Therapie_2013-verlaengert.pdf. Accessed May 19 2017

  12. Claudon M, Dietrich CF, Choi BI et al (2013) Guidelines and good clinical practice recommendations for contrast enhanced ultrasound (CEUS) in the liver–update 2012: a WFUMB-EFSUMB initiative in cooperation with representatives of AFSUMB, AIUM, ASUM, FLAUS and ICUS. Ultraschall Med 34:11–29

    Article  CAS  Google Scholar 

  13. Strobel D, Bernatik T, Blank W et al (2011) Diagnostic accuracy of CEUS in the differential diagnosis of small (</ = 20 mm) and subcentimetric (</ = 10 mm) focal liver lesions in comparison with histology. Results of the DEGUM multicenter trial. Ultraschall Med 32:593–597

    Article  CAS  Google Scholar 

  14. Strobel D, Seitz K, Blank W et al (2009) Tumor-specific vascularization pattern of liver metastasis, hepatocellular carcinoma, hemangioma and focal nodular hyperplasia in the differential diagnosis of 1,349 liver lesions in contrast-enhanced ultrasound (CEUS). Ultraschall Med 30:376–382

    Article  CAS  Google Scholar 

  15. Seitz K, Greis C, Schuler A et al (2011) Frequency of tumor entities among liver tumors of unclear etiology initially detected by sonography in the noncirrhotic or cirrhotic livers of 1349 patients. Results of the DEGUM multicenter study. Ultraschall Med 32:598–603

    Article  CAS  Google Scholar 

  16. Darnell A, Forner A, Rimola J et al (2015) Liver Imaging Reporting and Data System with MR imaging: evaluation in nodules 20 mm or smaller detected in cirrhosis at screening US. Radiology 275:698–707

    Article  Google Scholar 

  17. Seitz K, Piscaglia F (2013) Ultrasound: the only "one stop shop" for modern management of liver disease. Ultraschall Med 34:500–503

    Article  CAS  Google Scholar 

  18. Wildner D, Bernatik T, Greis C, Seitz K, Neurath MF, Strobel D (2015) CEUS in hepatocellular carcinoma and intrahepatic cholangiocellular carcinoma in 320 patients - early or late washout matters: a subanalysis of the DEGUM multicenter trial. Ultraschall Med 36:132–139

    Article  CAS  Google Scholar 

  19. Wildner D, Pfeifer L, Goertz RS et al (2014) Dynamic contrast-enhanced ultrasound (DCE-US) for the characterization of hepatocellular carcinoma and cholangiocellular carcinoma. Ultraschall Med 35:522–527

    Article  CAS  Google Scholar 

  20. McEvoy SH, McCarthy CJ, Lavelle LP et al (2013) Hepatocellular carcinoma: illustrated guide to systematic radiologic diagnosis and staging according to guidelines of the American Association for the Study of Liver Diseases. Radiographics 33:1653–1668

    Article  Google Scholar 

  21. Seitz K, Bernatik T, Strobel D et al (2010) Contrast-enhanced ultrasound (CEUS) for the characterization of focal liver lesions in clinical practice (DEGUM Multicenter Trial): CEUS vs. MRI–a prospective comparison in 269 patients. Ultraschall Med 31:492–499

    Article  CAS  Google Scholar 

  22. Schellhaas B, Wildner D, Pfeifer L et al (2016) LI-RADS-CEUS - proposal for a contrast-enhanced ultrasound algorithm for the diagnosis of hepatocellular carcinoma in high-risk populations. Ultraschall Med 37:627–634

    Article  CAS  Google Scholar 

  23. Karlas T (2017) LI-RADS-CEUS for the classification of HCC risk in liver lesions. Z Gastroenterol 55:507–508

    Article  CAS  Google Scholar 

  24. Piscaglia F, Wilson SR, Lyshchik A et al (2017) American College of Radiology Contrast Enhanced Ultrasound Liver Imaging Reporting and Data System (CEUS LI-RADS) for the diagnosis of hepatocellular carcinoma: a pictorial essay. Ultraschall Med. https://doi.org/10.1055/s-0042-124661

    Article  Google Scholar 

  25. ACR American College of Radiology: Liver Imaging Reporting and Data System https://www.acr.org/Quality-Safety/Resources/LIRADS. Accessed 12 Sep 2017

  26. Zhang YD, Zhu FP, Xu X et al (2016) Liver Imaging Reporting and Data System: substantial discordance between CT and MR for imaging classification of hepatic nodules. Acad Radiol 23:344–352

    Article  Google Scholar 

  27. Corwin MT, Fananapazir G, Jin M, Lamba R, Bashir MR (2016) Differences in liver imaging and reporting data system categorization between MRI and CT. AJR Am J Roentgenol 206:307–312

    Article  Google Scholar 

  28. Chernyak V, Flusberg M, Law A, Kobi M, Paroder V, Rozenblit AM (2017) Liver Imaging Reporting and Data System: discordance between computed tomography and gadoxetate-enhanced magnetic resonance imaging for detection of hepatocellular carcinoma major features. J Comput Assist Tomogr. https://doi.org/10.1097/rct.0000000000000642

  29. Hope TA, Aslam R, Weinstein S et al (2017) Change in Liver Imaging Reporting and Data System characterization of focal liver lesions using gadoxetate disodium magnetic resonance imaging compared with contrast-enhanced computed tomography. J Comput Assist Tomogr 41:376–381

    Article  Google Scholar 

  30. Barth BK, Donati OF, Fischer MA et al (2016) Reliability, validity, and reader acceptance of LI-RADS-An in-depth analysis. Acad Radiol 23:1145–1153

    Article  Google Scholar 

  31. Bashir MR, Huang R, Mayes N et al (2015) Concordance of hypervascular liver nodule characterization between the organ procurement and transplant network and liver imaging reporting and data system classifications. J Magn Reson Imaging 42:305–314

    Article  Google Scholar 

  32. Davenport MS, Khalatbari S, Liu PS et al (2014) Repeatability of diagnostic features and scoring systems for hepatocellular carcinoma by using MR imaging. Radiology 272:132–142

    Article  Google Scholar 

  33. Giorgio A, Montesarchio L, Gatti P et al (2016) Contrast-enhanced ultrasound: a simple and effective tool in defining a rapid diagnostic work-up for small nodules detected in cirrhotic patients during surveillance. J Gastrointestin Liver Dis 25:205–211

    PubMed  Google Scholar 

  34. Quaia E, Alaimo V, Baratella E et al (2010) Effect of observer experience in the differentiation between benign and malignant liver tumors after ultrasound contrast agent injection. J Ultrasound Med 29:25–36

    Article  Google Scholar 

  35. Berg WA, D'Orsi CJ, Jackson VP et al (2002) Does training in the Breast Imaging Reporting and Data System (BI-RADS) improve biopsy recommendations or feature analysis agreement with experienced breast imagers at mammography? Radiology 224:871–880

    Article  Google Scholar 

  36. Zhang YD, Zhu FP, Xu X et al (2016) Classifying CT/MR findings in patients with suspicion of hepatocellular carcinoma: comparison of liver imaging reporting and data system and criteria-free Likert scale reporting models. J Magn Reson Imaging 43:373–383

    Article  Google Scholar 

  37. Ehman EC, Behr SC, Umetsu SE et al (2016) Rate of observation and inter-observer agreement for LI-RADS major features at CT and MRI in 184 pathology proven hepatocellular carcinomas. Abdom Radiol (NY) 41:963–969

    Article  Google Scholar 

  38. McHugh ML (2012) Interrater reliability: the kappa statistic. Biochem Med (Zagreb) 22:276–282

    Article  Google Scholar 

Download references

Acknowledgements

We thank the editors of European Radiology and those who reviewed this article.

Funding

The authors state that this work has not received any funding.

Author information

Authors and Affiliations

  1. Department of Internal Medicine 1, Friedrich-Alexander-Universität Erlangen-Nürnberg, University Hospital Erlangen, Ulmenweg 18, 91054, Erlangen, Bayern, Germany

    Barbara Schellhaas, Deike Strobel, Lukas Pfeifer, Christian Kielisch, Ruediger S. Goertz & Markus F. Neurath

  2. Department of Radiology, Friedrich-Alexander-Universität Erlangen-Nürnberg, University Hospital Erlangen, Maximiliansplatz 3, 91054, Erlangen, Bayern, Germany

    Matthias Hammon, Alexander Cavallaro, Rolf Janka, Michael Uder & Hannes Seuss

Authors
  1. Barbara Schellhaas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthias Hammon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Deike Strobel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lukas Pfeifer
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Christian Kielisch
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ruediger S. Goertz
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Alexander Cavallaro
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Rolf Janka
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Markus F. Neurath
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Michael Uder
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Hannes Seuss
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hannes Seuss.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Dr. Hannes Seuss.

Conflict of interest

The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.

Statistics and biometry

One of the authors has significant statistical expertise.

Informed consent

Written informed consent was waived by the institutional review board.

Ethical approval

Institutional review board approval was obtained.

Methodology

• retrospective

• diagnostic study

• performed at one institution

Electronic supplementary material

ESM 1

(DOCX 164 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Schellhaas, B., Hammon, M., Strobel, D. et al. Interobserver and intermodality agreement of standardized algorithms for non-invasive diagnosis of hepatocellular carcinoma in high-risk patients: CEUS-LI-RADS versus MRI-LI-RADS. Eur Radiol 28, 4254–4264 (2018). https://doi.org/10.1007/s00330-018-5379-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-018-5379-1

Keywords

Search

Navigation