Skip to main content
SpringerLink
Log in

Inter-operator variability and source of errors in tumour response assessment for hepatocellular carcinoma treated with sorafenib

  • Gastrointestinal
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

To assess the inter-operator concordance and the potential sources of discordance in defining response to sorafenib in hepatocellular carcinoma (HCC).

Methods

All patients who received sorafenib between September 2008 and February 2015 were scrutinised for this retrospective study. Images were evaluated separately by three radiologists with different expertise in liver imaging (operator 1, >10 years; operator 2, 5 years; operator 3, no specific training in liver imaging), according to: response evaluation radiological criteria in solid tumours (RECIST) 1.1, modified RECIST (mRECIST) and response evaluation criteria in cancer of the liver (RECICL).

Results

The overall response concordance between the more expert operators was good, irrespective of the criteria (RECIST 1.1, ĸ = 0.840; mRECIST, ĸ = 0.871; RECICL, ĸ = 0.819). Concordance between the less expert operator and the other colleagues was lower. The most evident discordance was in target lesion response assessment, with expert operators disagreeing mostly on lesion selection and less expert operators on lesion measurement. As a clinical correlate, overall survival was more tightly related with “progressive disease” as assessed by the expert compared to the same assessment performed by operator 3.

Conclusions

Decision on whether a patient is a responder or progressor under sorafenib may vary among different operators, especially in case of a non-specifically trained radiologist. Regardless of the adopted criteria, patients should be evaluated by experienced radiologists to minimise variability in this critical instance.

Key Points

• Inter-operator variability in the assessment of response to sorafenib is poorly known.

• The concordance between operators with expertise in liver imaging was good.

• Target lesions selection was the main source of discordance between expert operators.

• Concordance with non-specifically trained operator was lower, independently from the response criteria.

• The non-specifically trained operator was mainly discordant in measurements of target lesions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Abbreviations

HCC:

Hepatocellular carcinoma

PD:

Progressive disease

NTL:

Non-target lesion

RECICL:

Response evaluation criteria in cancer of the liver

RECIST:

Response evaluation radiological criteria in solid tumours

References

  1. Nagilla M, Brown RL, Cohen EE (2012) Cabozantinib for the treatment of advanced medullary thyroid cancer. Adv Ther 29:925–934

    Article  PubMed  CAS  Google Scholar 

  2. Mocellin S, Baretta Z, Roqué I et al (2017) Second-line systemic therapy for metastatic colorectal cancer. Cochrane Database Syst Rev 1:CD006875

    PubMed  Google Scholar 

  3. Randrup Hansen C, Grimm D, Bauer J, Wehland M, Magnusson NE (2017) Effects and side effects of using sorafenib and sunitinib in the treatment of metastatic renal cell carcinoma. Int J Mol Sci 18. https://doi.org/10.3390/ijms18020461

  4. Wang Y, Schmid-Bindert G, Zhou C (2012) Erlotinib in the treatment of advanced non-small cell lung cancer: an update for clinicians. Ther Adv Med Oncol 4:19–29

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  5. Therasse P, Arbuck SG, Eisenhauer EA et al (2000) New guidelines to evaluate the response to treatment in solid tumors. European Organization for Research and Treatment of Cancer, National Cancer Institute of the United States, National Cancer Institute of Canada. J Natl Cancer Inst 92:205–216

    Article  PubMed  CAS  Google Scholar 

  6. Lencioni R, Llovet JM (2010) Modified RECIST (mRECIST) assessment for hepatocellular carcinoma. Semin Liver Dis 30:52–60

    Article  PubMed  CAS  Google Scholar 

  7. Kudo M, Ueshima K, Kubo S et al (2015) Response Evaluation Criteria in Cancer of the Liver (RECICL) (2015 Revised version). Hepatol Res 46:3–9

    Article  PubMed  Google Scholar 

  8. Choi H, Charnsangavej C, Faria SC et al (2007) Correlation of computed tomography and positron emission tomography in patients with metastatic gastrointestinal stromal tumor treated at a single institution with imatinib mesylate: proposal of new computed tomography response criteria. J Clin Oncol 25:1753–1759

    Article  PubMed  Google Scholar 

  9. Llovet JM, Ricci S, Mazzaferro V et al (2008) Sorafenib in advanced hepatocellular carcinoma. N Engl J Med 359:378–390

    Article  PubMed  CAS  Google Scholar 

  10. Bruix J, Sherman M, Practice Guidelines Committee, American Association for the Study of Liver Diseases (2005) Management of hepatocellular carcinoma. Hepatology 42:1208–1236

    Article  PubMed  Google Scholar 

  11. Renzulli M, Brocchi S, Cucchetti A et al (2016) Can current preoperative imaging be used to detect microvascular invasion of hepatocellular carcinoma? Radiology. 279:432–442

    Article  PubMed  Google Scholar 

  12. Sneag DB, Krajewski K, Giardino A et al (2011) Extrahepatic spread of hepatocellular carcinoma: spectrum of imaging findings. AJR Am J Roentgenol 197:W658–W664

    Article  PubMed  Google Scholar 

  13. Katyal S, Oliver JH 3rd, Peterson MS et al (2000) Extrahepatic metastases of hepatocellular carcinoma. Radiology 216:698-703

  14. Fleiss JL (1971) Measuring nominal scale agreement among many raters. Psychological Bulletin 76:352–378

    Article  Google Scholar 

  15. Portney LG, Watkins MP (2009) Foundations of clinical research: applications to practice, 3rd edn. Pearson/Prentice Hall, Upper Saddle River

    Google Scholar 

  16. Bruix J, Qin S, Merle P et al (2017) Regorafenib for patients with hepatocellular carcinoma who progressed on sorafenib treatment (RESORCE): a randomised, double-blind, placebo-controlled, phase 3 trial. Lancet 389:56–66

    Article  PubMed  CAS  Google Scholar 

  17. Bargellini I, Scionti A, Mismas V et al (2014) Identification of responders to sorafenib in hepatocellular carcinoma: is tumor volume measurement the way forward? Oncology. 86:191–198

    Article  PubMed  CAS  Google Scholar 

  18. Sato Y, Watanabe H, Sone M et al (2013) Tumor response evaluation criteria for HCC (hepatocellular carcinoma) treated using TACE (transcatheter arterial chemoembolization): RECIST (response evaluation criteria in solid tumors) version 1.1 and mRECIST (modified RECIST): JIVROSG-0602. Ups J Med Sci 118:16–22

    Article  PubMed  PubMed Central  Google Scholar 

  19. Seyal AR, Gonzalez-Guindalini FD, Arslanoglu A et al (2015) Reproducibility of mRECIST in assessing response to transarterial radioembolization therapy in hepatocellular carcinoma. Hepatology 62:1111–1121

    Article  PubMed  CAS  Google Scholar 

  20. Lee S, Kim JH, Lee JH et al (2018) Non-invasive monitoring of the therapeutic response in sorafenib-treated hepatocellular carcinoma based on photoacoustic imaging. Eur Radiol 28:372–381

    Article  PubMed  Google Scholar 

  21. Tovoli F, Renzulli M, Granito A et al (2017) Radiologic criteria of response to systemic treatments for hepatocellular carcinoma. Hepatic Oncology 4:129–137

    Article  PubMed  PubMed Central  Google Scholar 

  22. Lencioni R, Montal R, Torres F et al (2017) Objective response by mRECIST as a predictor and potential surrogate end-point of overall survival in advanced HCC. J Hepatol 66:1166–1172

    Article  PubMed  Google Scholar 

  23. Bruix J, Reig M, Sangro B (2017) Assessment of treatment efficacy in hepatocellular carcinoma: Response rate, delay in progression or none of them. J Hepatol 66:1114–1117

    Article  PubMed  Google Scholar 

  24. Ronot M, Bouattour M, Wassermann J et al (2014) Alternative Response Criteria (Choi, European association for the study of the liver, and modified Response Evaluation Criteria in Solid Tumors [RECIST]) Versus RECIST 1.1 in patients with advanced hepatocellular carcinoma treated with sorafenib. Oncologist 19:394–402

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  25. Gavanier M, Ayav A, Sellal C et al (2016) CT imaging findings in patients with advanced hepatocellular carcinoma treated with sorafenib: alternative response criteria (Choi), European Association for the Study of the Liver, and modified Response Evaluation Criteria in Solid Tumor (mRECIST) versus RECIST 1.1. Eur J Radiol 85:103–112

    Article  PubMed  CAS  Google Scholar 

  26. Arizumi T, Ueshima K, Takeda H et al (2014). Comparison of systems for assessment of post-therapeutic response to sorafenib for hepatocellular carcinoma. J Gastroenterol 49:1578-1587

Download references

Funding

The authors state that this work has not received any funding.

Author information

Author notes

  1. Francesco Tovoli and Matteo Renzulli contributed equally to this work.

Authors and Affiliations

  1. Department of Medical and Surgical Sciences, University of Bologna, via Massarenti 9, Bologna, Italy

    Francesco Tovoli, Giulia Negrini, Alessia Ferrarini, Francesca Benevento, Antonio Maria Morselli-Labate, Marianna Mastroroberto & Fabio Piscaglia

  2. Radiology Unit, S.Orsola-Malpighi Bologna Authority Hospital, Bologna, Italy

    Matteo Renzulli, Stefano Brocchi, Andrea Andreone & Rita Golfieri

  3. Radiology and Medical Imaging Unit, Regional Institute of Gastroenterology and Hepatology, Cluj Napoca, Romania

    Radu Ion Badea

Authors
  1. Francesco Tovoli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matteo Renzulli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Giulia Negrini
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stefano Brocchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alessia Ferrarini
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Andrea Andreone
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Francesca Benevento
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Rita Golfieri
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Antonio Maria Morselli-Labate
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Marianna Mastroroberto
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Radu Ion Badea
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Fabio Piscaglia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Francesco Tovoli.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Fabio Piscaglia.

Conflict of interest

The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.

Statistics and biometry

One of the authors has significant statistical expertise.

Informed consent

Written informed consent was obtained from all subjects (patients) in this study.

Ethical approval

Institutional Review Board approval was obtained.

Methodology

• retrospective

• observational/experimental

• performed at one institution

Electronic supplementary material

ESM 1

(DOC 27 kb)

ESM 2

(DOCX 16 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tovoli, F., Renzulli, M., Negrini, G. et al. Inter-operator variability and source of errors in tumour response assessment for hepatocellular carcinoma treated with sorafenib. Eur Radiol 28, 3611–3620 (2018). https://doi.org/10.1007/s00330-018-5393-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-018-5393-3

Keywords

Search

Navigation