To the editor:
We appreciate the opportunity to discuss and clarify our findings [1]. Although the letter authors’ work complements our own [2,3,4,5], the objectives and the methods differ. For example, in Feinberg et al., Response Evaluation Criteria in Solid Tumors (RECIST) was applied to real-world imaging data that were submitted in case report forms by compensated physicians. In contrast, our objective was to capture real-world progression (rwP) for tens of thousands of patients from existing, passively collected electronic-health record (EHR) data without additional processing by the treating clinician (refer to our follow-up publication [6]).
RECIST assessment is a resource-intensive process that requires detailed documentation (e.g., initial categorization of all lesions as measurable/non-measurable and identification of target lesions) and specific conditions (e.g., imaging obtained at pre-specified intervals with comparable modalities) that do not prevail in the real world [7]. In our first experiment with 26 patients, we employed a step-wise approach to evaluate whether RECIST could be applied retrospectively to passively collected RWD from EHRs. Although 58% of the charts had radiology reports with descriptions potentially appropriate for RECIST assessment, these descriptions were not necessarily sufficient. Only 31% of charts had documentation of direct comparison of all measured lesions between two time points (23% of charts for non-measured lesions). Notably, no charts explicitly documented target lesions. Therefore, we concluded there was insufficient documentation to apply RECIST criteria to the existing, passively collected EHR data available for our unselected cohort. As the authors point out, active collection of data from EHR records or independent evaluation of raw images may yield different results.
In our second experiment, we tested three alternate approaches for capturing progression from existing EHR documentation that differed with regard to which documents abstractors reviewed. For the radiology-anchored approach, abstractors only captured progression documented in existing radiology reports (regardless of how the radiologist made the determination) and did not make an independent assessment or independently apply RECIST to information in passively collected imaging reports. In the clinician-anchored approach, abstractors only captured progression events documented in the clinician’s note (regardless of how the determination was made). The combination approach captured progression events from both sources. Additional work is needed to evaluate the generalizability of the clinician-anchored approach to abstract rwP at scale from existing documentation for patients with other solid tumors.
In response to the authors’ request for clarification about the two experimental sub-groups, the study was conducted in two stages. Patients were drawn as two independent random samples from the cohort of patients meeting inclusion/exclusion criteria. By chance, one patient overlapped between the two sub-groups.
The clinical and research solutions proposed by the authors (RECIST training for radiologists, retrospective review of raw imaging, and adaptation of EHR workflows) must be weighed against their cost, time, and incremental clinical utility (e.g., is a formal RECIST assessment always necessary when the cancer is generally stable?). We developed an approach to capture rwP in a commonly encountered RWD context: large volumes of static, passively collected data in which imaging results are only available in unstructured reports and re-contact is not possible. We look forward to the continuing evolution of the RWD field for the benefits of patients through the contributions and collaborations of many stakeholders, including academic researchers, regulators, industry partners, and patients.
Sandra Griffith and Rebecca Miksad (on behalf of the co-authors).
References
Griffith SD, Tucker M, Bowser B, et al. Generating real-world tumor burden endpoints from electronic health record data: comparison of RECIST, radiology-anchored, and clinician-anchored approaches for abstracting real-world progression in non-small cell lung cancer. Adv Ther. 2019;36(8):2122–36.
Feinberg BA, Bharmal M, Klink AJ, et al. Using response evaluation criteria in solid tumors in real-world evidence cancer research. Future Oncol. 2018;14(27):2841–8.
Luke JJ, Ghate SR, Kish J, et al. BA. Targeted agents or immuno-oncology therapies as first-line therapy for BRAF-mutated metastatic melanoma: a real-world study. Future Oncol. 2019;15(25):2933–42.
Chatterjee D, Kish J, Wang EC, et al. Real-world outcomes with multiple second-line treatments after first-line lenvatinib in patients with radioactive-iodine refractory differentiated thyroid cancer (RAI-R DTC). Thyroid. 2019;29(s1): Poster 386. Presented at the 89th Annual Meeting of the American Thyroid Association.
Mougalian S, Wang E, Alexis K, et al. Utilization and outcomes of Eribulin Mesylate POst a cyclin-dependent kinase 4/6 inhibitor (CDK 4/6i): an observational real-World study in UnitEd States community oncology pRactices (EMPOWER). Ann Oncol. 2019;30(Supplement_3):mdz100-018. Poster 167P. Presented at the 2019 ESMO Breast Cancer Congress.
Griffith SD, Miksad RA, Calkins G, et al. Characterizing the feasibility and performance of real-world tumor progression end points and their association with overall survival in a large advanced non–small-cell lung cancer data set. JCO Clin Cancer Inform. 2019;3:1–13.
Eisenhauer EA, Therasse P, Bogaerts J. New response evaluation criteria in solid tumours: Revised RECIST guideline (version 1.1). Eur J Cancer. 2009;45(2):228–47.
Acknowledgements
Funding
The study in the originating article was sponsored by Flatiron Health Inc., which is an independent subsidiary of the Roche group. No additional funding was received for the publication of this letter.
Editorial Assistance
The authors thank Cody Patton, from Flatiron Health, for editorial assistance.
Authorship
All named authors meet the International Committee of Medical Journal Editors (ICMJE) criteria for authorship for this article, take responsibility for the integrity of the work as a whole, and have given their approval for this version to be published.
Disclosures
S.D.G, M.T., B.B., G.C., C.C., J.K., and P.Y. report employment at Flatiron Health Inc, which is an independent subsidiary of the Roche Group, and also report equity ownership in Flatiron Health Inc. and stock ownership in Roche. R.A.M reports employment at Flatiron Health Inc., which is an independent subsidiary of the Roche Group, and also reports equity ownership in Flatiron Health Inc. and stock ownership in Roche. She also reports being an advisor for the De Luca Foundation and a grant review committee member for the American Association for Cancer Research. E.G. reports employment at Genentech. D.S. reports work on related projects to augment capacity for using “real world” data in partnership with AACR’s project GENIE and research funding for GENIE has been awarded to Dana Farber Cancer Institute; she is a compensated Associate Editor of the Journal JAMA. S.K. declares that he has no conflicts to disclose.
Compliance with Ethics Guidelines
This is a response to a letter regarding previously conducted studies and does not contain any studies with human participants or animals performed by any of the authors.
Peer Review
Please note, contrary to the journal’s standard single-blind peer-review process, as a letter this article underwent review by a member of the journal’s Editorial Board.
Author information
Authors and Affiliations
Corresponding author
Additional information
This is a response to the letter to the editor by Gajra and Feinberg regarding the article: “Generating Real-World Tumor Burden Endpoints From Electronic Health Record Data: Comparison of RECIST, Radiology-Anchored, and Clinician-Anchored Approaches for Abstracting Real-World Progression in Non-Small Cell Lung Cancer”.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc/4.0/.
About this article
Cite this article
Griffith, S.D., Tucker, M., Bowser, B. et al. Author’s Response to “A Letter in Support of Real-World RECIST”. Adv Ther 37, 1691–1693 (2020). https://doi.org/10.1007/s12325-020-01246-9
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12325-020-01246-9