Abstract
Purpose
Deep learning (DL) is a state-of-the-art technique for developing artificial intelligence in various domains and it improves the performance of natural language processing (NLP). Therefore, we aimed to develop a DL-based NLP model that classifies the status of bone metastasis (BM) in radiology reports to detect patients with BM.
Materials and Methods
The DL-based NLP model was developed by training long short-term memory using 1,749 free-text radiology reports written in Japanese. We adopted five-fold cross-validation and used 200 reports for testing the five models. The accuracy, sensitivity, specificity, precision, and area under the receiver operating characteristics curve (AUROC) were used for the model evaluation.
Results
The developed model demonstrated classification performance with mean ± standard deviation of 0.912 ± 0.012, 0.924 ± 0.029, 0.901 ± 0.014, 0.898 ± 0.012, and 0.968 ± 0.004 for accuracy, sensitivity, specificity, precision, and AUROC, respectively.
Conclusion
The proposed DL-based NLP model may help in the early and efficient detection of patients with BM.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Savova GK, Danciu I, Alamudun F, Miller T, Lin C, Bitterman DS, et al. Use of natural language processing to extract clinical cancer phenotypes from electronic medical records. Cancer Res. 2019;79(21):5463–70.
Bao Y, Deng Z, Wang Y, Kim H, Armengol VD, Acevedo F, et al. Using machine learning and natural language processing to review and classify the medical literature on cancer susceptibility genes. JCO Clin Cancer Inform. 2019;3:1–9.
Hughes KS, Zhou J, Bao Y, Singh P, Wang J, Yin K. Natural language processing to facilitate breast cancer research and management. Breast J. 2020;26(1):92–9.
Savova GK, Masanz JJ, Ogren PV, Zheng J, Sohn S, Kipper-Schuler KC, et al. Mayo clinical text analysis and knowledge extraction system (cTAKES): architecture, component evaluation and applications. J Am Med Inform Assoc. 2010;17(5):507–13.
Dang NC, Moreno-García MN, De la Prieta F. Sentiment analysis based on deep learning: a comparative study. Electronics. 2020;9(3):483.
Yim WW, Yetisgen M, Harris WP, Kwan SW. natural language processing in oncology: a review. JAMA Oncol. 2016;2(6):797–804.
Fu S, Wyles CC, Osmon DR, Carvour ML, Sagheb E, Ramazanian T, et al. Automated detection of periprosthetic joint infections and data elements using natural language processing. J Arthroplasty. 2021;36(2):688–92.
Zhang K, Demner-Fushman D. Automated classification of eligibility criteria in clinical trials to facilitate patient-trial matching for specific patient populations. J Am Med Inform Assoc. 2017;24(4):781–7.
Cancer Statistics. Cancer Information Service, National Cancer Center, Japan (National Cancer Registry, Ministry of Health, Labour and Welfare). https://www.mhlw.go.jp/content/10900000/000942181.pdf. Accessed 23 Mar 2023
Hara H, Sakai Y, Kawamoto T, Fukase N, Kawakami Y, Takemori T, et al. Surgical outcomes of metastatic bone tumors in the extremities (Surgical outcomes of bone metastases). J Bone Oncol. 2021;27:100352.
Ulas A, Bilici A, Durnali A, Tokluoglu S, Akinci S, Silay K, et al. Risk factors for skeletal-related events (SREs) and factors affecting SRE-free survival for nonsmall cell lung cancer patients with bone metastases. Tumour Biol. 2016;37(1):1131–40.
Callen JL, Westbrook JI, Georgiou A, Li J. Failure to follow-up test results for ambulatory patients: a systematic review. J Gen Intern Med. 2012;27(10):1334–48.
Poon EG, Gandhi TK, Sequist TD, Murff HJ, Karson AS, Bates DW. I wish I had seen this test result earlier: dissatisfaction with test result management systems in primary care. Arch Intern Med. 2004;164(20):2223–8.
Singh H, Sethi S, Raber M, Petersen LA. Errors in cancer diagnosis: current understanding and future directions. J Clin Oncol. 2007;25(31):5009–18.
Do RK, Lupton K, Causa Andrieu PI, Luthra A, Taya M, Batch K, et al. Patterns of metastatic disease in patients with cancer derived from natural language processing of structured CT radiology reports over a 10-year period. Radiology. 2021;301(1):115–22.
Kehl KL, Elmarakeby H, Nishino M, Van Allen EM, Lepisto EM, Hassett MJ, et al. Assessment of deep natural language processing in ascertaining oncologic outcomes from radiology reports. JAMA Oncol. 2019;5(10):1421–9.
MeCab: Yet Another Part-of-Speech and Morphological Analyzer. https://taku910.github.io/mecab/. Accessed 12 May 2022
MANBYO Dictonary. Large-scale disease name dictionary for tabulating and analyzing diesase names acutually used in clinical settings. https://sociocom.jp/~data/2018-manbyo/index.html. Accessed 12 May 2022
Wu S, Roberts K, Datta S, Du J, Ji Z, Si Y, et al. Deep learning in clinical natural language processing: a methodical review. J Am Med Inform Assoc. 2020;27(3):457–70.
Yu Y, Si X, Hu C, Zhang J. A review of recurrent neural networks: lstm cells and network architectures. Neural Comput. 2019;31(7):1235–70.
Han S, Oh JS, Lee JJ. Diagnostic performance of deep learning models for detecting bone metastasis on whole-body bone scan in prostate cancer. Eur J Nucl Med Mol Imaging. 2022;49(2):585–95.
Zhao Z, Pi Y, Jiang L, Xiang Y, Wei J, Yang P, et al. Deep neural network based artificial intelligence assisted diagnosis of bone scintigraphy for cancer bone metastasis. Sci Rep. 2020;10(1):17046.
Macedo F, Ladeira K, Pinho F, Saraiva N, Bonito N, Pinto L, et al. Bone metastases: an overview. Oncol Rev. 2017;11(1):321.
Jang M, Kim M, Bae SJ, Lee SH, Koh JM, Kim N. Opportunistic osteoporosis screening using chest radiographs with deep learning: development and external validation with a cohort dataset. J Bone Miner Res. 2022;37(2):369–77.
Rizk B, Brat H, Zille P, Guillin R, Pouchy C, Adam C, et al. Meniscal lesion detection and characterization in adult knee MRI: A deep learning model approach with external validation. Phys Med. 2021;83:64–71.
Nguyen Q-N, Chun SG, Chow E, Komaki R, Liao Z, Zacharia R, et al. Single-fraction stereotactic vs conventional multifraction radiotherapy for pain relief in patients with predominantly nonspine bone metastases. JAMA Oncol. 2019;5(6):665.
Palma DA, Olson R, Harrow S, Gaede S, Louie AV, Haasbeek C, et al. Stereotactic ablative radiotherapy for the comprehensive treatment of oligometastatic cancers: long-term results of the sabr-comet phase ii randomized trial. J Clin Oncol. 2020;38(25):2830–8.
Kawazoe Y, Shibata D, Shinohara E, Aramaki E, Ohe K. A clinical specific bert developed using a huge Japanese clinical text corpus. PLoS ONE. 2021;16(11):0259763.
Funding
This work was supported by JSPS KAKENHI [grant number JP18K15567].
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflicts of interest.
Ethical approval
This study was performed in accordance with the Declaration of Helsinki and was approved by the Institutional Review Board of the Kansai Medical University Hospital (approval number: 2018244).
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
About this article
Cite this article
Doi, K., Takegawa, H., Yui, M. et al. Deep learning-based detection of patients with bone metastasis from Japanese radiology reports. Jpn J Radiol 41, 900–908 (2023). https://doi.org/10.1007/s11604-023-01413-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11604-023-01413-2