AI-Driven Pathology Laboratory Utilization Management via Data- and Knowledge-Based Analytics
- 821 Downloads
Inappropriate pathology test orders are an economic burden on laboratories and compromise patient safety. We pursue a laboratory utilization management strategy that involves raising awareness amongst physicians regarding their test ordering behaviour. We are employing an AI-driven approach for laboratory utilization management, whereby we apply both machine learning and semantic reasoning methods to analyze pathology laboratory data. We are analyzing over 6-years of primary care physician’s pathology test order ‘big’ data. Our analysis generates physician order profiles, based on their case-mix and orders-sets, to inform physicians about their laboratory utilization. We developed an AI-driven platform—i.e. Pathology Laboratory Utilization Scorecards (PLUS) that offers an interactive means for physicians to self-examine their test ordering pattern. PLUS aims to optimize the utilization of the Central Zone pathology laboratory of the Nova Scotia Health Authority.
KeywordsMachine learning Data analytics Semantic web Pathology Laboratory utilization Big data
Pathology laboratory testing is central to medical practice as most diagnostic and therapeutic decisions are guided by the patient’s pathology test results. Pathology tests are routinely ordered by physicians and it has been observed that a significant number of tests ordered are inappropriate—i.e. the test is either redundant, clinically irrelevant or non-compliant with clinical guidelines. There are multiple reasons for the inappropriate ordering of pathology lab tests including inconsistencies in test nomenclature , poor implementation of evidence-based guidelines  and physician’s discretionary behaviour when ordering tests . A meta-analysis of 108 studies, examining 1.6 million results from 46 of the 50 most commonly ordered pathology tests, concluded that on average 30% of all tests ordered by physicians were likely to be inappropriate . Inappropriate pathology test ordering  not only affects laboratory resource utilization, but it also compromises patient safety by producing falsely abnormal results which may require unnecessary interventions . Given rising healthcare costs whilst the need to meet quality and efficiency targets, there is an awareness to minimize inappropriate pathology testing. In Canada, the ‘Choosing Wisely’ initiative aims to optimize healthcare services by reducing waste, and pathology test ordering is an area that needs innovative strategies to minimize inappropriate test ordering by physicians . Utilization management is a strategy to evaluate the appropriateness and efficiency of healthcare services. As such, pathology laboratory utilization management aims to optimize pathology test ordering—i.e. the right test is ordered at the right time for the right patient—by reducing both over- and under-utilization of the pathology laboratory.
In our work, we pursue pathology laboratory utilization management by raising awareness amongst physicians about their inappropriate test ordering behaviour. Our approach is to provide physicians personalized insights into their laboratory utilization profile and peer comparisons via a self-auditing tool . In this paper, we present an AI-based framework for laboratory utilization management that employs (1) machine learning methods to tackle overutilization of laboratory tests by (a) clustering physicians based on their patient case-mix for inter-physician peer comparisons; (b) using association rules to identify the unconventional order-sets of individual physicians with respect to their peers; and (2) knowledge-based reasoning to tackle underutilization of laboratory tests by implementing test appropriateness rules to recommend essential tests in response to the results of prior tests. We have implemented a Pathology Laboratory Utilization Scorecard (PLUS) platform that offers (i) scorecards for physicians to examine their test ordering pattern over time, and compare it with peers having the same case-mix; and (ii) dashboards for laboratory managers to assist with waste minimization. PLUS has been implemented to optimize the Central Zone pathology laboratory in Halifax that processes 8 million general pathology test orders yearly.
2 Laboratory Utilization Management Approaches: A Review
Current approaches for laboratory utilization management focus on reducing physician options in test ordering, physician education, decision support and peer comparisons.
Strategies to reduce physician options to tests include specialist vetting of lab orders  and at the CPOE level McDonald et al. removed the option to order daily tests beyond 2 days , Neilson et al.  utilized prompts to reduce the ease of repeating targeted tests, and Iturrate et al.  disallowed daily recurring tests entirely. In terms of physician education, Ryskina et al.  provided social comparison feedback linked to patients’ EMR records, Bunting et al.  discussed with the physician their lab utilization and compared it with other physicians; Iams et al.  sent out weekly feedback e-mails to physicians comparing their lab ordering rates with the ordering rates of all others as well as the pre-set goal ordering rates; and. Srivastava et al.  illustrated the utility of reflex rules (and reflective testing) in lab test recommendation.
Pathology laboratory utilization management strategies have yielded encouraging results, for instance a saving of 19% of the total costs for genetic test orders by reviewing each ordered test , a 21% reduction in B-type natriuretic peptide test orders by employing a decision support system , unbundling of test panels and providing pocket cards with laboratory test costs to physicians resulted in a 21% reduction in costs , and an 8% test volume reduction was noted by leveraging social influence of opinion leaders, academic detailing, and giving test prices in newsletters to physicians . The pathology department in Halifax provided written feedback to individual primary care physicians about their orders for specific tests and a 25% reduction in orders was noted . In a Canadian study , physicians were provided feedback on their laboratory utilization rate along with peer-comparisons, and as a result a reduction in inappropriate ordering by physicians was noted.
We note from pathology laboratory utilization management strategies that addressing inappropriate test orders at the physician level can yield the highest impact given the significant variability in the test ordering pattern of physicians despite them treating patients with the same diagnosis. This alludes to physician’s discretionary behaviour when ordering tests —a behaviour that can be modified by providing physicians with education, self-audit of test ordering profile  and peer comparisons.
3 AI-Driven Laboratory Utilization Management Approach
We are targeting laboratory utilization optimization at the primary care physician level since they are the heaviest users of the pathology laboratory. To minimize inappropriate testing, our strategy is to engage physicians to (a) self-examine their test ordering pattern and its implications on laboratory utilization, (b) show how their laboratory utilization compares with their peers, and (c) recommend essential follow-up tests.
We argue that peer comparisons are meaningful when a physician is compared with similar physicians as opposed to all physicians. Typically, peer comparison of physician’s pathology test ordering profile is based on the type, volume and frequency of tests they order . However, such peer comparisons are inconclusive as it does not consider the physician’s patient case-mix—i.e. if a physician is treating more elderly patients with chronic kidney and cardiac conditions then a higher volume of CBC and creatinine tests is not an inappropriate test ordering pattern; hence, they should not be flagged as a high laboratory user compared to a physician treating younger patients.
To determine the laboratory overutilization by a physician, our approach is to generate a multi-faceted physician test ordering profile that takes into account: (i) patient case-mix managed by the physician to justify the test orders based on medical necessity, and in turn to allow a fair comparison with peers having a similar case-mix; (ii) test co-occurrence pattern (i.e. order-sets) to determine the medical necessity of tests that are frequently ordered simultaneously; (iii) temporal variations in ordering patterns to account for seasonal needs, and also to detect changes in the physician’s knowledge and guideline compliance over time, and (iv) geographical location of the physician to provide a fairer comparison with peers practicing in the same health zone. Machine learning methods, applied to a ‘big’ pathology laboratory order dataset, are used to generate a physician ordering profile to determine laboratory overutilization by the physician.
To handle laboratory underutilization by physicians our approach is to computerize diagnostic testing rules, derived from clinical guidelines and domain experts, that recommend follow-up laboratory tests that are essential for the diagnostic process in a timely and safe manner. We use semantic web based ontologies and decision rules to represent the test ordering protocols, and we apply logic-based reasoning to the decision rules to determine the follow-up tests based on the results of the ordered tests.
To engage physicians to perform a self-audit of their laboratory utilization, our approach is to provide them a web-based interactive physician-specific scorecard that they can securely access and privately view to examine their overall test ordering profile and its implications on the provincial laboratory’s utilization.
4 Our Pathology Test Ordering Data
Annual volume of general orders for 15 common tests.
5 Data Analytics for Laboratory Utilization Management to Minimize Laboratory Overutilization by Physicians
5.1 Data Clustering to Generate Physician Case-Mix Clusters
A tenet of our approach for minimizing overutilization of laboratory is to provide physicians with a comparison of their laboratory utilization with that of their peers. The key to peer comparison is that a physician is compared only with physicians that have a similar practice and case-mix of patients, and not with all physicians in the province.
We used machine learning based clustering methods to generate groups of physicians with similar case-mix of patients. Our dataset does not include the patient’s diagnosis which is essential to determine a physician’s case-mix. Given that specific tests are ordered to confirm the presence/absence of specific diseases, we can assume that physicians having patients with specific diseases will order more tests associated with those diseases and the test results will further confirm that the physician is treating a patient with a specific disease. For instance, high abnormal values for the potassium test and low abnormal values for the sodium and glucose AC tests are associated with Addison’s disease. Thus, physicians treating a high number of patients with Addison’s disease will order the potassium, sodium, and glucose AC tests in higher proportions than a physician treating less patients with Addison’s disease. Building on existing mappings between pathology tests and diseases, we developed a test-disease mapping between 26 diseases and 40 pathology tests which were validated by pathologists.
5.2 Association Rule Mining to Generate Test Order-Sets
Frequent order-sets at the regional level for the period of Jul-2011 to Jun-2014
Frequent order-sets at the regional level for the period of Jul-2014 to May-2017
When comparing the order-sets over the two 3-year time periods, we noted that in general the most frequent order-sets remain the same over time—out of the 14 order-sets (i.e. comprising 1–14 items), 7 order-sets remained the most frequent. This suggests that the order-sets are well-defined with fluctuations in their ordering frequency over time. The identified order-sets were implemented within the PLUS system as a benchmark for a physician to compare his/her order-sets with peers, where the unit of comparison is test order-set as opposed to individual test orders.
6 Knowledge-Based Analytics to Overcome Laboratory Underutilization by Physicians
Underutilization of laboratory—i.e. physicians not prescribing tests that are needed, or should be ordered as a follow-up to the earlier tests—leads to future increased laboratory utilization, delays in proper treatments and compromises patient safety.
As future work, we target to link our reflex testing strategy via PLUS with the LIS’s clinical pathway to recommend and conduct appropriate reflective pathology tests, based on the latest evidence and test results, to aid in accurate and early diagnosis.
7 Visualization of Laboratory Utilization: The PLUS System
Advance data visualization has been implemented for users to dynamically interact with the data analysis. PLUS provides a web-based (a) scorecard for physicians to understand their own test ordering profile over time and across different patient cohorts, and to compare their laboratory utilization (adjusted to case-mix) with similar peers. The scorecard presents physician’s laboratory utilization in terms of abnormal results, test volumes and frequency over time, peer comparisons, cost incurred, case-mix and the rate and cost of inappropriate test orders. The physician’s scorecard is private and cannot be viewed by practice auditing bodies; and (ii) dashboard for laboratory managers provides broad operational intelligence by aggregating the patient-level analytics to the regional level, displaying tests ordered, completion rates and flagged as inappropriate.
8 Concluding Remarks
As health care transitions from volume- to value-based care, there is an increasing need for efficient and effective laboratory utilization by physicians. In this paper, we presented an innovative and sustainable laboratory utilization management approach, targeting physicians, that leverages (a) data analytics methods to develop and understand each physician’s test ordering profiles; and (b) data visualization techniques to display the physician’s test ordering pattern as an interactive scorecard so that they can self-audit and -regulate their test ordering behaviour. We posit that given healthcare budgetary pressures and increasing test volumes, our sustainable, data-informed and physician-engaged approach will help to minimize inappropriate laboratory utilization, improve sustainability of the laboratory operations, and achieve value-based care. PLUS is being implemented to optimize test utilization at the Central Zone pathology laboratory in Nova Scotia. As literature estimates that a minimum of 25% of tests are inappropriately ordered [1, 2, 3], we believe that the utilization of Central Zone laboratories (in Halifax) can potentially be reduced by 2 million tests annually from the 8 million currently performed, leading to huge cost savings and improved patient safety.
We thank the NSHA Central Zone pathology lab for supporting the project, and Nova Scotia Health Research Foundation for giving the catalyst grant.
- 1.Freedman, D.B.: Towards better test utilization - strategies to improve physician ordering and their impact on patient outcomes. EJIFCC 26, 15–30 (2015)Google Scholar
- 7.Choosing Wisely Canada. www.choosingwiselycanada.org/about/what-is-cwc/
- 26.Abusharekh, A., Stewart, S., Abidi, S.S.R.: H-DRIVE: a big data analytics platform for evidence-informed decision making. In: 4th IEEE International Congress on Big Data, New York (2015)Google Scholar
- 27.Cha, S., Abusharekh, A., Abidi, S.S.R.: Towards a ‘big’ health data analytics platform. In: 1st IEEE International Conference on Big Data Computing Service and Applications, San Francisco (2015)Google Scholar