Advertisement

AI-Driven Pathology Laboratory Utilization Management via Data- and Knowledge-Based Analytics

  • Syed Sibte Raza AbidiEmail author
  • Jaber Rad
  • Ashraf Abusharekh
  • Patrice C. Roy
  • William Van Woensel
  • Samina R. Abidi
  • Calvino Cheng
  • Bryan Crocker
  • Manal Elnenaei
Conference paper
  • 821 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11526)

Abstract

Inappropriate pathology test orders are an economic burden on laboratories and compromise patient safety. We pursue a laboratory utilization management strategy that involves raising awareness amongst physicians regarding their test ordering behaviour. We are employing an AI-driven approach for laboratory utilization management, whereby we apply both machine learning and semantic reasoning methods to analyze pathology laboratory data. We are analyzing over 6-years of primary care physician’s pathology test order ‘big’ data. Our analysis generates physician order profiles, based on their case-mix and orders-sets, to inform physicians about their laboratory utilization. We developed an AI-driven platform—i.e. Pathology Laboratory Utilization Scorecards (PLUS) that offers an interactive means for physicians to self-examine their test ordering pattern. PLUS aims to optimize the utilization of the Central Zone pathology laboratory of the Nova Scotia Health Authority.

Keywords

Machine learning Data analytics Semantic web Pathology Laboratory utilization Big data 

1 Introduction

Pathology laboratory testing is central to medical practice as most diagnostic and therapeutic decisions are guided by the patient’s pathology test results. Pathology tests are routinely ordered by physicians and it has been observed that a significant number of tests ordered are inappropriate—i.e. the test is either redundant, clinically irrelevant or non-compliant with clinical guidelines. There are multiple reasons for the inappropriate ordering of pathology lab tests including inconsistencies in test nomenclature [1], poor implementation of evidence-based guidelines [2] and physician’s discretionary behaviour when ordering tests [3]. A meta-analysis of 108 studies, examining 1.6 million results from 46 of the 50 most commonly ordered pathology tests, concluded that on average 30% of all tests ordered by physicians were likely to be inappropriate [4]. Inappropriate pathology test ordering [5] not only affects laboratory resource utilization, but it also compromises patient safety by producing falsely abnormal results which may require unnecessary interventions [6]. Given rising healthcare costs whilst the need to meet quality and efficiency targets, there is an awareness to minimize inappropriate pathology testing. In Canada, the ‘Choosing Wisely’ initiative aims to optimize healthcare services by reducing waste, and pathology test ordering is an area that needs innovative strategies to minimize inappropriate test ordering by physicians [7]. Utilization management is a strategy to evaluate the appropriateness and efficiency of healthcare services. As such, pathology laboratory utilization management aims to optimize pathology test ordering—i.e. the right test is ordered at the right time for the right patient—by reducing both over- and under-utilization of the pathology laboratory.

In our work, we pursue pathology laboratory utilization management by raising awareness amongst physicians about their inappropriate test ordering behaviour. Our approach is to provide physicians personalized insights into their laboratory utilization profile and peer comparisons via a self-auditing tool [8]. In this paper, we present an AI-based framework for laboratory utilization management that employs (1) machine learning methods to tackle overutilization of laboratory tests by (a) clustering physicians based on their patient case-mix for inter-physician peer comparisons; (b) using association rules to identify the unconventional order-sets of individual physicians with respect to their peers; and (2) knowledge-based reasoning to tackle underutilization of laboratory tests by implementing test appropriateness rules to recommend essential tests in response to the results of prior tests. We have implemented a Pathology Laboratory Utilization Scorecard (PLUS) platform that offers (i) scorecards for physicians to examine their test ordering pattern over time, and compare it with peers having the same case-mix; and (ii) dashboards for laboratory managers to assist with waste minimization. PLUS has been implemented to optimize the Central Zone pathology laboratory in Halifax that processes 8 million general pathology test orders yearly.

2 Laboratory Utilization Management Approaches: A Review

Current approaches for laboratory utilization management focus on reducing physician options in test ordering, physician education, decision support and peer comparisons.

Strategies to reduce physician options to tests include specialist vetting of lab orders [9] and at the CPOE level McDonald et al. removed the option to order daily tests beyond 2 days [10], Neilson et al. [11] utilized prompts to reduce the ease of repeating targeted tests, and Iturrate et al. [12] disallowed daily recurring tests entirely. In terms of physician education, Ryskina et al. [13] provided social comparison feedback linked to patients’ EMR records, Bunting et al. [14] discussed with the physician their lab utilization and compared it with other physicians; Iams et al. [15] sent out weekly feedback e-mails to physicians comparing their lab ordering rates with the ordering rates of all others as well as the pre-set goal ordering rates; and. Srivastava et al. [16] illustrated the utility of reflex rules (and reflective testing) in lab test recommendation.

Pathology laboratory utilization management strategies have yielded encouraging results, for instance a saving of 19% of the total costs for genetic test orders by reviewing each ordered test [17], a 21% reduction in B-type natriuretic peptide test orders by employing a decision support system [18], unbundling of test panels and providing pocket cards with laboratory test costs to physicians resulted in a 21% reduction in costs [19], and an 8% test volume reduction was noted by leveraging social influence of opinion leaders, academic detailing, and giving test prices in newsletters to physicians [19]. The pathology department in Halifax provided written feedback to individual primary care physicians about their orders for specific tests and a 25% reduction in orders was noted [20]. In a Canadian study [14], physicians were provided feedback on their laboratory utilization rate along with peer-comparisons, and as a result a reduction in inappropriate ordering by physicians was noted.

We note from pathology laboratory utilization management strategies that addressing inappropriate test orders at the physician level can yield the highest impact given the significant variability in the test ordering pattern of physicians despite them treating patients with the same diagnosis. This alludes to physician’s discretionary behaviour when ordering tests [3]—a behaviour that can be modified by providing physicians with education, self-audit of test ordering profile [21] and peer comparisons.

3 AI-Driven Laboratory Utilization Management Approach

We are targeting laboratory utilization optimization at the primary care physician level since they are the heaviest users of the pathology laboratory. To minimize inappropriate testing, our strategy is to engage physicians to (a) self-examine their test ordering pattern and its implications on laboratory utilization, (b) show how their laboratory utilization compares with their peers, and (c) recommend essential follow-up tests.

We argue that peer comparisons are meaningful when a physician is compared with similar physicians as opposed to all physicians. Typically, peer comparison of physician’s pathology test ordering profile is based on the type, volume and frequency of tests they order [22]. However, such peer comparisons are inconclusive as it does not consider the physician’s patient case-mix—i.e. if a physician is treating more elderly patients with chronic kidney and cardiac conditions then a higher volume of CBC and creatinine tests is not an inappropriate test ordering pattern; hence, they should not be flagged as a high laboratory user compared to a physician treating younger patients.

To determine the laboratory overutilization by a physician, our approach is to generate a multi-faceted physician test ordering profile that takes into account: (i) patient case-mix managed by the physician to justify the test orders based on medical necessity, and in turn to allow a fair comparison with peers having a similar case-mix; (ii) test co-occurrence pattern (i.e. order-sets) to determine the medical necessity of tests that are frequently ordered simultaneously; (iii) temporal variations in ordering patterns to account for seasonal needs, and also to detect changes in the physician’s knowledge and guideline compliance over time, and (iv) geographical location of the physician to provide a fairer comparison with peers practicing in the same health zone. Machine learning methods, applied to a ‘big’ pathology laboratory order dataset, are used to generate a physician ordering profile to determine laboratory overutilization by the physician.

To handle laboratory underutilization by physicians our approach is to computerize diagnostic testing rules, derived from clinical guidelines and domain experts, that recommend follow-up laboratory tests that are essential for the diagnostic process in a timely and safe manner. We use semantic web based ontologies and decision rules to represent the test ordering protocols, and we apply logic-based reasoning to the decision rules to determine the follow-up tests based on the results of the ordered tests.

To engage physicians to perform a self-audit of their laboratory utilization, our approach is to provide them a web-based interactive physician-specific scorecard that they can securely access and privately view to examine their overall test ordering profile and its implications on the provincial laboratory’s utilization.

4 Our Pathology Test Ordering Data

Our dataset comprises pathology test orders received by the Central Zone pathology lab (in Halifax) during the period 2011–2017. We analyzed 15 general tests—i.e. PT, CBC Auto Diff, Creatinine, Alkaline Phosphatase, Urea, Electrolyte Panel, AST, ALT, GGT, Glucose AC, Glucose Random, Cholesterol, HDL Cholesterol, Triglycerides, TSH. Note that a single test may comprise 1 or more procedures (i.e. the CBC test comprises 11 procedures, each generating an individual result), and a physician can order one or more tests for a patient in a single test order. The dataset covers around 2000 physicians and 250,000 patients. The annual breakdown of test orders is given in Table 1.
Table 1.

Annual volume of general orders for 15 common tests.

2012

2013

2014

2015

2016

2017

1,447,798

1,481,788

1,567,964

1,569,422

1,583,671

1,679,531

5 Data Analytics for Laboratory Utilization Management to Minimize Laboratory Overutilization by Physicians

5.1 Data Clustering to Generate Physician Case-Mix Clusters

A tenet of our approach for minimizing overutilization of laboratory is to provide physicians with a comparison of their laboratory utilization with that of their peers. The key to peer comparison is that a physician is compared only with physicians that have a similar practice and case-mix of patients, and not with all physicians in the province.

We used machine learning based clustering methods to generate groups of physicians with similar case-mix of patients. Our dataset does not include the patient’s diagnosis which is essential to determine a physician’s case-mix. Given that specific tests are ordered to confirm the presence/absence of specific diseases, we can assume that physicians having patients with specific diseases will order more tests associated with those diseases and the test results will further confirm that the physician is treating a patient with a specific disease. For instance, high abnormal values for the potassium test and low abnormal values for the sodium and glucose AC tests are associated with Addison’s disease. Thus, physicians treating a high number of patients with Addison’s disease will order the potassium, sodium, and glucose AC tests in higher proportions than a physician treating less patients with Addison’s disease. Building on existing mappings between pathology tests and diseases, we developed a test-disease mapping between 26 diseases and 40 pathology tests which were validated by pathologists.

To generate the dataset for clustering physicians based on their case-mix, we retrieved the test orders of each physician for each of the 40 test types with their results, and then applied the test-disease mapping to assign a plausible disease diagnosis to each patient seen by a physician to determine his/her case-mix. We created ratios for each disease by using the total of ordered tests per physician, and used the 26 disease ratios along with physician attributes such as patient demographics, practice location, test results, test frequency per patient, etc. to generate the input vector for physician clustering. We standardized the data by centering (removing the mean) and scaling (dividing by standard deviation) the 26 diseases (ratios) to ensure that each disease contributes proportionately to determine similarity between two physicians. Next, we applied metric Multi-Dimensional Scaling (MDS), a non-linear dimension reduction approach, to reduce the inter-physician similarity from a 26-dimensional disease space to a 2-dimensional space. We used the Partitioning Around Medoids method with the Euclidian distance between physicians in the 2-dimensional MDS space to generate the physician clusters. To select the optimal number of physician clusters, we used the average silhouette width [23]. Based on the average silhouette widths, the best solution was K = 4 clusters (average silhouette width of 0.36). Figure 1 shows the silhouette widths of all physicians in the 4 clusters. Figure 2 shows the clusters of physicians on the 2-dimensional space. The physician clusters were annotated and validated by experts by comparing the inherent characteristics of physicians within a cluster. Since a physician’s case-mix can vary over time, we also generated period-sensitive physician clusters at a 2-year interval, thus allowing peer comparisons across a given period, and across the overall study period (of 6 years). Our PLUS system applies the clustering results to group physicians, based on their case-mix, for peer comparisons.
Fig. 1.

Silhouette widths for the 4 clusters.

Fig. 2.

Clustering of physicians in 4 clusters.

5.2 Association Rule Mining to Generate Test Order-Sets

We examine a physician’s order-sets to establish whether the high-volume of test order-sets justify clinical need or that the physician was overprescribing tests, maybe due to practice behaviour or lack of awareness of clinical guidelines. A frequent pattern refers to a set of items appearing as a pattern beyond a pre-specified frequency threshold. A frequent order-set illustrates a frequent pattern of tests ordered by a physician in a single test order. To analyze a physician’s test ordering pattern—i.e. which tests are ordered simultaneously and which tests are ordered for specific patient groups. To generate frequent order-sets, we used the constrained association rule mining method to generate n-order association rules (where n = 2–15 tests) based on order frequency and test relevance at the physician cluster. To account for temporal changes, we generated order-sets for 3-year periods (Tables 2 and 3 show the 14, 13, 12, 10, 7, 5, 3 order-sets).
Table 2.

Frequent order-sets at the regional level for the period of Jul-2011 to Jun-2014

Table 3.

Frequent order-sets at the regional level for the period of Jul-2014 to May-2017

When comparing the order-sets over the two 3-year time periods, we noted that in general the most frequent order-sets remain the same over time—out of the 14 order-sets (i.e. comprising 1–14 items), 7 order-sets remained the most frequent. This suggests that the order-sets are well-defined with fluctuations in their ordering frequency over time. The identified order-sets were implemented within the PLUS system as a benchmark for a physician to compare his/her order-sets with peers, where the unit of comparison is test order-set as opposed to individual test orders.

6 Knowledge-Based Analytics to Overcome Laboratory Underutilization by Physicians

Underutilization of laboratory—i.e. physicians not prescribing tests that are needed, or should be ordered as a follow-up to the earlier tests—leads to future increased laboratory utilization, delays in proper treatments and compromises patient safety.

To handle laboratory underutilization, we devised an evidence-based reflex testing strategy [24] that recommends (or “reflexes”) follow-up tests in response to results of prior tests (shown in Fig. 3). We use knowledge-based analytics, employing semantic web methods, to represent and execute reflex rules that suggest follow-up pathology tests to confirm a diagnosis (e.g. early diagnosis of pituitary dysfunction). Responding to abnormal test results noted for certain elements (such as abnormal patterns of basal pituitary hormones), our reflex testing strategy [24] firstly identifies additional reflective pathology tests, and then directly conducts the follow-up tests if the patient’s existing blood sample can be used, or recommends the tests to the patient’s physician.
Fig. 3.

Reflex strategy to identify abnormal lab result patterns and suggesting reflective testing.

To implement our reflective testing strategy, we developed an OWL ontology and a set of Description Logics (DL-safe) rules. We used the Protégé ontology engineering tool to construct the Reflex ontology and rules and utilized Hermit reasoner [25] for implementing the reasoning process. We explain reflex testing and its knowledge-based implementation using the example of diagnosing pituitary dysfunction. Based on pathology test results, as first step of our strategy we apply a set of context-sensitive ReflexRules to identify abnormal patterns in the pathology test results. For instance, the following rule “reflexes” when finding tests for women over 55 with a measurement of FSH (Follicle Stimulating Hormone) under 15:
$$ \begin{aligned} & Female\left( {?\,p} \right) \wedge age\left( {?\,p,?\,a} \right) \wedge ?\,a \ge 55 \wedge test\left( {?\,p, ?\,t} \right) \wedge hormone\left( {?\,t, FSH} \right) \wedge \\ & \quad \quad \quad outcome\left( {?\,t, ?\,fsh} \right) \wedge ?\,fsh < 15 \to reflexed\left( {?\,t, ReflexRule1} \right) \\ \end{aligned} $$
Note that, ReflexRule1 is associated with a set of exclusion rules—based on clinical information, such rules exclude special cases from consideration. Following our example, this rule excludes cases where the patient is pregnant, or on HCT or HRT (Hormone Contraception/Replacement Therapy):
$$ \begin{aligned} & Patient\left( {?\,p} \right) \wedge test\left( {?\,p,?\,t} \right) \wedge reflexed\left( {?\,t, ?\,rr} \right) \wedge hasExclusion\left( {?\,rr, ?\,excl} \right) \wedge \\ & \quad \quad \quad \left( {?\,excl = ExclRule31 \wedge isPregnant\left( {?\,p,true} \right)} \right) \vee \\ & \quad \quad \quad \left( {?\,excl = ExclRule32 \wedge followsTherapy\left( {?\,p, HRT} \right)} \right) \vee \\ & \quad \quad \quad \left( {?\,excl = ExclRule33 \wedge followsTherapy\left( {?\,p, HC} \right)} \right)) \to excluded\left( {?\,p,?\,excl} \right) \\ \end{aligned} $$
Once abnormal test patterns are identified and special cases are excluded, ReflexRule1 suggests a series of reflective tests to confirm pituitary dysfunction; e.g., measuring Thyroid Stimulating Hormone (TSH) and Free Thyroxine (FT4). Once results from these new tests are available, a set of FollowupRules, related to the initial ReflexRule1, check whether follow-up with an endocrinologist is recommended. For instance, the below rule recommends follow-up if TSH is non-raised and FT4 is low:
$$ \begin{aligned} & Patient\left( {?\,p} \right) \wedge test\left( {?p,?t} \right) \wedge reflexed\left( {?\,t, ?\,rr} \right) \wedge hasFollowup\left( {?\,rr, FollowupRule1} \right) \wedge \\ & reflexTest\left( {?\,p,?\,rt_{1} } \right) \wedge hormone\left( {?\,rt_{1} ,TSH} \right) \wedge outcome\left( {?\,rt_{1} ,?\,tsh} \right) \wedge NonRaised\left( {?\,tsh} \right) \\ & \wedge \,reflextTest\left( {?\,p,?\,rt_{2} } \right) \wedge hormone\left( {?\,rt_{2} ,FT4} \right) \wedge outcome\left( {?\,rt_{2} ,?\,ft4} \right) \wedge Low\left( {?\,ft4} \right) \\ & \to followup\left( {?\,p,FollowupRule1} \right) \\ \end{aligned} $$

As future work, we target to link our reflex testing strategy via PLUS with the LIS’s clinical pathway to recommend and conduct appropriate reflective pathology tests, based on the latest evidence and test results, to aid in accurate and early diagnosis.

7 Visualization of Laboratory Utilization: The PLUS System

PLUS implements a web-based health data analytics system using machine learning methods [26, 27] for laboratory utilization management (shown in Fig. 4).
Fig. 4.

PLUS functional architecture

Advance data visualization has been implemented for users to dynamically interact with the data analysis. PLUS provides a web-based (a) scorecard for physicians to understand their own test ordering profile over time and across different patient cohorts, and to compare their laboratory utilization (adjusted to case-mix) with similar peers. The scorecard presents physician’s laboratory utilization in terms of abnormal results, test volumes and frequency over time, peer comparisons, cost incurred, case-mix and the rate and cost of inappropriate test orders. The physician’s scorecard is private and cannot be viewed by practice auditing bodies; and (ii) dashboard for laboratory managers provides broad operational intelligence by aggregating the patient-level analytics to the regional level, displaying tests ordered, completion rates and flagged as inappropriate.

We present a working example of PLUS use by a physician. Figure 5 illustrates the opening page of a physician’s scorecard. The date selector (not shown) allows the physicians to select specific year(s) or quarter(s) within a year, following which the right-hand side visualizations are dynamically updated to show volume of tests ordered and the rate of abnormal and normal results. Physicians can hover over a visualization to get additional information. Figure 6 shows the physician’s scorecard with peer comparisons (on a yearly basis) across all tests, with options for filtering the order-tests.
Fig. 5.

Physician scorecard main page

Fig. 6.

Physician scorecard showing peer comparison across tests over a 6-year window

8 Concluding Remarks

As health care transitions from volume- to value-based care, there is an increasing need for efficient and effective laboratory utilization by physicians. In this paper, we presented an innovative and sustainable laboratory utilization management approach, targeting physicians, that leverages (a) data analytics methods to develop and understand each physician’s test ordering profiles; and (b) data visualization techniques to display the physician’s test ordering pattern as an interactive scorecard so that they can self-audit and -regulate their test ordering behaviour. We posit that given healthcare budgetary pressures and increasing test volumes, our sustainable, data-informed and physician-engaged approach will help to minimize inappropriate laboratory utilization, improve sustainability of the laboratory operations, and achieve value-based care. PLUS is being implemented to optimize test utilization at the Central Zone pathology laboratory in Nova Scotia. As literature estimates that a minimum of 25% of tests are inappropriately ordered [1, 2, 3], we believe that the utilization of Central Zone laboratories (in Halifax) can potentially be reduced by 2 million tests annually from the 8 million currently performed, leading to huge cost savings and improved patient safety.

Notes

Acknowledgements

We thank the NSHA Central Zone pathology lab for supporting the project, and Nova Scotia Health Research Foundation for giving the catalyst grant.

References

  1. 1.
    Freedman, D.B.: Towards better test utilization - strategies to improve physician ordering and their impact on patient outcomes. EJIFCC 26, 15–30 (2015)Google Scholar
  2. 2.
    Misra, S., Barth, J.H.: Guidelines are written, but are they followed? Ann. Int. J. Biochem. Lab. Med. 50, 400–402 (2013)CrossRefGoogle Scholar
  3. 3.
    van Walraven, C., Goel, V., Austin, P.: Why are investigations not recommended by practice guidelines ordered at the periodic health examination? J. Eval. Clin. Pract. 6(2), 215–224 (2000)CrossRefGoogle Scholar
  4. 4.
    Zhi, M., Ding, E.L., Theisen-Toupal, J., Whelan, J., Arnaout, R.: The landscape of inappropriate laboratory testing: a 15-year meta-analysis. PLoS ONE 8, e78962 (2013)CrossRefGoogle Scholar
  5. 5.
    Smellie, W.S.: Appropriateness of test use in pathology: a new era or reinventing the wheel? Ann. Clin. Biochem. 40, 585–592 (2003)CrossRefGoogle Scholar
  6. 6.
    Salinas, M., Lopez-Garrigós, M., et al.: Laboratory false-positive results: a clinician responsibility or a shared responsibility with requesting clinicians? Clin. Chem. Lab. Med. 51, e199–e200 (2013)CrossRefGoogle Scholar
  7. 7.
  8. 8.
    Vidyarthi, A.R., Hamill, T., Green, A.L., Rosenbluth, G., Baron, R.B.: Changing resident test ordering behavior: a multilevel intervention to decrease laboratory utilization at an academic medical center. Am. J. Med. Qual. 30(1), 81–87 (2015)CrossRefGoogle Scholar
  9. 9.
    Fryer, A.A., Smellie, W.S.A.: Managing demand for laboratory tests: a laboratory toolkit. J. Clin. Pathol. 66, 62–72 (2013)CrossRefGoogle Scholar
  10. 10.
    McDonald, E.G., Saleh, R.R., Lee, T.C.: Mindfulness-based laboratory reduction: reducing utilization through trainee-led daily ‘Time Outs’. Am. J. Med. 130, e241–e244 (2017)CrossRefGoogle Scholar
  11. 11.
    Neilson, E.G., Johnson, K.B., Rosenbloom, S.T., et al.: Resource utilization committee: the impact of peer management on test-ordering behavior. Ann. Intern. Med. 141, 196–204 (2004)CrossRefGoogle Scholar
  12. 12.
    Iturrate, E., Jubelt, L., Volpicelli, F., et al.: Optimize your electronic medical record to increase value: reducing laboratory overutilization. Am. J. Med. 129, 215–220 (2016)CrossRefGoogle Scholar
  13. 13.
    Ryskina, K., et al.: Effect of social comparison feedback on laboratory test ordering for hospitalized patients: a randomized controlled trial. J. Gen. Intern. Med. 33, 1639–1645 (2018)CrossRefGoogle Scholar
  14. 14.
    Bunting, P.S., van Walraven, C.: Effect of a controlled feedback intervention on laboratory test ordering by community physicians. Clin. Chem. 50, 321–326 (2004)CrossRefGoogle Scholar
  15. 15.
    Iams, W., Heck, J., Kapp, M., et al.: A multidisciplinary housestaff-led initiative to safely reduce daily laboratory testing. Acad. Med. 91, 813–820 (2016)CrossRefGoogle Scholar
  16. 16.
    Srivastava, R., Bartlett, W.A., Kennedy, I.M., et al.: Reflex and reflective testing: efficiency and effectiveness of adding on laboratory tests. Ann. Clin. Biochem. 47, 223–227 (2010)CrossRefGoogle Scholar
  17. 17.
    Dickerson, J.A., et al.: Improving the value of costly genetic reference laboratory testing with active utilization management. Arch. Pathol. Lab. Med. 138(1), 110–113 (2014)CrossRefGoogle Scholar
  18. 18.
    Levick, D.L., Stern, G., et al.: Reducing unnecessary testing in a CPOE system through implementation of a targeted CDS intervention. BMC Med. Inform. Decis. Mak. 13, 43 (2013)CrossRefGoogle Scholar
  19. 19.
    Vegting, I.L., van Beneden, M., Kramer, M.H., et al.: How to save costs by reducing unnecessary testing: lean thinking in clinical practice. Eur J Intern Med. 23(1), 70–75 (2012)CrossRefGoogle Scholar
  20. 20.
    Elnenaei, M.O., Campbell, S.G., Thoni, A.J., Lou, A., Crocker, B.D., Nassar, B.A.: An effective utilization management strategy by dual approach of influencing physician ordering and gate keeping. Clin. Biochem. 49, 208–212 (2016)CrossRefGoogle Scholar
  21. 21.
    Huang, Y., et al.: Improving serological test ordering patterns for the diagnosis of celiac disease through clinical laboratory audit of practice. Clin. Biochem. 45(6), 455–459 (2012)CrossRefGoogle Scholar
  22. 22.
    Yeh, D.D.: A clinician’s perspective on laboratory utilization management. Clin. Chim. Acta 427, 145–150 (2014)CrossRefGoogle Scholar
  23. 23.
    Rousseeuw, P.: Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J. Comput. Appl. Math. 20, 53–65 (1987)CrossRefGoogle Scholar
  24. 24.
    Elnenaei, M., Minney, D., Clarke, D.B., Kumar-Misir, A., Imran, S.A.: Reflex and reflective testing strategies for early detection of pituitary dysfunction. Clin. Biochem. 54, 78–84 (2018)CrossRefGoogle Scholar
  25. 25.
    Glimm, B., Horrocks, I., et al.: An OWL 2 reasoner. J. Autom. Reason. 1, 1–25 (2014)zbMATHGoogle Scholar
  26. 26.
    Abusharekh, A., Stewart, S., Abidi, S.S.R.: H-DRIVE: a big data analytics platform for evidence-informed decision making. In: 4th IEEE International Congress on Big Data, New York (2015)Google Scholar
  27. 27.
    Cha, S., Abusharekh, A., Abidi, S.S.R.: Towards a ‘big’ health data analytics platform. In: 1st IEEE International Conference on Big Data Computing Service and Applications, San Francisco (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Syed Sibte Raza Abidi
    • 1
    Email author
  • Jaber Rad
    • 1
  • Ashraf Abusharekh
    • 1
  • Patrice C. Roy
    • 1
  • William Van Woensel
    • 1
  • Samina R. Abidi
    • 1
    • 2
  • Calvino Cheng
    • 3
  • Bryan Crocker
    • 3
  • Manal Elnenaei
    • 3
  1. 1.NICHE Research Group, Faculty of Computer ScienceDalhousie UniversityHalifaxCanada
  2. 2.Medical Informatics, Department of Community Health and EpidemiologyDalhousie UniversityHalifaxCanada
  3. 3.Department of Pathology and Laboratory MedicineNova Scotia Health AuthorityHalifaxCanada

Personalised recommendations