Introduction

Clinical utility for a molecular diagnostic test refers to a demonstration of its usefulness within a clinical pathway (i.e. the test’s ability to influence patient care management), and includes the test’s impact on patient well-being, avoidance of unnecessary or detrimental treatment, and mitigating collateral adverse events (e.g. effects of a treatment or invasive procedure on a patient that could have been avoided, and the additional costs incurred thereof) [1–3]. This can also be extended to the impact of the investigation on the patient, as determined by the level of invasiveness.

As an example, blood in the urine, or hematuria, is a common finding in clinical practice which is most commonly associated with benign causes [4]. However, hematuria can be a symptom of urothelial carcinoma (UC), and as the degree of hematuria is not correlated with disease stage or grade [5–7], the American Urological Association (AUA) and American College of Physicians recommend that all patients with asymptomatic microscopic hematuria (AMH) undergo a full urological work-up, including invasive procedures, such as cystoscopy and contrast computed tomography (CT) scans [8, 9].

As non-invasive molecular diagnostic tests for UC have yet to clearly demonstrate utility in changing clinical practice and improving outcomes for patients with hematuria, concern has been expressed that results derived from such tests may encourage the unnecessary use of invasive and costly diagnostic procedures to either confirm or rule out UC in the absence of a clear benign cause of hematuria [8–11]. Performing invasive procedures also increases the risk of a patient experiencing an adverse event during the diagnostic process [8, 12–17].

Taking these factors into consideration, a framework was developed to assess the clinical utility of Cxbladder, a non-invasive test performed on a single unfractionated urine sample, using a concurrent clinical decision-making approach. Using a single voided urine sample, Cxbladder measures the expression of five genomic markers (IGF, HOXA, MDK, CDC and IL8R gene expression) in its Detect modality, while data from these genomic markers and clinical variables (age, gender, frequency of macrohematuria and smoking history) are considered in its Triage clinical modality. Cxbladder is used to identify patients who have a low risk of UC and may not require a full urological work-up (Triage clinical modality), and patients with a high probability of UC (Detect clinical modality) [18, 19]. Cxbladder has been validated for ruling out UC in 40% of patients with macrohematuria when used in the Triage modality (sensitivity 0.95; negative predictive value 0.98) [18], and has proven effective at separating samples from healthy patients and those with UC in the Detect modality (specificity 0.85; sensitivity 0.82) [19, 20].

Using a decision-making approach, urologists in the present study were asked to make clinical decisions in the context of referral data from real-life cases of patients presenting with hematuria. These physicians were then provided with new diagnostics data from Cxbladder tests and given an opportunity to change their clinical decisions, without any risk of detriment to patient care or clinical outcome.

The aim of this study was to investigate the clinical utility of Cxbladder when used in the urological evaluation of patients presenting with hematuria and no obvious benign cause by assessing the change in clinical decision-making in the context of results from Cxbladder in the Triage and Detect modalities. Specifically, the study objective was to identify and quantify changes in urologist behavior once information from Cxbladder testing was added to the primary clinical case data from patients with microhematuria and the possibility of UC, who underwent invasive diagnostic procedures as part of a real-world full urological work-up. It was expected that disclosing information from Cxbladder in the Triage and Triage and Detect modalities would reduce the number and/or extent of the use of total investigative procedures, including invasive procedures, requested by participating urologists.

Methods

Study Design

A sample population of urologists (participant physicians) with experience in the use and application of Cxbladder in their clinical settings were individually informed of the clinical purpose of Cxbladder in both the Triage and Detect modalities. Cases were individually evaluated in the same sequence under the supervision of a study coordinator. Participant physicians were asked to review and recommend diagnostic procedures for each case based on the patient’s evaluation from normal referral data.

Requested diagnostic tests and procedures were recorded following a standardized protocol, demonstrated by the coordinators prior to beginning the study. Participant physicians were required to provide a ‘yes’ or ‘no’ answer as to whether additional urological investigations were necessary following review of the referral data on each case in the primary evaluation. For patients designated ‘yes’, tests and procedures could be chosen from a list of investigations recommended in the AUA guidelines for a urological work-up for a patient with hematuria. Participant physicians were also offered the option of requesting additional tests or procedures not listed in the standard protocol and offering further comments.

For the purposes of this study, participant physicians were instructed to consider all tests and procedures to be fully funded, with no additional cost incurred by the patient, to remove any decision-making bias related to the reimbursement status of any tests or procedures. The participant physicians were able to ask questions of the study coordinator regarding each case, but no additional clinical background was provided beyond the case report form. Participant physicians were not informed of the final UC status of each case, nor were they informed that all cases had been previously selected for a urological work-up by a physician in one of the participating physicians’ clinical practices.

Participant physicians were then asked to repeat their recommended clinical evaluation for each case in the context of the information derived from Cxbladder testing, firstly in the Triage modality alone, and secondly when results from both the Triage and Detect modalities were provided. Cxbladder results were presented in a format that was consistent with the commercially available Cxbladder tests, including a report outcome, explanation of the Cxbladder test methodology [18, 19] and guide to interpreting the results. Outcomes were defined as ‘negative’ or ‘physician-directed protocol’ in the Triage modality; and ‘normal’, ‘elevated’ or ‘high’ in the Detect modality. Participant physicians were required to answer ‘yes’ or ‘no’ as to whether their clinical work-up and management plan changed in light of each set of Cxbladder test results.

Case Selection

To provide a fully representative cross-section of patient demographics, cases were systematically selected from the database of patients enrolled in previous prospective clinical studies of Cxbladder who met the AUA definition of microhematuria with the possibility of UC requiring a urological work-up, including invasive diagnostic procedures [4]. All cases are patients who had consented to the anonymous use of their urine sample and clinical information. Each anonymous case was assigned a random case number after inclusion. Of the 40 available cases of patients with microhematuria undergoing investigation for UC, 33 patient cases met the study inclusion criteria (see Appendix 1). Seven cases did not have sufficient clinical information in their case report form to fully support a decision on a recommended clinical evaluation. Overall, the study generated data from 792 decision nodes from 396 participant physician–patient case interactions.

Compliance with Ethics Guidelines

All cases selected for use in this study were collected from patients who had provided informed consent prior to being enrolled in clinical studies of Cxbladder. All data were collected in accordance with all relevant ethical guidelines. This article does not contain any new studies with human or animal subjects performed by any of the authors.

Study Participant Physicians

Twelve urologists from the USA (n = 5), New Zealand (n = 4), Australia (n = 2) and Singapore (n = 1) were invited to participate in the study (see Appendix 2). Six participants practice in public hospitals, four are in private practice and two in tertiary referral centers and 6 are based in public hospitals. Participant physicians had prior experience in the use and application of Cxbladder in real-world situations, either by participating in clinical studies of Cxbladder or as part of clinical diagnostic investigations for real-world patients presenting with hematuria. All participant physicians were offered honoraria to compensate for their time.

Study Endpoints

The co-primary endpoints were the changes in both the total number of diagnostic procedures used over all patient cases and the number of invasive procedures requested following disclosure of diagnostic information from Cxbladder in the Triage and Triage and Detect modalities. The co-primary endpoints were dependent on a base assumption that all participant physicians would recommend invasive diagnostic procedures for all patients confirming the validity of the sample of participating urologists. It was expected that all selected patients should receive at least one invasive procedure, in accordance with the AUA guidelines and diagnostic procedures performed in the real world. The relative proportion of patients not having invasive diagnostic procedures requested by US-based urologists versus non-US-based urologists was reviewed to investigate whether there were any differences in diagnostic choice as a result of region-specific practices.

Secondary endpoints included (1) the proportion of patients with nil invasive procedures requested following disclosure of diagnostic information from Cxbladder tests in the Triage modality and Triage and Detect modalities after at least one invasive diagnostic procedure was requested following primary evaluation, and (2) the total number of procedures requested following disclosure of diagnostic information from Cxbladder tests in the Triage modality and Triage and Detect modalities.

Additional exploratory endpoints were added when it became apparent that not all participant physicians would recommend invasive diagnostic procedures at primary evaluation, including (1) the total number of invasive diagnostic procedures either requested or removed per participant physician, and (2) the overall proportion of patients with at least one invasive diagnostic test requested following disclosure of diagnostic information from Cxbladder in the Triage and Detect modalities after non-invasive or no further diagnostic tests were requested following primary evaluation.

Cystoscopy (flexible or rigid) and CT scans (contrast or non-contrast) were defined as invasive diagnostic procedures. Ultrasound, urine cytology and UroVysion® FISH were defined as non-invasive procedures and tests.

Results

Preliminary Evaluation of the Sample Population Using Referral Data

In the context of initial evaluation data provided following referral, at least one invasive diagnostic test was requested in 259/396 (65.4%) interactions. The proportion of cases in which at least one invasive diagnostic procedure was requested ranged from 15 to 97% of patient cases for individual participant physicians (see Appendix 3 for further information). No differentiating pattern was observed between US-based urologists and those from the rest of the world, indicating no regional bias in the selection of invasive diagnostic procedures following primary evaluation, and enabling the inclusion of all participating physicians in the study.

A total of 689 diagnostic procedures were requested following primary evaluations across the set of 396 participant physician–patient case interactions. At least one additional diagnostic test or procedure was requested for 24–100% of patient cases evaluated. Of these procedures, 425 were classified as invasive, with individual participant physicians applying a range of 5–67 invasive procedures across all patient cases.

Change in the Number of Invasive Diagnostic Procedures Requested with Disclosure of Information from Cxbladder Tests

The total number of invasive diagnostic procedures requested was reduced to 379 following disclosure of information from Cxbladder in the Triage modality alone, representing an 11% reduction compared with primary evaluation of the referral data (Fig. 1). The use of flexible cystoscopy and CT scans was reduced by 19% and 5%, respectively, while an increase in the use rigid cystoscopy was reported in patients with data available from Cxbladder in the Triage modality (Fig. 1).

Fig. 1
figure 1

Overall number of invasive diagnostic procedures requested by participant physicians across 396 participant physician–case interactions after primary evaluation and following disclosure of Cxbladder results in the Triage modality and in the Triage and Detect modalities

Full size image

Following the disclosure of information from Cxbladder in the Triage and Detect modalities, 292 invasive procedures were requested. This represented a 31% reduction compared with primary evaluation of the referral data. Specifically, the use of flexible cystoscopy and CT scans was reduced by 44% and 20%, respectively, and while the increased intention to use a rigid cystoscopy following disclosure of results from Cxbladder in the Triage modality was maintained, fewer requests were made once data from Cxbladder in the Detect modality was also disclosed.

All participant physicians displayed a change in their diagnostic behavior, either increasing or decreasing the number of invasive procedures in at least one case (Fig. 2). Overall, invasive diagnostic tests were eliminated in 82 (31.7%) interactions following the disclosure of additional diagnostic information provided to the physicians from Cxbladder in the Triage and Detect modalities (Fig. 2).

Fig. 2
figure 2

Change in requests for invasive diagnostic procedures by individual participants after disclosure of Cxbladder results in both the Triage and Detect modalities compared with requested invasive diagnostic procedures following primary evaluation

Full size image

Change in the Number of Total Diagnostic Procedures Requested with Disclosure of Information from Cxbladder Tests

In several cases, participant physicians continued to use a similar work-up, but altered their choice of procedure within that category following disclosure of information provided by Cxbladder. The total number of diagnostic procedures was reduced to 657 following disclosure of Cxbladder results in the Triage modality, representing a 5% reduction compared with primary evaluation using referral data (Fig. 1). Following subsequent disclosure of Cxbladder results in the Detect modality in addition to the Triage modality, the total number of diagnostic procedures requested was reduced to 514, representing a 25% reduction compared with primary evaluation. In contrast, the use of ultrasound was increased following Cxbladder in the Triage modality (21%) and in the Triage and Detect modalities (12%). A clinically relevant reduction in the use of urine cytology was observed (37%) (Fig. 1).

Change in the Number of Non-Invasive Diagnostic Tests Requested with Disclosure of Information from Cxbladder Tests

No invasive diagnostic procedures were requested for 137/396 (34.5%) patient-by-physician interactions in the context of primary evaluation data (see Appendix 3 for further information). Invasive diagnostic procedures were requested following disclosure of information from Cxbladder in the Triage and Detect modalities for 21 (15.3%) of these interactions.

Selection of Invasive Diagnostic Procedures for Patients with UC

The use of invasive diagnostic procedures that aid in the detection of UC was more prevalent amongst patients ultimately diagnosed with UC following the addition of information from Cxbladder tests. Following the primary evaluation, at least one invasive diagnostic procedure was selected for 16/24 (67%) case interactions involving patients who were ultimately diagnosed with UC. Following disclosure of information from Cxbladder in the Triage modality and in the Triage and Detect modalities, the proportion of invasive diagnostic procedures requested increased to 24/24 (100%) case interactions for patients ultimately diagnosed with UC.

Discussion

This is the first study to demonstrate the clinical utility of a molecular diagnostic, non-invasive urine test in improving the management of patients with asymptomatic hematuria by focusing the diagnostic process on performing appropriate procedures to improve clinical resolution. The use of Cxbladder reduced the total number of procedures requested, including a reduction in the number of invasive procedures, resulting in improved outcomes for patients with microhematuria referred to a urologist for a urological work-up. The addition of Cxbladder data changed clinical procedures allocated to patients and enabled urologists to modify their diagnostic procedures, resulting in the more appropriate use of invasive procedures, particularly in all patients who had UC.

The clinical utility of diagnostic tests for investigating the possibility of UC in patients with hematuria is perhaps best defined by Schmitz-Dräger and colleagues [11] in the World Health Organization/International Consultation on Urologic Diseases consensus manuscript as providing “additional information that is helpful to the clinician for the management of the disease”. A molecular diagnostic test should also “be able to make the information available in an efficient and timely manner” [11], which is enabled by Cxbladder using a single urine sample to evaluate patients with hematuria. All urologists (100%) participating in this study made clinically relevant changes to their diagnostic algorithm following the addition of Cxbladder data used in both the Triage and Detect modalities. The study also identified wide physician variation in the clinical diagnostic algorithm between physicians. The level of this variation was consistent between urologists and between countries, yet Cxbladder facilitated all patients ultimately diagnosed with UC being identified and referred for appropriate invasive diagnostic procedures.

Until now, the clinical utility of molecular diagnostic tests used in patients with hematuria has been unclear, as early non-invasive urine tests, currently available in the market, lack the clinical resolution necessary for inclusion in diagnostic guidelines [11]. Hematuria may be caused by a number of diseases and conditions, resulting in most patients being worked-up for the possibility of UC in the absence of a non-invasive urine test offering sufficient clinical resolution to effectively triage out patients with possible UC [11]. The outcome of this study challenges the assertion that urologists should not pursue urine-based molecular markers for UC detection in the initial evaluation of hematuria [9].

Cxbladder is effective in ruling out patients who have a low probability of UC in its Triage modality [18]. Furthermore, its Detect modality can also be applied to the same urine sample to assess the risk of UC, and to segregate patients accordingly [19]. The present study demonstrates that Cxbladder overcomes concerns that the use of molecular diagnostics in patients with hematuria can lead to the detection of more cases of UC, but only at the cost of unnecessary invasive investigations [10]. Using Cxbladder in the Triage and Detect modalities in tandem may have utility in decreasing the clinical burden on urologists and healthcare systems by reducing the number of total procedures, including invasive procedures, performed, while assisting with the accurate detection of UC. Patients could also expect fewer adverse events and improved quality of life by avoiding invasive procedures [8, 12–17].

This study also represents a population-level analysis of the impact of Cxbladder on treatment decisions amongst patients with microhematuria who are expected to have a low prevalence of UC [21–24]. As demonstrated in this study, when patients with the same characteristics present to individual urologists, a wide variation in diagnostic decisions may be made. While demographic factors may influence the decision on how to work-up a patient presenting with microhematuria, Cxbladder in the Triage modality combines key risk factors, i.e. gender, age and smoking history, with genotypic data to support these decisions [18]. An earlier study has already demonstrated how phenotypic risk factors are accounted for in Cxbladder in the Triage modality, with a higher proportion of patients with microhematuria testing negative compared with patients with macrohematuria [18]. Therefore, as evidenced in this study by the differences in work-ups chosen for patients ultimately diagnosed with UC, additional information derived from Cxbladder testing may not only lead to a more favorable outcome for the individual, but also to an overall improvement in outcomes across a population of urologists and patients, especially when both tests are used in tandem. In particular, in this study, Cxbladder provided greater clinical resolution, allowing physicians to define a final clinical diagnostic algorithm to best manage each patient. This results in a more consistent approach to the use of invasive diagnostic procedures, with urologists opting to either increase or decrease their use appropriately based on a patient’s risk and disease prevalence (see Appendix 3).

The investigators considered the limitations associated with the study design used here and believed it to be the most pragmatic method of evaluating the clinical utility of Cxbladder, given the logistical, consistency and ethical challenges in prospectively investigating the selection of diagnostic tests in patients with microhematuria by urologists in several countries. The results of this study provide data justifying a prospective investigation of the clinical utility of Cxbladder as a clinical intervention in the diagnostic process. While the present study is a retrospective analysis of a prospectively recruited cohort of real-world patients presenting for a urological work-up, including invasive procedures, following a primary diagnosis of AMH without a clear benign cause, this format has been effective in supporting the hypothesis that Cxbladder used in the Triage or Triage and Detect modalities may improve the standard of care for patients presenting with hematuria. By using a consistent patient set across all participant physicians, the investigators were able to limit the variance between patients and focus on the variance between physicians and their diagnostic algorithms. The participant physicians evaluated real-world patients who had been systematically selected for representation with real-world referral data, on which to base their decisions. Finally, all participant urologists assessed the case studies in the same sequence, which may increase the risk of bias if earlier diagnostic decision-making was found to affect decisions made in later cases. However, any bias would be consistent across all participant physicians.

Conclusions

Urologists make compelling changes to their clinical decision-making behavior for patients presenting with hematuria when presented with Cxbladder results. The resultant increase in clinical utility associated with Cxbladder is characterized by the more appropriate allocation of invasive diagnostic procedures, resulting in a reduction in both the total number of diagnostic procedures and the number of invasive procedures used, improving the diagnostic experience for patients.