Introduction

Comprehensive sequencing initiatives are shifting our understanding of which genetic factors drive cancer growth and response to treatment. The aim of these programs is to generate a breadth of genomic data to facilitate research and discovery (Genomics England 2017; Investigators 2019; Stark et al. 2019). While designed to benefit future patients, these initiatives can provide immediate impact for those undergoing sequencing and their families. Sequencing results may indicate likely response, resistance, or toxicity to a targeted treatment, leading to a change in clinical management (Laskin et al. 2015; Wong et al. 2020). Results may also reveal germline findings that enable targeted cancer risk reduction in susceptible family members. Analyses of non-randomized trial data signal the potential for genomic sequencing to improve clinical outcomes through altered management (Zimmer et al. 2019; Tuxen et al. 2019; Bertucci et al. 2021). Owing to limited scope and duration of existing study endpoints and infrequent inclusion of counterfactuals, the downstream effects of comprehensive genomic sequencing for research participants and healthcare systems are poorly understood (Schwarze et al. 2018; Stark et al. 2019).

When considering resource allocation for emerging genomic technologies, healthcare decision-makers require evidence of the expected upfront costs and downstream consequences of sequencing (Barna et al. 2018; Faulkner et al. 2012). Clinical uptake of sequence data, resultant patient health, and downstream healthcare expenditures will collectively affect systems’ readiness and willingness for adoption (Vellekoop et al. 2021). Real-world administrative data presents an opportunity to quantify the impacts of genomic sequence data on patient-level and health system outcomes. The systematic capture of healthcare utilization can identify when cancer management diverges from standard care as a result of sequencing, and support evaluation of subsequent cost and health effects. Detailed histology information recorded by cancer registries can enable stratified analyses across patient subgroups likely to experience differential effectiveness and costs. Studies are beginning to draw on real-world data to evaluate the survival impacts and cost-effectiveness of multi-gene panels for treatment assignment (Presley et al. 2018; Steuten et al. 2018). None have evaluated the real-world economic impacts of research-based whole genome sequencing or whole transcriptome analysis.

To begin building outcomes and cost evidence for research-based comprehensive genomic sequencing, we draw on real-world data and describe the early-stage cost consequences of a Canadian precision oncology research initiative, the British Columbia (BC) Personalized OncoGenomics (POG) Program (NCT02155621). This single-arm study applies whole genome and transcriptome analysis (WGTA) to characterize and interpret genomic landscapes in advanced cancers (Laskin et al. 2015; Pleasance et al. 2020). Generated sequence data may also inform clinical decision-making and treatment for participating patients, which we hypothesize will affect individual health outcomes and health system resource utilization.

Our study uses BC population-based administrative data combined with quasi-experimental matching to identify a usual care counterfactual for POG’s single-arm application of WGTA (Weymann et al. 2020). We apply cost consequence analysis using weighted regression analysis and non-parametric bootstrapping to estimate the downstream effects of WGTA versus usual care on patient management, survival, and health system costs in the year following enrollment. To understand differential impacts on costs and benefits across patient subgroups, we undertake stratified analysis. Scenario analysis drawing on observed heterogeneity identifies potentially modifiable or avoidable factors likely to drive future cost consequences of WGTA.

In contrast to cost-effectiveness analysis, which seeks to inform reimbursement decisions and summarizes comparisons across technologies using a single metric, cost consequence analysis produces disaggregated estimates of patient and health system outcomes for stakeholders to simultaneously consider (Drummond et al. 2015; Mauskopf et al. 1998). Generated evidence highlights the real-world implications of genomic research participation, accounting for health system dynamics. These dynamics are critical for healthcare decision-makers to understand when considering implementation and for informing early market assessments (IJzerman et al. 2017; Van Harten and Retèl 2016). Disaggregated estimates can also be used by stakeholders for determining research investment, by clinical researchers in refining indications for future trial designs, and by patients and publics in setting their expectations for precision medicine.

Materials and methods

Study design

Our retrospective cohort comprised adult patients with cancers of varying primary sites who participated in the second phase of the POG Program between July 2014 and December 2015 and genetic algorithm-matched usual care controls (Weymann et al. 2020). POG inclusion criteria are (i) metastatic disease considered incurable by the oncologist, (ii) good performance status, and (iii) life expectancy greater than 6 months (NCT02155621). We excluded 60 patients whose oncologist consented them to POG but who either withdrew, were lost to follow-up, or were classified as ineligible by POG Program coordinators verifying fulfillment of enrolment criteria (21% of patients). Usual care controls were eligible for matching after receiving systemic therapy treatment for advanced stage disease, as indicated by BC Cancer systemic therapy protocol codes (British Columbia Cancer 2018. Chemotherapy Protocols). To ensure sufficient overlap across groups, we excluded patients diagnosed with stage 0, benign, non-invasive, or in situ cancers, and patients whose performance status indicated they were completely disabled at diagnosis (0.01% of patients). After matching, we conducted a cost consequence analysis. Primary endpoints included 1-year survival and healthcare costs measured from a healthcare payer perspective.

Data sources and derived variables

We based our analysis on de-identified linked administrative datasets provided by Population Data BC and the BC Cancer Registry (BC Cancer & BC Ministry of Health n.d.). Data sources, presented in Fig. 1, were selected to support matching analysis, measure patient survival, and enable cost estimation from the healthcare payer perspective. We measure all healthcare expenditures incurred by British Columbian taxpayers, including hospitalizations, physician services, cancer care, and outpatient prescription drugs (Allin 2008). To determine our eligible study cohort and inform matching, we obtained data for adult patients diagnosed with cancer in BC, identifying eligible control patients from the BC Cancer Registry, a population-based provincial cancer registry that records demographic, disease, and mortality information for all cancer diagnoses in BC. We identified POG patients from the BC Cancer Outcomes and Surveillance Integration System (OaSIS) POG Module Database.

Fig. 1
figure 1

Summary of data sources

Full size image

Mean costs of applying POG-initiated WGTA and mean costs of each component of WGTA were estimated in a previous study using data from Canada’s Michael Smith Genome Sciences Centre at BC Cancer (Weymann et al. 2017). To estimate WGTA costs in each month of the study period, we drew on a system of equations model that determined whole genome and transcriptome sequencing, bioinformatics, and other WGTA costs over time (reported in the Supplemental Material). This model specified a different regression equation for estimating the pattern that each cost component follows over time. Total WGTA costs were equal to the sum of the monthly cost component estimates. We assumed that WGTA-related costs incurred by POG patients who experienced biopsy failure, for which pathology assessed tumor content was below 40%, incurred only costs of initial biopsy and sample processing.

To assess treatment history and estimate systemic therapy costs, we obtained prescription records for drugs dispensed by BC Cancer pharmacies. In BC, all approved systemic therapy treatments administered in regional cancer centers, community hospitals, or taken at home are dispensed by BC Cancer pharmacies. Records capture systemic therapy type, date of administration, BC Cancer protocol code, and billed drug costs for publicly funded systemic therapy drugs. To determine the number of lines of systemic therapy received by each patient prior to the index date, we applied an automated algorithm validated for use in administrative prescription drug data (Weymann et al. 2019). This algorithm classified lines of therapy on the basis of repeating drug dispensation patterns. Algorithm development considered a range of parameter values for the following three variables when classifying new lines of therapy: repeating drug names (1–20), repeating protocol codes (1–20), and the number of days elapsed between prescriptions (1–120). Optimal parameter values were determined based on estimated mean squared error, correlation coefficients, and non-parametric hypothesis test statistics. In this study, we employed the best performing algorithm which classified new lines of therapy when new drug names and/or protocol codes appeared more than 38 days after a patient’s previous dispensation.

We used the BC Cancer Radiation and Brachytherapy database and Cancer Agency Information System (CAIS) to identify radiotherapy treatments (including the total number of fractions), medical appointments, and tests received by the cohort. Records contained the date of the service provided and codes generally describing the type, nature, and location of the service. Given that BC Cancer care services are not delivered on a fee-for-service basis, we obtained corresponding unit costs from published fees for similar services, literature sources, and/or expert opinion (detailed in the Supplemental Material) (BC Medical Services Commission 2017).

To measure broader provincial healthcare system utilization, we drew on hospital services data from the Discharge Abstract Database, which tracks inpatient admissions and discharges from all hospitals in BC. We determined hospitalization cost by multiplying the resource intensity weight of each hospital stay by the BC government’s estimated cost per standard hospital stay (formerly, cost per weighted case). We obtained Medical Services Plan data, which recorded information on provider type, service type, and cost of service for all fee-for-service physician visits in BC. Information on outpatient prescription drug purchases reimbursed by the provincial public drug program came from BC PharmaNet, a province-wide system that captures all prescription drugs dispensed outside of BC’s acute care hospitals.

Matching analysis

Our detailed matching methods were described previously (Weymann et al. 2020). Briefly, we used genetic algorithm-based matching to match cases (POG patients) and controls (usual care patients) at the index date, date of POG biopsy (Diamond and Sekhon 2013). This method matches patients on a generalized weighted Mahalanobis distance metric rather than a difference in parametrically estimated propensity scores. We 1:1 matched cases and controls based on baseline covariates and a propensity score estimated using logistic regression. The inclusion of a propensity score is not necessary during genetic matching, but it is recommended to provide reasonable starting values for the genetic evolutionary search algorithm (Diamond and Sekhon 2013). When matching, we allowed for ties and replacement.

Biopsy date indicates the beginning of WGTA and, similar to past studies comparing groups who did not experience the same index date (de Oliveira et al. 2016; Jarosek et al. 2015; Walker et al. 2013), was randomly assigned for each control based on (1) the distribution of time between first treatment for advanced stage disease and POG biopsy for POG patients who had received advanced systemic therapy (n = 152, 66%) and (2) the distribution of month and year of biopsies for POG patients who had not yet received advanced systemic therapy (n = 78, 34%). Control patients had to be alive, diagnosed with cancer, and treated for advanced stage disease prior to their assigned pseudo-biopsy date to be eligible for the study (Weymann et al. 2020). To explore the sensitivity of our survival results to our method of index date assignment, we used multiple imputation involving predictive mean matching, as detailed in the Supplemental Material. This approach introduced additional uncertainty, as indicated by higher standard errors around point estimates, but we found no substantive changes in estimated overall survival probabilities, mean survival times, or statistical hypothesis test results. Our conclusions across POG and usual care patients remained unchanged. To calculate propensity scores, we used logistic regression models estimating each patient’s probability of participating in POG, as reported in a prior study (Weymann et al. 2020). Baseline covariates included patient demographics, clinical characteristics, and treatment histories at the index date.

Baseline covariates affecting the probability of enrolling in POG and subsequent outcomes are likely to vary across tumor subgroups. To further account for this variation beyond inclusion of interaction terms, we stratified our matching analysis for patients diagnosed with breast cancer and those diagnosed with other cancers owing to known differences in disease trajectories, availability of treatments, and observed sample sizes. Among other cancers, we considered interaction terms between primary cancer site and baseline covariates to allow for heterogeneity. We matched breast cancer patients based on the index date (biopsy date), age, rurality, year of diagnosis, grade at initial diagnosis, cancer stage at initial diagnosis, and number of lines of systemic therapy treatment received prior to index date. Matches for patients with other cancers also considered sex, primary cancer site, and performance status at initial diagnosis. Matching covariates were selected based on data availability as well as known and hypothesized correlations with the probability of enrolling in POG, healthcare costs, and survival. To account for the potentially informative nature of missing data, our matching analysis considered missing categories alongside measured covariates.

We selected matches to maximize the balance of baseline covariates, quadratic terms, and relevant interaction terms across cases and controls. Quadratic terms were considered for all continuous variables, and potentially relevant interaction terms were determined in consultation with clinician co-authors, who advised that the effects of clinical characteristics and treatment histories, such as stage at diagnosis and number of lines of prior systemic therapy received, were most likely to vary according to primary tumor site. We compared balance across matched and unmatched cohorts using standardized mean differences, variance ratios, quantile-quantile plots, bootstrapped Kolmogorov-Smirnov tests in continuous variables, and paired t tests in binary variables for a fixed sample size (Kolmogorov 1933; Smirnov 1939).

Cost consequence analysis

After matching, we examined the downstream patient management, in terms of uptake of genomic information for informing patient care, survival, and health system costs of POG’s approach versus usual care in the year following patients’ index date. All analyses accounted for weights related to matching with ties and replacement to avoid false imprecision. Given that POG enrollment was ongoing throughout the period, patients may not have had 1 year of follow-up prior to the study end date, December 2015. We adjusted for outcome censoring arising from this incomplete follow-up data in all survival and cost analyses. We conducted analyses in R and Stata (RC 2014; StataCorp 2017). We identified statistical significance using a threshold of p < 0.05.

Survival analysis

To assess survival differences across POG and usual care patients, we estimated Kaplan-Meier survival functions and used log rank tests. One-year survival rates were inferred based on estimated survival functions. As in previous real-world health technology assessment studies (Khor et al. 2014), we measured patient-level 1-year survival times and used inverse probability of censoring weighting analysis to account for censoring (Willan et al., 2005; Lin 2003; Bang and Tsiatis 2000). We estimated inverse probability weights for each 1-month time interval using Kaplan-Meier product limit estimates of the probability of censoring (Kaplan and Meier 1958). This multiple time-point approach to inverse probability weighting reduces estimation bias by enabling patients to factor into outcome estimates for observed months and recreating the sample population expected in the absence of censoring.

Inverse probability weighting assumes coarsening is at random beyond the specified censoring mechanism. Sensitivity analysis involving covariate-adjusted Cox proportional hazards models predicting the probability of censoring, reported in Supplemental Table 3, did not substantially alter outcome estimates or conclusions, shown in Supplemental Table 4. Weighted linear regression generated mean 1-year survival time estimates and calculated differences across patient groups. To ensure associated variance estimation reflected the two-step process of weight and outcome estimation as well as allowed for non-normality, we applied non-parametric bootstrapping to simulate sampling distributions. Reported confidence intervals are bias-corrected (Efron 1987).

To explore heterogeneity in overall survival patterns, we stratified our survival analysis according to whether POG patients experienced biopsy failure or received a treatment change based on their WGTA results. Both events may modify the effect of precision oncology research participation on downstream outcomes. The POG Program classified WGTA information as clinically actionable if a target or risk factor was found with the potential to affect the patient’s treatment plan or if the results yielded additional information beyond what was already known from prior genetic tests (Laskin et al. 2015). A multi-disciplinary tumor board reviewed WGTA results and prioritized clinically actionable alterations and corresponding treatment options. Patients who received at least one of the possible WGTA-informed treatments suggested by the tumor board were considered to be on a genomics-informed treatment plan. To assess effect modification for the three most common primary cancer types observed within-cohort, we also stratified our survival analysis.

Cost analysis

We estimated patient-level 1-year healthcare costs from the healthcare payer perspective and reported all costs in 2015 Canadian dollars. Given the time horizon, we did not discount costs or effects. Using the previously generated inverse probability of censoring weights, we also used weighted linear regression to generate mean 1-year cost estimates and calculate differences (Willan et al., 2005). Monthly cost analysis is reported on in Supplemental Fig. 2. Non-parametric bootstrapping simulated corresponding sampling distributions, and we pursued stratified analysis according to POG patient trajectories and primary tumor sites.

Scenario analyses

Scenario analysis was used to determine the impact of future changes in the pipeline for comprehensive genomic sequencing and clinical uptake of sequence data on cost consequences. Scenarios were based on both observed and predicted changes in the POG Program and cost trajectory for WGTA (Weymann et al. 2017) and were evaluated through oversampling in bootstrapped simulation. Key metrics included incremental costs, survival, and net monetary benefit (NMB) for willingness to pay (WTP) thresholds of $100,000 and $150,000 per life-years gained (LYG). These metrics were estimated in each of 1000 bootstrapped simulations. Future costs of WGTA components were forecast using our system of equations model (shown in Fig. 2) and assumed a reduction in the rate at which expected other costs changed per month after January 2020 to allow for inefficiencies introduced by the COVID-19 pandemic. Co-authors with clinical, genetics, and genomic service administration expertise (JL, SJ, RR, HL, DR, KS, SS, SY, MM) reviewed the feasibility of forecast scenarios and credibility of WGTA cost estimates.

Fig. 2
figure 2

WGTA cost trajectory forecast to 2030. WGS & RNA-seq: whole genome sequencing and transcriptome sequencing; WGTA: whole genome and transcriptome analysis. Dynamic forecasts were produced based on the structural equation model estimated by three-stage least squares

Full size image

Our first scenario reflects the current state of the POG Program. We forecast 2021 WGTA costs and assume that 15% of POG patients experienced biopsy failure and 30% receive WGTA-informed treatment. The second scenario reflects optimizations achievable in the next 5 years, using 2025 WGTA cost forecasts and assuming 10% of POG patients experience biopsy failure and 45% receive WGTA-informed treatment. The third scenario represents a best-case WGTA scenario and forecasts sequencing, bioinformatics analysis, and other WGTA cost drivers in 2030, assuming biopsy failure is eliminated, and the rate of WGTA-informed treatment reaches 60%. Our fourth scenario expands on the third by accounting for changes in the trajectory of usual care. In 2016, BC Cancer implemented a multi-gene panel for directing clinical management among patients diagnosed with specific advanced cancer types (BC Cancer 2016). We examine the incremental cost and NMB of replacing multi-gene panel testing with WGTA in eligible cancer types observed within the POG Program (gastrointestinal, lung, gynecological, hematologic, and skin cancers), given a panel cost of $1200 per patient and a 5% increase in mean survival for usual care patients.

Results

From July 2014 to December 2015, 230 patients participated in the POG Program and 93,736 patients were identified as possible controls. Of these, 5224 control patients were eligible for matching. Genetic algorithm matching identified controls for all POG patients, with 204 unique controls selected (nweighted = 230), and each repeat control matched at most three times (n = 4). Unadjusted for censoring, patients were observed over 272.67 person-years. After inverse probability of censoring weighting, this period was 460 person-years. After matching, there was no evidence of strong imbalance or significant differences in baseline characteristics across POG patients and usual care controls (p > 0.05). Detailed balance statistics are previously published (Weymann et al. 2020).

Table 1 summarizes baseline characteristics in the matched cohort. The most common primary cancer sites involved gastrointestinal cancers (30 to 31%), breast cancer (21%), and lung cancer (11 to 12%). In the month prior to the index date, average healthcare costs ranged from $2356 per POG patient (SE: $283) to $2811 per usual care patient (SE: $311). On average, patients were followed for 0.59 year prior to death or censoring. During follow-up, 63% of POG patients and 62% of usual care patients were right-censored, with 1-year cost and survival outcomes measured until the end of December 2015.

Table 1 Baseline study characteristics
Full size table

Among POG patients, 21% (n = 48) provided biopsy samples containing insufficient tumor content for reliable WGTA. This rate of biopsy failure varied across primary tumor sites, ranging from 32% in gastrointestinal cancers to 28% in lung cancer, 10% in breast cancer, and 15% in other cancers (χ2 = 11.23, p value = 0.001). While the remaining 79% (n = 182) of POG patients underwent WGTA, not all experienced a treatment change based on their results. During the study period, 82% (n = 146) of POG patients who underwent WGTA received a clinically actionable finding, 15% (n = 35) of POG patients received WGTA-informed treatment, and remaining patients did not, owing to a number of factors previously reported in Weymann et al. (2020). Common reasons other than no actionable findings included no targeted treatments available or accessible to patients (n = 11, 6%), currently responding to a non-targeted treatment option (n = 24, 12%) or declining health status (n = 31, 16%). Rates of WGTA-informed treatment ranged from 10% in gastrointestinal cancers to 16% in breast cancer and 17% in lung and other cancers, although differences were not statistically significant (χ2 = 1.81, p value = 0.61). WGTA-informed treatments were accessed through off-label or experimental use of drugs either in clinical trials, restricted in funding, or not publicly funded (29%), and as standard therapies (71%). Average turnaround time from date of biopsy to WGTA report date among sequenced patients was significantly shorter for WGTA-informed than for non-informed POG patients (Δ\(\overline{x}\)TAT = 14.22 days; 95% CI: 3.16, 25.29).

Costs

Table 2 describes the average costs and survival times for POG and usual care patients. On average during the study period, the application of WGTA cost $26,211 per patient (SD: 14,191). WGTA costs were driven by whole genome sequencing and transcriptome sequencing (mean: $12,277; SD: 6858), bioinformatics analysis (mean: $4034; SD: 2221), and other WGTA cost components (mean: $9901; SD: 5219) including biopsy and sample processing where required beyond standard care surgical procedures, validation through panel sequencing, verification experiments and/or confirmatory testing, PET scans, and fixed program costs for staff salaries, equipment, and platform upgrades. Changes in the WGTA cost trajectory depicted in Fig. 2 demonstrate that while sequencing costs are likely to continue declining rapidly in the future, certain cost components, such as bioinformatics analysis, remain relatively stable. Actual expenses measured beyond the model development period, from January 2016 to March 2018, highlight the high level of uncertainty involved when extrapolating costs. While sequencing and bioinformatics cost forecasts performed reasonably, rapid declines in other WGTA cost components were not well predicted. These unforeseen cost reductions reflect changes in the POG pipeline, including terminating panel sequencing, reducing the use of PET scans, and shifting to routine maintenance of automated bioinformatics tools. Subsequent initiation of parallel precision oncology research initiatives provided further opportunity for efficiency gains at the Genome Sciences Centre conducting WGTA (e.g., NCT02869802; Malkin et al. 2020).

Table 2 Average costs and survival outcomes over 1 year
Full size table

We also estimated that mean healthcare costs were $5203 (95% CI: 75, 10,424) higher for POG patients than for usual care patients (Table 2). These results indicate that integrating WGTA information into clinical decision-making will have downstream resource utilization impacts beyond the upfront costs of sequencing. Subgroup analyses presented in Table 3 show that cost differences between POG and usual care patients were highest in patients with gastrointestinal cancers (Δ\(\overline{x}\)total cost, GI $10,979; 95% CI: 666, 22,029) and lowest, albeit not statistically significant, in patients with lung cancer at −$2609 (95% CI: − 14,323, 9206). Additional stratification within POG patients signaled higher healthcare costs in those who experienced biopsy failure (Δ\(\overline{x}\)total cost, Bx fail = $8553; 95% CI: 768, 18,883) or received WGTA-informed treatment (Δ\(\overline{x}\)total cost, WGTA-informed = $12,964; 95% CI: 5597, 24,830) compared to usual care, and no difference in those who underwent WGTA and received no treatment change (Table 2).

Table 3 Subgroup analysis of mean healthcare costs and survival
Full size table

Observed differences were driven by variation in service utilization within POG patients. Compared to usual care patients, POG patients who received WGTA-informed treatment and those who experienced biopsy failure incurred weakly significantly higher physician service costs (Δ\(\overline{x}\)physician cost, WGTA-informed = $1025; 95% CI: 43, 2148; Δ\(\overline{x}\)physician cost, Bx fail $1227; 95% CI: − 104, 2643) and higher other costs (Δ\(\overline{x}\)other cost, WGTA-informed = $1674; 95% CI: 503, 3037; Δ\(\overline{x}\)other cost, Bx fail $1058; 95% CI: − 69, 2325), including outpatient prescription drugs, radiation therapy, cancer-related medical appointments, and testing. The costs of WGTA-informed systemic therapy were also weakly significantly higher than usual care systemic therapy (Δ\(\overline{x}\)ST cost, WGTA-informed = $7567; 95% CI: 1058, 16,873), suggesting that targeted treatments informed by genomic sequence data were more costly than standard care.

Monthly cost analysis depicted in Supplemental Figs. 2A and 2B demonstrates that the majority of healthcare cost accrual occurred early in the study period for all patient groups. In the first month following the index date, POG patients incurred significantly higher costs than usual care patients (Δ\(\overline{x}\)month 1, total costs = $1828; 95% CI: $597, $3119), with few differences observed later in the period. Declining trends reflect the link between survival and cost accrual, with an increasing number of deaths occurring in this advanced cancer cohort over time. Higher survival rates in WGTA-informed patients drove higher costs in later months compared to other patient groups.

Survival

During the follow-up period, 59 deaths (26%) occurred in POG patients and 61 (26%) occurred in usual care patients. Estimated KM survival functions are presented in Fig. 3, and mean 1-year survival times are presented in Table 2. Across POG and usual care patients, there were no significant differences in overall survival (χ2 = 0.00, p value = 0.966) or in mean 1-year survival (\(\overline{x}\)survival, POG = 8.13 months and \(\overline{x}\)survival, usual = 8.16 months; p > 0.05). Estimated 1-year survival rates were 58.56% (95% CI: 47.98%, 67.72%) in POG patients and 59.73% (95% CI: 49.35%, 68.65%) in matched controls. Subgroup analysis according to primary tumor site suggested possible outcome heterogeneity, with the largest incremental 1-year survival gains observed in lung cancers (Δ\(\overline{x}\)survival, lung = 0.88 month; 95% CI: − 1.78, 3.39) and losses observed in breast cancers (Δ\(\overline{x}\)survival, breast = − 0.83 month; 95% CI: − 2.90, 1.13), but no significant differences were detected (p > 0.05) (Table 3). Stratified overall survival functions presented in Supplemental Fig. 1 signaled similar non-significant variation across tumor sites.

Fig. 3
figure 3

Kaplan-Meier survival estimates. Sub-graphs depict Kaplan-Meier survival functions across usual care and POG patients, either pooled or stratified according to whether they experienced biopsy failure or a treatment change based on their WGTA results. Risk tables present the number of uncensored patients at risk of death at the beginning of each interval across groups

Full size image

Further stratification within POG patients signaled improved overall survival in POG patients who received WGTA-informed treatment and reduced survival in those who experienced biopsy failure (χ2 = 15.44, p value = 0.002). POG patients who experienced biopsy failure had a weakly significant incremental 1-year survival loss of 1.39 months (95% CI: − 2.89, − 0.01) compared to usual care patients. In contrast, POG patients who underwent WGTA and received a genomics-informed treatment had a significant survival gain of 2.49 months (95% CI: 1.32, 3.64) compared to usual care, and those who received no treatment change incurred no survival difference. Estimated 1-year survival rates ranged from 72.79% (95% CI: 44.69%, 88.23%) in POG patients who received WGTA-informed treatment to 61.26% (95% CI: 46.19%, 73.28%) in those who underwent WGTA but received no treatment change and to 44.34% (95% CI: 27.27%, 60.10%) in patients who experienced biopsy failure.

Scenario analysis

Table 4 describes the results of our scenario analyses exploring how changes in observed patterns may affect the cost-effectiveness of WGTA. In our first scenario reflecting the current state of the POG Program, we forecast WGTA costs reach $11,763 per patient and find incremental survival gains of 0.53 month (95% CI: − 0.41, 1.37) and incremental downstream healthcare costs of $6805 (95% CI: $1414, $12,375). Accounting for the additional costs of applying WGTA, probabilistic estimates revealed that 0–3% of the samples were cost-effective at either $100,000 or $150,000 per LYG. In our second scenario, reflecting short-term optimizations in WGTA costs, rates of biopsy failure, and WGTA-informed treatment, WGTA cost $7491 per patient and POG resulted in higher incremental survival gains and higher incremental healthcare costs than usual care, such that 7% and 37% of the samples were cost-effective at either $100,000 or $150,000 per LYG, respectively. In our third scenario, forecasting that WGTA cost $4721 per patient and assuming biopsy failure is eliminated and the rate of WGTA-informed treatment reaches 60%, estimated incremental survival gains were 1.67 month (95% CI: 0.81, 2.53) and incremental healthcare costs were $9502 (95% CI: $4301, $14,842). In this scenario, 47% of the samples were cost-effective at $100,000 per LYG and 88% were cost-effective at $150,000 per LYG. Our fourth scenario assumes that usual patients diagnosed with gastrointestinal, lung, skin, hematologic, or gynecologic cancers become eligible for a $1200 multi-gene panel. By introducing cost savings through avoiding multi-gene tests among WGTA recipients and increasing mean survival among usual care patients, we find that 25% of the samples were cost-effective at $100,000 per LYG and 65% were cost-effective at $150,000 per LYG. This scenario analysis highlights the sensitivity of future cost-effectiveness to changes in the rates of WGTA-informed treatment, biopsy failure, WGTA cost drivers, and comparators.

Table 4 Results of scenario analysis
Full size table

Discussion

Our study is the first to estimate the downstream cost consequences of research-based genomic sequencing versus usual care in oncology. We draw on real-world data and apply quasi-experimental matching to identify a well-balanced usual care counterfactual for patients enrolled in the single-arm POG research program. This counterfactual includes many patients who developed metastatic disease after their initial cancer diagnosis (82%), a patient group often ignored because population-based cancer registries only record initial staging information (Warren et al., 2015; Soysal et al. 2017). We find that in the absence of sequencing, patients with advanced cancer incurred an average of $23,306 per year (95% CI: 19,277, 27,650) in healthcare costs, predominantly driven by hospitalization and systemic therapy costs. Weighted regression estimated that over 1 year, applying WGTA in a research setting yielded higher downstream healthcare costs of $5203 per patient (95% CI: 75, 10,424) and no significant survival differences compared to usual care. These estimates suggest that the application of research-based comprehensive genomic sequencing in advanced cancer settings will have important downstream resource utilization implications that decision-makers need to consider before implementing into a learning healthcare system. Our results contrast with cost reductions observed in prior studies examining multi-gene panels for treatment de-escalation in earlier-stage cancers or whole genome and whole exome sequencing for diagnosing rare childhood illness (Zambelli et al. 2020; McSorley et al. 2021; Mackay et al. 2020; Vrijenhoek et al. 2018). Additional research is needed to determine whether similar patterns in advanced cancers hold in other jurisdictions.

Subgroup analysis signaled a variation in cost and survival outcomes across tumor sites and according to clinical uptake of sequence data. Consistent with other applications of genomics in oncology, tumor content of POG patients’ biopsy samples was not always sufficient to support reliable sequencing and bioinformatics analyses (Damodaran et al. 2015). Patients who experienced biopsy failure appeared to have relatively poor survival and incurred higher costs compared to usual care patients. These results may reflect prognostic differences among patients likely to experience biopsy failure, although past research remains inconclusive about which prognostic factors beyond organ biopsied correlate with tumor content (Tacher et al. 2016; Sabir et al. 2017). Patients with successful biopsies who later received a WGTA-informed treatment instead experienced improved survival, albeit at higher cost. Stratified analysis highlights an opportunity to improve the prognoses of patients experiencing biopsy failure through the identification of targetable aberrations and provision of genomics-informed treatment. Additional research is needed in verifying the reproducibility of these results.

Study findings broadly align with non-randomized evidence signaling the potential efficacy of genomics-informed care, although randomized controlled trials have yet to detect significant health benefits (Zehir et al. 2017; Zimmer et al. 2019; Tourneau et al. 2015). Our results are promising, considering that the most accessible WGTA-informed treatment to patients during the study period was not always the preferred drug indicated by sequencing. These patients also had their WGTA results 2 weeks faster than patients who did not receive a WGTA-informed treatment, reflecting a variation in automated pipelines over time and whether automated reports identified immediately actionable therapeutic targets (Laskin et al. 2015; Tsang et al. 2019). Variation in incremental costs and survival reflects differences across patients more likely to experience biopsy failure or to receive a genomics-guided therapy. For example, patients with gastrointestinal cancers had the highest rate of biopsy failure and experienced the largest difference in downstream healthcare costs across POG and usual care. Such heterogeneity is to be expected in a tumor-agnostic setting and demonstrates that universal access to genomic sequencing is unlikely to yield equal benefit or costs for all indications (Murphy et al. 2021).

To understand the implications of future program changes on downstream costs, survival, and NMB, we conducted scenario analysis using a novel application of non-parametric bootstrapping. In contrast to decision models that draw on multiple disparate literature sources to inform outcome projections, our approach takes advantage of the detailed individual-level data available and infers future outcomes based on observed heterogeneity. We found that the probability that comprehensive genomic sequencing yields overall survival benefits and is cost-effective compared to usual care is highly sensitive to changes in sequencing and bioinformatics costs as well as the generation and clinical uptake of sequence data. After forecasting that WGTA costs fall to $4721 per patient and assuming that biopsy failure is eliminated and the rate of WGTA-informed treatment falls to 60%, we found that 47% of the samples were cost-effective at $100,000 per LYG and 88% were cost-effective at $150,000 per LYG. While these assumptions may seem optimistic, 82% of sequenced patients had clinically actionable findings generated within-sample and an uptick in the number of trials for molecularly targeted therapeutics since the end of the study period may give rise to increasing rates of genomics-informed treatment. Further improvements in biopsy sampling and processing techniques, patient selection, and sequencing technology advances reducing tumor content thresholds may serve to minimize biopsy failure. These results will inform future program optimizations, trial designs, as well as provide stakeholders with evidence highlighting when the cost consequences of comprehensive precision oncology are likely to become favorable.

Our early-stage cost consequence estimates based on historical data from 2014 to 2015 necessarily involve considerable uncertainty. Genomic sequencing technologies are evolving rapidly, as is the knowledge base for interpreting and integrating genomic data into clinical care (Goodwin et al. 2016; Zhao et al. 2019). Since the termination of our study period, many new biomarkers and corresponding targeted treatments have emerged, potentially increasing the accessibility and efficacy of genomics-informed care (Collins et al. 2017; Lee et al. 2018). Costs of WGTA-informed treatments dispensed in clinical trials or paid for out of pocket by patients are not captured in this study. Some of these therapies may now be publicly funded, and our results may underestimate current healthcare payer expenditures. Standard care in BC has also progressed, with multi-gene panels becoming accessible for certain advanced cancer indications, enabling the identification and treatment of biomarkers originally only detectable through WGTA (Cancer 2016). To ensure that stakeholders are able to recognize associated uncertainty, we generate disaggregated outcome estimates and highlight the sensitivity of our findings in scenario analysis.

Our study also involves a small sample size and relatively short follow-up period, introducing right censoring and limiting the statistical precision of our effect estimates. Further research powered to detect more detailed subgroup effects is needed to determine whether optimal genomic sequencing implementation involves targeting specific tumor sites or certain clinical contexts. While our time horizon enables characterization of immediate downstream impacts of research-based genomic sequencing for participants and health systems, more evidence on the long-term implications of generating sequence data over a larger cohort is warranted. For example, focusing on a 1-year time horizon may underestimate the effects of genomics-informed treatment if survival is substantially prolonged for these patients. Sequencing may also reveal germline findings that can identify hereditary cancer families and lead to targeted risk reduction in susceptible family members, resulting in additional downstream benefits. Re-biopsy and re-sequencing may become more common over time, introducing repeat costs as new information on potential therapeutic targets and resistance mechanisms are discovered and tumor evolution continues.

Another limitation of our study stems from our use of an observational rather than RCT study design. While non-randomized and single-arm studies are common in precision oncology (Rodon et al. 2019; Voest et al. 2020; Zimmer et al. 2019), quasi-experimental matching is a tool to identify counterfactuals and mitigate selection bias. Matching involves strict assumptions around ignorability and balance, deviations from which can introduce bias into effect estimates (Rosenbaum and Rubin 1983; Rubin and Thomas 1996). We found that genetic algorithm matching achieved balance on entire distributions of all covariates of interest, including quadratic and interaction terms. Although these balance statistics support the validity of our conclusions, unobserved confounding remains a possibility in any observational study relying on routinely captured administrative data fields. For example, while POG inclusion required physician-assessed life expectancy to exceed 6 months, this information is not systematically recorded for usual care patients in BC Cancer’s administrative data. We are unable to verify whether balance on this variable was achieved, potentially introducing upward bias into estimated survival for POG patients. Future observational studies would benefit from the development of validated algorithms designed to recognize prognostic factors and changes in patients’ disease trajectories. Linkage with additional data sources, such as prior laboratory testing and genetic test results, may enable more comprehensive covariate adjustment. The study index date, date of POG biopsy, was unobserved for control patients and is critical to measuring downstream cost consequences. While we employed a random date assignment technique similar to prior studies of groups who did not experience the same index date (de Oliveira et al. 2016; Jarosek et al. 2015; Walker et al. 2013), and explored an alternative multiple imputation-based date assignment in sensitivity analysis, additional research is needed in refining data assignment methods for matched cohort analysis.

Conclusion

Our study demonstrates the ability of real-world data to support evaluations of precision oncology research initiatives. Through quasi-experimental matching, we identified changes in patients’ clinical trajectory attributable to genomic sequencing and found evidence of higher downstream healthcare costs among research participants compared to usual care. The magnitude and significance of detected effects varied across patient subgroups, supporting stakeholders’ need to consider outcome heterogeneity. Scenario analysis revealed that changes in rates of WGTA-informed treatment, biopsy failure, and the price of WGTA will have a large impact on the probability that population-wide sequencing in advanced cancers is cost-effective compared to usual care. This evidence informs the early-stage economic value of precision oncology and will support future evaluations of only-in-research health technologies.