Introduction

RSV is a highly infectious disease, with an unpredictable course and a significant clinical and medical burden, as well as social disruption and economic costs. Approximately 70% of children ≤ 12 months and 90% of children ≤ 2 years [1] became infected by this virus, asserting that a significant portion of the infant population faces a high risk of contracting this infectious agent. Clinic evolution of infant cases is difficult to predict, and can include lower respiratory tract infection (LRTI), such as pneumonia and bronchiolitis, entailing hospitalization, use of pediatric care units, oxygen support, palliative care or, in some cases, leading to death [2]. Furthermore, a significant proportion of cases occur in previously healthy infants, without underlying pathologies or at risk [3]. It is estimated that 43.2 out of every 1,000 children aged ≤ 1 are hospitalized with RSV (data from September 2017 until June 2018, in Spain) [4].

Over the past 60 years there have been several unsuccessful attempts to find candidates for RSV prevention or effective antiviral treatment for infants [5,6,7,8]. Thereby, RSV infections are a public health problem recognized by the World Health Organization (WHO), which advocates for a comprehensive RSV prevention strategy, preferably during the initial 15 months of life [9].

Decisions on the allocation of public health resources are complex, as they must incorporate patient access to innovations whilst safeguarding the financial sustainability of the system, in an environment of prominent demographic, technological, social, and budgetary challenges. Healthcare decision-makers use manifold tools to guide the decision-making process, such as economic evaluation and budget impact analysis. However, other factors are often also considered, such as severity of disease, availability of preventive and treatment alternatives, size of population affected, equity, social impact, quality of available evidence or degree of technological innovation.

In light of the aforementioned factors, the multi-criteria decision analysis (MCDA) is an instrument that yields ways of informing the preferences inherent to the decisions, in a consistent, explicit and transparent manner [10]. Despite the recent use of MCDA in health (it has been applied in other areas such as environment/waste, logistics/transportation, management, agriculture), this methodology has been implemented in practice in decision-making in many countries, including Spain [11,12,13].

In Spain, a new preventive measure for RSV (nirsevimab) has been recently approved [14]. Nirsevimab (Beyfortus®, Sanofi, Paris, France, and AstraZeneca AB, Södertälje, Sweden) is a recombinant human IgG1 kappa engineered monoclonal antibody that binds the F1 and F2 subunits of the RSV fusion (F) protein (antigenic site 0 of the protein in its pre-F form) to block viral entry into the host cell. It has been authorized by the European Medicines Agency (EMA) for the prevention of RSV LRTI in neonates and infants during their first RSV season, in October 2022 [14]. Moreover, other preventive measures (monoclonal antibodies, vaccines for infants, etc.) are expected to be introduced in the future [15]. All these interventions will require a specific evaluation framework.

MCDA assessments often necessitate the inclusion of a comparator for analysis. Presently, the prevention of RSV has only one available alternative, palivizumab, which is utilized in a mere 1.4% of the population in Spain [16, 17]. Palivizumab is authorized for preventing severe lower respiratory tract disease caused by RSV in children at high risk [18]. Consequently, opting for “placebo” as the comparator in this MCDA is the most appropriate choice, as it accurately represents the current standard of care in Spain. Thus, in the context of the ongoing MCDA in RSV, the term “placebo” should be understood as indicating no intervention.

The central objective of this research was to develop a novel MCDA framework for RSV antibody-based preventive alternatives and to assess the value of nirsevimab vs. placebo as a systematic immunization approach to prevent RSV in neonates and infants during their first RSV season in Spain.

To accomplish these objectives, we opted for Vaccinex [19] as the reference model, given its comprehensive nature in comparison to other identified frameworks used for evaluating preventive measures, such as vaccines [19,20,21,22]. Building upon this framework, we developed a tailored set of criteria adhering to the ISPOR guidelines on good practices [23]. These guidelines highlight the significance of selecting criteria that meet specific requirements, including completeness, non-redundancy, non-overlap, and preferential independence [23].

Methods

Study design

The following steps were undertaken in order to carry out this MCDA (Fig. 1): (1) a multidisciplinary committee of experts was constituted; (2) a first narrative review of the literature was performed, and as a result, a pre read document was elaborated (supplementary file 1); (3) experts were trained in the MCDA methodology; (4) based on the aforementioned pre-established framework (Vaccinex [19]) experts defined the framework for RSV, i.e. the set of criteria which should be used when evaluating any current or future antibody-based preventive measures in RSV; (5) experts weighted the selected criteria using the 1–5 scale method, thus making explicit their preferences on the relative importance of each one of them (where 1 means that the criterion is not very relevant in the evaluation of any preventive measure in this pathology, and 5, very relevant), regardless of the preventive measure to be evaluated; (6) a second narrative review of the literature was undertaken, resulting in an evidence summary document (supplementary file 2); (7) experts scored, online, individually, and blinded, aspects related to the pathology and the value contribution of nirsevimab vs. placebo in the prevention of RSV. For the absolute criteria (which do not compare the alternatives), the score ranged on a scale from 0 to 5, with 0 being the lowest value and 5 being the highest. For the relative criteria (which compare nirsevimab vs. placebo), the scale ranged from − 5 to 5 to reflect the full range of comparative effects; (8) an analysis of the results and the calculation of the final estimated value was realized; (9) the results of the scores and the final estimated value of nirsevimab vs. placebo were presented and discussed; (10) a retest of the weights and scores was performed; 11) three sensitivity analysis were conducted.

Fig. 1
figure 1

Study design

Full size image

Main results derive from the test (Sect. 3.1–3.5), whilst the retest and sensitivity analyses are portrayed in a specific Sect. (3.6).

Expert panel

This study was developed through a multidisciplinary committee of experts (MCE) formed by 9 members with different academic profiles. The choice of the MCE considered a balance between geographical representation (5 Autonomous Communities of Spain), the presence of different academic background (professionals in the field of pediatric, virology, preventive medicine, public health, epidemiology, government affairs and health economics), and the experience of the committee members in the management and prevention of respiratory virus and healthcare decision-making. Finally, an additional criterion for inclusion was the absence of conflict of interest from the MCE experts.

Literature review

The information obtained by the two comprehensive non-systematic literature reviews were assembled into a pre read document (supplementary file 1) and an evidence summary (supplementary file 2), which were reviewed and validated by the clinicians from the MCE. The search was carried out in the main biomedical databases, such as PubMed, clinical trial registries, clinical practice guidelines, web pages of the official European and Spanish health assessment bodies, such as EMA, the Spanish Agency for Medicines and Health Products (AEMPS) or the health technology assessment agencies, as well as in grey literature sources. Publications in English and Spanish were included.

Framework

The final framework consists of a set of selected criteria (see results section). Details such as how criteria were selected, the names of each criterion, the domains to which they belong, their definitions and their scoring scales, can be found in the supplementary files 3 and 4.

Data analysis

An overall estimated value (ranging from − 1 to 1) was obtained for the comparison between nirsevimab and placebo, through an additive linear model of all individual criteria value contributions, which were calculated as the product of normalized weights and scores:

$$ V = \sum\limits_{x = 1}^n {{V_x}}= \sum\limits_{x = 1}^n {\left( {\frac{{{w_x}}}{{\sum {{W_n}} }}{S_x}} \right)}$$
(1)

where V is the total estimated value, Vx the value contribution of the criterion x, Wx the weighting of the criterion x, ∑Wn the sum of all weights, and Sx the normalized score of the criterion x (Sx = score / 5).

Retest and sensitivity analysis

A retest and three sensitivity analysis were performed. Firstly, to evaluate the consistency, replicability, and internal validity of the results, a final estimated value was calculated with the results obtained by the retest and was contrasted with the base case value. The degree of agreement between the responses given at the two time points (test and retest) was evaluated by means of intra-rater correlation coefficients (ICC 3.1) using STATA® version 14 (STATA Corp., LP, College Station, TX, USA). Secondly, to assess the extent to which a change in the expert weights would affect the final estimated value of this MCDA, the expert weights were replaced with the original weights from the Vaccinex framework [19], and the impact on the final estimated result was analyzed. Thirdly, an outlier (measured by more than 1.5 times the central quartiles of the final estimated results) was excluded from the analysis, and this result was compared with the one from the base case. Finally, an evaluation of nirsevimab vs. palivizumab (a monoclonal antibody authorized for the prevention of serious lower respiratory tract disease requiring hospitalization caused by RSV in children at high risk for RSV disease [18, 24]) was also performed.

Results

Framework

Sixteen criteria were excluded from the Vaccinex framework (eradication potential, mode of transmission, image and goodwill, resistance offered by anti-vaccination groups, impact on the population of pregnant women, impact on school activities, time to development of symptoms, costs related to the production platform, productivity costs [presenteeism], generation of jobs in the country, impact on migrant population, impact on fertility, impact on the Lesbian, Gay, Bisexual, Transgender, and Intersex (LGTBI) population, impact on the female population, perception and fear, legal liability), and two criteria which were not present in Vaccinex were included (burden of disease – incidence on the outpatient setting; burden of disease – incidence on the inpatient setting). In addition, the disease prevalence criterion was renamed as “population in which the prevention strategy would be indicated”. The final framework for RSV assessment consists of 26 criteria (Tables 1 and supplementary files 3 and 4).

Table 1 Multi-criteria decision analysis framework for the evaluation of antibody-based preventive measures for respiratory syncytial virus infections
Full size table

Weights

Based on the mean weights given by the experts, the five criteria considered as most relevant to any evaluation of antibody-based preventive measures in RSV were efficacy of the preventive measure (4.6 ± 0.7 out of 5.0), severity of symptoms (4.4 ± 0.7), incidence on the inpatient setting (4.4 ± 0.9), lethality risk (4.2 ± 1.2) and serious adverse events (4.2 ± 1.3), while those of lesser relevance were innovation stimulus (2.3 ± 1.1), impact on productivity cost (2.6 ± 0.9), impact on caregivers (2.6 ± 1.0), public health awareness (2.7 ± 1.1) and cost of the disease on the patient (out of pocket expenses) [2.7 ± 0.5] (Fig. 2). See supplementary file 5 for detailed weights, overall and by subgroups.

Fig. 2
figure 2

Relevance of each individual criterion in the assessment of any antibody-based preventive measures for respiratory syncytial virus (mean, min, max and median weights, and standard deviations). The 1–5 scale method was applied, where 1 means that the criterion is not very relevant in the evaluation of any preventive measure in this pathology, and 5 means that it is a very relevant criterion

Full size image

Absolute scores

The examination of the absolute scores (Fig. 3 and Table 2) suggests that RSV is a disease with no effective alternative treatment and with a prevention alternative which is very limited (mean scores, availability of treatment: 4.4 ± 0.7 out of 5.0; prevention alternatives: 4.2 ± 0.7). The only antiviral treatment available (ribavirin®) can only be administered in severe cases, requires aerosolization in specific devices and its routine use is not recommended [25, 26]. The only prevention alternative, palivizumab®, is only indicated for high-risk premature infants and in a hospital setting [18, 27].

Fig. 3
figure 3

Scores assigned per absolute criterion (mean, standard deviations, min, max, median scores, and weights). Scale from 0 to 5

Full size image
Table 2 Mean scores and key comments from the multidisciplinary committee of experts or based on evidence, for each absolute criterion
Full size table

Furthermore, children with a disease caused by an infection by RSV are at high risk of developing comorbidities in the short, medium, and long term (mean score comorbidity risk: 3.6 ± 0.9). Additionally, the incidence of RSV is very high (mean score incidence of RSV cases: 4.0 ± 0.7), with infections occurring in 90% of children within their first 2 years of life [1, 28]. This translates into a very high burden of disease, inappropriate consumption of antibiotics and healthcare resources, as reflected by its inpatient and outpatient incidence (39,690 RSV cases/100,000 children ≤ 1 years, 9 medical visits per infected patient [4]) vs. (2,520 hospitalizations for RSV/100,000 children ≤ 2 years [29]), respectively (mean scores: incidence on the inpatient setting: 3.3 ± 1.3; incidence on the outpatient setting: 4.1 ± 0.8).

Moreover, it is a highly transmissible pathology (R0 = 4.5 [30], mean score transmissibility: 3.7 ± 1.0), with a high impact on the population of children (mean score: 4.2 ± 1.0), who present losses in quality of life after diagnosis and stress levels after hospital discharge of 39% and 79%, respectively [31, 32].

Finally, bronchiolitis (one of the main diseases caused by RSV) has a rate of 82 deaths per every 100,000 children ≤ 2 years hospitalized [33], which is not considered as very high by the experts (mean score lethality risk: 2.1 ± 1.2). In addition, it presents moderate symptoms (mean score severity of symptoms: 3.0 ± 1.1) that usually last for three to seven days, which is regarded as not very long lasting by the MCE (mean score time of duration of acute symptoms: 2.7 ± 1.0). See supplementary files 5 and 6 for detailed absolute scores.

Relative scores

Overall, relative scores (Fig. 4 and Table 3) indicate that nirsevimab was considered by the MCE as a preventive measure with clinical benefits, as it is much more effective than placebo (80% efficacy in preventing medically attended RSV-associated LRTI [34]; mean score efficacy of the preventive measure: 4.3 ± 0.5 out of 5.0), with a robust safety profile (safety outcomes associated with nirsevimab are comparable to those observed with placebo) (mean score serious adverse events: -0.1 ± 0.3).

Fig. 4
figure 4

Scores assigned per relative criterion (mean, standard deviations, and weights). Scale from − 5 to 5

Full size image
Table 3 Mean scores and key comments from the multidisciplinary committee of experts or based on evidence, for each relative criterion, nirsevimab vs. placebo
Full size table

This clinical benefit, added to the fact that nirsevimab is indicated for neonates and infants during their first RSV season (mean score population in which the prevention strategy would be indicated: 4.2 ± 1.6), is also associated with several economic and social benefits.

To assess the economic benefits, since the price of nirsevimab was unknown at the time of the study, and for the purpose of this exercise, it was benchmarked with the price of innovative vaccines in Spain. Thereby, experts granted a negative score on the monetary cost of the preventive measure, in relation to placebo (-3.7 ± 1.8).

Furthermore, there was consensus (> 85% positive scores) that the implementation of nirsevimab vs. placebo would generate savings in other health system costs, such as hospitalizations, outpatient consultations, or emergency room care (mean score cost of the disease on the health system: 3.3 ± 1.5); that it would improve the labor productivity of caregivers, generating savings in costs related to absenteeism (mean score productivity cost – absenteeism: 2.6 ± 1.4); and in patients’ out-of-pocket expenses (mean score cost of the disease on the patient: 1.8 ± 1.8).

Moreover, the social benefits of using nirsevimab in comparison to placebo would translate, on the one hand, into an increase in public health awareness through greater social debate and greater parental attention in the case of introducing this preventive measure in the children childhood immunization schedule (mean score public health awareness: 2.3 ± 1.6). On the other hand, its universal recommendation would result in the elimination of health inequity (mean score impact on health inequity: 2.8 ± 1.6), contributing to avoiding RSV infections in the most disadvantaged populations, for whom RSV represents a greater burden in terms of comorbidity and mortality (see supplementary files 5 and 6 for detailed relative scores).

Final estimated value

The final estimated value (overall means, n = 9) obtained in this MCDA in RSV was 0.56 ± 0.11 (0.32–0.67) for the comparison between nirsevimab and placebo (Fig. 5), meaning that the introduction of nirsevimab would provide positive value in the view of the MCE.

Fig. 5
figure 5

Value contribution of nirsevimab compared to placebo according to the multi-criteria decision analysis framework for the assessment of antibody-based preventive measures for respiratory syncytial virus (mean, standard deviation, min, max)

Full size image

Out of this total, 0.43 was related to the absolute criteria, whilst 0.13 was associated with the relative criteria. Individually, the criteria with the highest contribution to the final estimated value were efficacy of the preventive measure (7.5% of total value), availability of treatment (6.7%), impact on the population of children (6.5%), incidence on the outpatient setting (6.5%), and incidence of RSV cases (6.1%). See supplementary file 5 for detailed results.

Retest and sensitivity analysis

The consistency of the weights between test and retest was high, with an average intra-rater correlation coefficient (ICC) of 0.8366. Likewise, the retest scores and value estimates were very similar to the test, with average ICCs of 0.9189 and 0.9196, respectively. The retest value estimate was 4.2% higher than the test´s.

When replacing the original weights of this MCDA in RSV with the weights assigned in the Vaccinex study, the impact on outcomes was close to none, with final estimated values ranging from 0.55 to 0.56 (vs. 0.56 in the MCDA in RSV).

By excluding the outlier, the final estimated value increased 5.4%, resulting in 0.59. See supplementary file 7 for more details.

An alternative evaluation comparing nirsevimab vs. palivizumab was also undertaken in the context of RSV prevention in neonates and infants during their first RSV season, obtaining very similar results (final estimated value of 0.58). The criteria which mark the distinction between these two comparisons were the efficacy of the preventive measure (nirsevimab has an estimated value of 0.04 vs. placebo and 0.02 vs. palivizumab) and its monetary cost (nirsevimab obtained an estimated value of -0.03 vs. placebo, and 0.02 vs. palivizumab) (see supplementary file 8).

Discussion

Complex decision contexts, such as evaluations of new preventive measures for diseases with many important unmet needs, require evaluators and decision makers to balance multiple needs and ways to address them. The main methodological approaches currently used to support evaluations are budget impact and efficiency analysis models [35].

However, the MCDA methodology can be particularly useful as a complement to this approach, as it consists of a structured (stepwise validated methodology), multidimensional (participation of experts from different professional fields), transparent (criteria, weights and scores are explicit) and systematic (replicable) approach, which incorporates several criteria and their individual value contribution to the decision or resource allocation problem [36]. Its popularity has been evident in the healthcare field in recent years, both nationally and internationally [36,37,38,39], in various areas, such as oncology [40,41,42], rare diseases [43,44,45], dermatology [46,47,48], and in the field of decisions related to the implementation of vaccines [19, 20, 22, 49,50,51,52].

This MCDA has adopted a holistic and transparent methodological approach in the evaluation of the value contribution of nirsevimab compared to placebo for the prevention of RSV in neonates and infants during their first RSV season in Spain, through a multidisciplinary panel of experts involved in the clinical, management and decision-making aspects of the pathology.

Preventive measures have some characteristics that are different from most drugs and health technologies, such as the ability to eradicate disease and more holistic social effects; therefore, it is essential to evaluate them according to other broader criteria [19,20,21,22, 49,50,51,52]. To this end, the Vaccinex framework [19] was adapted into a set of 26 criteria relevant to the context of evaluating new (present or future) antibody-based preventive measures for RSV. Based on the mean weights assigned to each criterion in the current exercise, the importance of applying a broader list of attributes became clear, as criteria related to the disease, unmet needs and social impacts represented a total weight of 58% in the current MCDA framework.

Other MCDA have been carried out in the field of vaccine evaluation and prioritization [19, 20, 22, 49,50,51,52]. However, these studies are carried out in different contexts (other countries, such as Bangladesh and Indonesia; evaluation of vaccines and not preventive measures, etc.) and using methodologies different from that applied in this MCDA in RSV (fewer criteria, and some of them interrelated, such as quality-adjusted life years [QALYs] and cost per QALY; different weighting methods [100-point distribution, rank-order centroid method], scoring [100% objective]), so they are not comparable.

The exercise performed in this MCDA in RSV found that nirsevimab adds value to the prevention of RSV in neonates and infants during their first RSV season in Spain, compared to placebo. The final estimated value in this MCDA was 0.56. This value is higher when compared to several other MCDA focused on the value of innovative drugs [43, 53,54,55,56,57,58]. Conjointly with the positive final estimated value obtained, the importance of this MCDA lies in understanding the value drivers of the preventive measure evaluated. In this sense, the multidisciplinary debate generated was key to understanding the strengths and weaknesses of nirsevimab compared to placebo in each of the elements considered.

Nirsevimab was perceived as an effective measure in the prevention of medically attended RSV infections (> 80%, outpatient and inpatient setting [34]), with a remarkably robust safety profile (safety outcomes associated with nirsevimab are comparable to those observed with placebo [59]). In addition, nirsevimab would be indicated for a much larger number of infants (nirsevimab in neonates and infants during their first RSV season vs. placebo). Moreover, its implementation could generate savings in hospitalization costs in Spain (50% or 30 million euros [17]), and greater public health awareness (generating a debate in society and increased parental awareness in relation to RSV infections). This, in turn, would contribute to diminishing health disparities among newborns and infants in their initial RSV season, consequently lowering the prevalence of RSV infections in poor regions.

These effects would be given for a disease with currently no available treatments or prevention alternatives, of high incidence (90% <2 years are infected [1]), very contagious [30] and with an important risk of hospitalization [60]. Moreover, despite the short duration of the acute symptoms of RSV infections, according to the MCE, there is a considerable percentage of mid-long-term consequences of bronchial hyperreactivity (bronchial wheezing, asthma) caused by this infection. Finally, RSV has a great impact in the quality of life and stress levels of the affected infants and their carers [31, 32].

This study has some limitations, inherent to all MCDA, which should be pointed out. The first comes from the composition of the expert committee itself, as other important stakeholders are not represented (e.g., mother, pregnant women). Similarly, it should be borne in mind that the final assessment of the MCDA comes largely from the experience, training, and value judgments of the committee members, so this type of tool is associated with a certain subjectivity. However, this is intrinsic to most decision-making processes in healthcare, and the development and use of MCDA is not intended to provide an objective ratio or a single answer to a problem requiring a decision but should be used as a complementary tool to the existing set of tools and frameworks.

In this context, it is crucial to underscore that certain criteria, such as “Costs related to the production platform,” were excluded based on expert evaluations. This specific criterion fell below the predefined threshold for inclusion in the framework, as only a minority of experts advocated for its integration. The decision to exclude it is underpinned by its connection to implementation feasibility, particularly in the context of integrating with existing vaccinations. The framework prioritizes criteria that garner broader consensus, ensuring a robust and widely accepted assessment.

Similarly, the criterion “Perception and fear of RSV” elicited diverse opinions within the expert panel and failed to meet the inclusion threshold. Subsequent discussions provided clarity on the interpretation of this criterion, ultimately leading to the decision not to incorporate it into the MCDA framework. The committee justified this exclusion based on the absence of consensus and the varied interpretations among experts.

Furthermore, the analysis entails some cognitive complexity, especially considering that this MCDA was performed with 26 criteria. Moreover, systematic literature reviews were not carried out for the creation of the pre read and evidence summary documents. Nonetheless, a comprehensive narrative review was undertaken, and all material was validated by the clinicians of the MCE, hence reducing the probabilities of any gap or inconsistency in the information set used as a base for the assessment. Finally, the evidence summary document gathered publicly available information at the time of the study: some evidence were not available (i.e., price of nirsevimab) and some were scarce (i.e., impact on other direct and indirect costs). Hence, results could show differences in light of additional information and evidence.

The strengths of this study lie in several aspects. First, it should be noted that, to our knowledge, this is the first MCDA to develop an ad hoc framework that can be applied in the assessment of any antibody-based preventive measures (present and future) for RSV infections, and to evaluate nirsevimab against placebo for those infections. Additionally, the scoring was preceded by a thorough explanation of the MCDA methodology, the assumptions made and the interpretation of the values, and the exercise was followed by the implementation of a retest to assess the consistency of the results. Finally, the evidence summary was based on a thorough review of the available information, which is fundamental for this type of exercise.

Conclusions

Unmet needs in the prevention of RSV infections and severe disease in neonates and infants during their first RSV season in Spain are still substantial. Moreover, the burden of disease (ambulatory, emergency room and hospital incidence) is very high, and there are no effective treatments or preventive alternatives available. Under a methodology of increasing use in the healthcare setting such as MCDA, nirsevimab has been evaluated by a MCE as a measure that brings added value to the current scenario of RSV prevention in Spain, by providing efficacy, with a robust safety profile (safety outcomes associated with nirsevimab are comparable to those observed with placebo) and being indicated for a much wider infant population. This might generate several economic and social benefits, such as cost savings in hospitalizations and to the healthcare system in general, an increase in the level of public health awareness towards RSV infections, and a reduction in health inequities which are inherent to respiratory infections. This type of exercise allows us to understand where the value of preventive measures lies for the different agents, encourages communication between them and can serve as a reference in decision-making on evaluation, financing, and reimbursement. In the future, it would be desirable to continue advancing in the development of the MCDA methodology and to extend its use, so that health care decision-making is carried out in a framework of greater transparency, consistency, and efficiency.