Advertisement

Quality of Life Research

, Volume 28, Issue 2, pp 399–410 | Cite as

Appraisal of patient-reported outcome measures in analogous diseases and recommendations for use in phase II and III clinical trials of pyruvate kinase deficiency

  • M. S. SalekEmail author
  • T. Ionova
  • J. R. Johns
  • E. N. Oliva
Open Access
Article

Abstract

Purpose

Pyruvate kinase deficiency (PKD) is a rare disease and understanding of its epidemiology and associated burden remains limited. With no current curative therapy, clinical manifestations can be life threatening, clinically managed by maintaining adequate hemoglobin levels through transfusion and subsequent support, but with frequent complications. Treatment goals are to maintain/improve the patient’s quality of life. With new therapies, reliable, valid, and relevant patient-reported outcome (PRO) tools are required for use in clinical trials.

Methods

Systematic literature search identified no current PRO tools for capturing/measuring the impact of PKD and treatments in clinical trials. Therefore, the search strategy was revised to consider conditions analogous to PKD in terms of symptoms and impacts that might serve as parallels to the experience in PKD; this included sickle cell anemia, thalassemia, and hemolytic anemia. Psychometric properties, strengths, and weakness of selected appropriate PRO instruments were compared, and recommendations made for choice of PRO tools.

Results

In adult populations, EORTC QLQ C30 and SF-36v2 are recommended, the former being a basic minimum, covering generic HRQoL, and core symptoms such as fatigue. In pediatric populations, PedsQL Generic Core Scale to measure HRQoL and PedsQL MFS scale to measure fatigue are recommended.

Conclusions

Some symptoms/life impacts may be unique to PKD and not observable in analogous conditions. A ‘Physico-Psychosocial Model’ derived from the ‘Medical Model’ is proposed to form the basis for a hypothesized conceptual framework to address the development of PKD-specific PRO instruments.

Keywords

Pyruvate kinase deficiency Patient-reported outcome PRO Quality of life 

Introduction

Pyruvate kinase deficiency (PKD) is the most common cause of chronic non-spherocytic hemolytic anemia (CNSHA), and is inherited as an autosomal recessive trait [1]. PKD is regarded as a rare disease, with gene frequency studies estimating prevalence of 1 per 20,000 persons in the white population [2]. Understanding of the epidemiology and burden associated with PKD remains limited, due to the low incidence of the condition. Clinical manifestations of the condition may include chronic mild or fully compensated anemia to life-threatening neonatal hemolysis requiring exchange transfusions and subsequent transfusion support [3]. Neonatal jaundice is common. In adults, the degree of anemia seems to be relatively constant, with exacerbations during acute infections and pregnancy [3]. Consequently, the disease is associated with significant negative impact on a patient’s quality of life (QoL). Thus, frequent complications of PKD include aplastic crises, extramedullary hematopoiesis, pregnancy complications of affected mothers, and need for exchange transfusion in the newborn period. Increasing severity of PKD (based on frequency of transfusion) has been associated with a trend of increasing ferritin and liver iron concentration [4].

There is currently no curative therapy for PKD, and the condition is clinically managed by maintaining adequate hemoglobin levels through transfusion and using splenectomy for patients with severe anemia or symptomatic hypersplenism [1, 5, 6, 7], both resulting in a marked increase of patient morbidity and mortality. Although hematopoietic stem cell transplantation may offer a cure for PKD, this has yet to demonstrate favorable benefit-risk balance, and is currently not standard practice [8]. Iron overload, in part due to chronic blood transfusions, often necessitates treatment with iron chelation therapy, which has been reported to result in various complications in nearly a quarter of patients receiving chelation [9]. Thus, the treatment goal for PKD is to maintain/improve the patient’s QoL.

An investigational therapy of AG-348, a small molecule allosteric activator of WT red cell pyruvate kinase, which directly addresses the underlying pathology of PK deficiency, is undergoing clinical development [10]. This new agent has potential to significantly improve a patient’s QoL. In addition, gene therapy offers promise as a safe and efficient treatment modality [11, 12, 13, 14], similar to that of genetic correction for other diseases such as β-thalassemia in humans [15].

With the advancement of new therapies and trials, there is thus a need to identify reliable, valid, and relevant patient-reported outcome (PRO) tools for use in planned clinical trials. This paper presents a systematic search of the literature to identify current PRO tools that may be used to capture/measure the impact of PKD and its treatment in clinical trials. It aimed to (1) identify appropriate PRO instruments, based on analogous diseases with similar symptoms, and compare their psychometric properties, strengths, and weaknesses; (2) build a ‘Physico-Psychosocial Model’ for PKD; and (3) provide recommendations for the choice of PRO tools for use in future clinical trials.

Materials and methods

The approach employed in this study involved three steps. First, understanding how patients experience symptoms and impacts associated with PKD. Second, developing a hypothetical conceptual framework for relevant PRO concepts in PKD and, finally, identifying the appropriate PRO measures for assessing such concepts.

The literature searches were performed in two stages. Initially, searches were carried out for literature reporting results on PROs in PKD. Secondly, studies reporting key evidence on the development or validation of the most frequently used PRO measures were searched. Initial MEDLINE searches for studies reporting PROs in PKD (using the search terms in Table 1, block 1 and block 2) yielded no relevant results, and therefore the search strategy was revised to consider conditions analogous to PKD in terms of symptoms and impacts, which might serve as parallels to the experience in PKD; this included sickle cell anemia, thalassemia, and hemolytic anemia. Records identified, screened, excluded and included, and extracted for final analysis are shown in the flow chart in Fig. 1.

Table 1

Key terms used for literature searches

Block 1

Pyruvate kinase deficiency

Block 2

Quality of life; functional status; health status; functional impairment; functional limitations; activities of daily living

Fig. 1

Flow diagram of studies in the literature review process, for the revised searches based on conditions analogous to pyruvate kinase deficiency

Thalassemia, sickle cell anemia, and PKD all give rise to splenomegaly, which results in symptoms of abdominal pain [16], chest pain [16], and anemia due to accompanying cytopenia in these diseases. All three diseases also give rise to hemolytic anemia resulting in symptoms of fatigue [17] and shortness of breath [18], sleep disturbances [19, 20], and jaundice [16, 17]. As a result of blood transfusions used as a treatment in these three diseases, iron overload occurs, resulting in heart [21, 22] or liver disease [22, 23], infections [16], and osteoporosis [24, 25]. Leg ulcers are another complication of all three diseases [16, 21, 26]. Finally, thalassemia can give rise to bone deformities [27], sickle cell disease to avascular necrosis of the bone [28], and PKD to joint pain [16]. It is these symptoms and their impacts that reduce patients’ reported reduction in quality of life, and thus it is very appropriate to use these analogous diseases to serve as parallels to the experience in PKD.

Following the development of a hypothetical conceptual framework and the identification of possible PRO measures, pragmatic searches were then carried out to identify psychometric attributes of the identified PROs. A short description of each tool is given in the supplementary data.

Results

Symptoms and health-related quality of life issues

No studies were found reporting symptoms or life impacts in PKD, except for a protocol of an ongoing observational study registered in a clinical trials database (https://clinicaltrials.gov). This study (NCT02053480), aimed at describing the range and symptoms, treatment, and complications related to PKD is underway in the USA at the Children’s Boston Hospital [29]. The study includes patients of all ages diagnosed with PKD, or patients with hemolytic anemia and a family member with genetically diagnosed PKD. The primary outcome was transfusion burden, while the secondary outcomes were patient-reported outcomes assessed on various measures such as the EuroQoL-5D-5L, Functional Assessment of Cancer Therapy-Anemia (FACT-An), Pediatric Quality of Life Inventory 4.0 (pedsQL 4.0), Pediatric Functional Assessment of Chronic Illness-Fatigue (pedsFACIT-F), and Patient-Reported Outcomes Measurement Information System Fatigue (PROMIS Fatigue).

Three qualitative studies describing the experience of patients with sickle cell disease (SCD) and patients receiving infusion iron chelation therapy provided insights into the possible patient-reported outcomes (PROs) in PKD. In addition, results from patients with chronic kidney disease (CKD)-related anemia are reported.

In another qualitative study, Stegenga et al. [30] explored the QoL of children with SCD aged 6–12 years (n = 10) receiving chronic transfusion therapy for stroke, using semi-structured interviews. Five themes reflecting the concerns of the children emerged from their data, including physical and psychological pain; school issues, i.e., attendance and being treated differently by teachers and peers; disease knowledge; transfusion therapy; and having a stroke. The authors mentioned that the children did not distinguish between disease and treatment-related aspects. In addition, given the sample size it is unclear whether these findings are generalizable.

Further, the experiences of adolescents and young adults with SCD, aged 15–35 years, have been reported by Thomas and Tylor [31], based on focus groups with patients (n = 17). Patients reported impacts in six major areas including growing up with SCD, education, the recurrent nature of the disease, employment, relationship, and hospitalization. The authors concluded that the aspects of life affecting SCD were equivalent to the core domains of the multidimensional WHOQOL including physical, psychological, social, and occupational and well-being, as well as levels of independence and environment.

In addition to direct disease impacts, complications such as iron overload are also of particular interest, especially due to the adverse consequences of iron chelation therapy. The QoL impacts of iron overload and infusion iron chelation therapy have been investigated by Abetz and colleagues [32] based on literature review and interviews with patients and experts.

The authors found no literature focusing on the impact of iron overload or iron chelation therapy on QoL, although this was reported incidentally in a few papers. In one study, 50% of patients reported that daily activities were negatively affected (prevented) due to desferoxamine treatment. Iron chelation therapy (ICT) with desferoxamine was associated with a degree of discomfort resulting in limitations in daily life activities. Characteristics of the infusion such as higher frequency also resulted in HRQoL impairment.

Results from the interviews with patients (thalassemia, sickle cell disease, myelodysplastic syndromes) performed by Abetz et al. [32] were consistent with the current literature as reported above. HRQoL impact was particularly profound where treatment started from a young age and persisted throughout life, for example in thalassemia. In addition to effects experienced by the patients, the impacts on family/parents of pediatric patients included parental stress, related to daily inserting of a needle into their child, and strain on the relationship between parent–child (family–patient). Adolescents experienced impacts on self-image, were unable to wear certain clothes, and felt embarrassed to go out as a result of bumps/bruises on their skin. Adults reported impacts on work, sex life, and social life. The impact of ICT on QoL was noted to be greatest among adolescents and young adults. These results suggest that infusion ICT has substantial impacts across multiple QoL domains. It is still unclear whether similar impacts are experienced in patients treated with other ICT.

Lasch et al. [33] investigated how adults with chronic kidney disease on dialysis experienced the symptoms of anemia, using semi-structured interviews with 55 patients. Their data revealed 20 major themes including energy, tiredness, endurance, shortness of breath, strength/weakness, and needing more sleep. Themes associated with the impacts of anemia included need to move slower; feeling bad/better; restless sleep; and effort required for exercise. Lasch and colleagues commented that all symptoms/impacts reported by the patients could be tied to a symptom cluster related to energy. The authors further mentioned that the patients related the energy symptoms cluster to rising and falling of hematocrit levels.

Further, in addition to the qualitative evidence presented above, five studies applying quantitative methods, using generic HRQoL measures such as the SF-36, EORTC QLQ C30, FACIT-fatigue, and PedsQL Generic Core Scale, showed reduced HRQoL across multiple domains, and increased levels of fatigue.

McClish et al. [34] assessed QoL in adults with SCD (n = 308) participating in the Pain in Sickle Cell Epidemiology Study (PiSCES) in the US. SF-36 scores indicated worse HRQoL relative to US national norms on all subscales except mental health. In comparison with dialysis patients, adults with SCD scored similarly on physical role and emotional role function, social functioning and mental health, worse on bodily pain, general health and vitality, and better on physical functioning. In comparison with people with cystic fibrosis, adults with SCD showed worse SF-36 scores on all subscales except mental health.

Dampier et al. [35] investigated HRQoL in children with SCD using data from the Collaborating Project of the Comprehensive Sickle Cell Centers (CSCC) clinical trial consortium, employing the PedsQL scales (Generic Core scale/Multidimensional Fatigue scale). Their results showed a progressive decline in all parent-reported PedsQL scale scores (physical, emotional, school functioning, total HRQoL, general fatigue, sleep/rest fatigue, total fatigue) across younger to older age groups (except for cognitive fatigue). Similar decline was observed in child-reported PedsQL scale scores (physical, emotional, social, school functioning, sleep/rest fatigue, cognitive fatigue scales). Parents and children differed on the rating of social functioning and cognitive fatigue, and on the overall, PedsQL scores; child reports were higher than parent reports. In comparison with age-matched healthy children, children with SCD showed much lower PedsQL scores (lower HRQoL and higher fatigue). The authors attributed the gradual but significant decline in HRQoL to acute and chronic complications (such as vaso-occlusive pain or priapism) which increased in frequency over time.

Ameringer et al. [36] recently described fatigue and its key biological and behavioral correlates, as well as its relationship with HRQoL, in adolescents and young adults with SCD (n = 60) aged 15–30. The majority of adolescents and young adults (69%) reported feeling unusually tired or fatigued in the previous week, and scores of various fatigue scales (MFSI-SF, PROMIS Fatigue) showed mild to moderate fatigue. Levels of fatigue did not differ by disease severity or biomarkers of inflammation. The correlation of hemoglobin and fatigue depended on the scale used for measuring fatigue. Higher fatigue was correlated (r = 0.31–0.70) with increased anxiety, pain, and sleep disruptions. Fatigue scales showed a moderate correlation with all subscales of the SF-36 (r = − 0.34 to − 0.74).

In thalassemia, HRQoL impairment similar or worse than that in SCD has been reported due to need for chronic transfusion and chelation therapy. A study by Sobota et al. [37] compared HRQoL reported in the Thalassemia Clinical Research Network’s Longitudinal Cohort (TLC) (n = 264, age ≥ 14) to US norms and published literature. In comparison to the US norms, TLC patients showed worse HRQoL on five of eight SF-36 subscales (physical functioning, role-physical, general health, social functioning, and role-emotional) and on both summary scales (physical and mental components). The greatest effect was in general health and physical domains. Women, older patients, and those with more disease complications and side effects from chelation reported lower HRQoL.

Further, Sobota and colleagues [37] argued that their results were similar to results obtained from two earlier studies of Payne et al. [38, 39] which reported lower age/gender-matched scores for all SF-36 domains in patients with thalassemia in comparison with UK norms. However, Sobota et al. [37] reported contrasting results from two studies from Italy, one reporting that scores in thalassemia patients were not much different from Italian SF-36 score norms. The other study reported that patients showed lower scores only for social functioning, role-emotional and mental component summary. Sobota and colleagues [37] argued that the older age of patients in the Italian studies explains the contrasting results.

Schrezenmeier et al. [40] reported the initial findings from the international Paroxysmal Nocturnal Haemoglobinuria (PNH) registry observational study (n = 1610) to describe clinical characteristics and disease-associated comorbidities in this patient population. Half of the patients were diagnosed with bone marrow disorder including aplastic anemia, hypoplastic anemia, myelodysplastic syndromes, myelofibrosis, and/or acute myeloid leukemia. The most common symptoms included fatigue (80%), dyspnea (64%), headache (63%), hemoglobinuria (62%), abdominal pain, scleral icterus, erectile dysfunction, chest pain, confusion, and dysphagia. Scores for all EORTC QLQ C30 domains were statistically significantly lower (i.e., lower HRQoL) for patients who had reported a clinical symptom of abdominal pain, chest pain, confusion, dysphagia, dyspnea, erectile dysfunction, fatigue, headache, hemoglobinuria, or scleral icterus in the six months prior to completing the baseline questionnaire compared with patients who had not experienced the symptoms.

Physico-psychosocial model for pyruvate kinase deficiency

Based on the evidence of symptoms and impairment in PKD analogous conditions, the possible patient-reported symptoms and impacts expected in PKD were hypothesized (Fig. 2).

Fig. 2

Pyruvate kinase deficiency physico-psychosocial model

Based on the major clinical features of PKD, i.e., anemia, iron overload, acute infections, and splenomegaly, the broad areas of patient outcomes include symptoms, HRQoL impacts, treatment-related impacts, and family life impacts. The key symptoms in PKD are likely to be related to anemia, jaundice, and splenomegaly, and include fatigue, sleep disturbances, abdominal pain, chest pain, shortness of breath, and depression. The requirement for chronic treatment, e.g., chronic blood transfusion, and iron chelation, from an early age is associated with substantial treatment burden, which includes the time/effort required to manage treatments, the physical discomfort associated with treatment, and the side effects of treatment. Ultimately, disease symptoms and treatment burden result in impairment in multiple domains of HRQoL including daily activities, psychological functioning, self-image, physical functioning, and social life. The substantial impacts on HRQoL and the treatment burden, consequently, have an impact on the family as a whole (these may be exaggerated for pediatric patients but are otherwise equally important for all patients).

HRQoL measures

The most frequently used measures in PKD/PKD analogous conditions that address concepts in the hypothesized Physico-Psychosocial model outlined in Fig. 2 were searched. A total of 11 PRO measures, 1 disease-specific measure, and 1 generic family life impact measure were identified. The EQ-5D-5L was excluded as it was considered to be more appropriate for evaluating utilities for the purposes of economic evaluation, and less fitting for capturing treatment effect in trial settings, and FACIT-Fatigue excluded as a stand-alone measure, as its contents are already included as the fatigue scale in the FACT-An. Details of each measure are given in the Supplementary Data.

The PRO measures were evaluated against standards published in the FDA PRO guidance [41]. In brief, the adequacy of any PRO measure to support medical product labeling claims depends on its characteristics, conceptual, content validity, and measurement properties. A comparison of the content coverage is reported in Table 2, while an evaluation of the psychometric attributes is presented in Table 3.

Table 2

Similarities and differences in conceptual coverage of health-related quality of life instruments used in diseases analogous to PK deficiency

Health status/HRQoL domains/symptoms

WHOQOL-Bref [42]

SF-36v2 [43]

EORTC QLQ C30 [44, 45]

CHQ [46, 47]

PedsQL [46, 48]

PedsQL MFS [49, 50]

PedsFACIT-F [51]

PedsQL SCD [52]

ASCQ-Me [53]

FROM-16 [54]

Physical well-being

  

 

Psychological distress

   

 

Impact on daily activities

  

  

Impact on school/productivity

   

     

Impact on work/productivity

       

Impact on self-image

  

    

 

Impact on social life

   

 

Perceived general health/HRQoL

      

Cognitive impairment

 

     

 

Family life impacts

  

     

Fatigue

 

  

   

Shortness of breath

  

       

Pain (abdomen, chest, headache)

   

 

Sleep disturbance

  

     

 

Depression

       

In general, a concept was considered to have been addressed by an item if there was at least 1 item addressing that concept in a particular PRO measure

SF-36 short-form medical outcome, WHOQOL-BREF WHO Quality of Life-BREF, FACT-An functional assessment of cancer therapy-anemia, FROM-16 Family-Reported Outcome Measure-16, PedsQL Pediatric Quality of Life Inventory, MFS Multidimensional Fatigue Scale, pedsFACIT-F pediatric Functional Assessment of Chronic Illness Therapy-Fatigue, CHQ Children’s Health Questionnaire, ASCQ-Me adult sickle cell quality of life measurement information system, FROM family-reported outcome measure

Table 3

Evaluation of psychometric attributes of PRO measures

 

Content validity

Construct validity

Internal consistency

Test–retest

Responsiveness

MCID

Respondent burden; completion time

Language translations

SF-36

0

++

++

0

+

0

Minimal; 5–10 min

> 170 countries

WHOQOL-BREF

++

++

++

+

0

0

Minimal; < 5 min

> 23 countries (cultural adaptations)

FACT-An

++

++

++

++

++

++

Minimal; 5–15 min

> 45 languagesa

EORTC QLQ C30

++

++

++

++

++

++

Minimal 5–15 min

> 80 languages

FROM-16

++

++

++

++

++

Minimal; < 3 min

2 languages

PedsQL 4.0 Generic Core Scale

+

++

++

++

++

++

Minimal; < 4 min

> 21 languages

PedsQL 4.0 MFS

+

++

++

++

++

++

Minimal; < 4 min

> 21 languages

PedsQL SCD

++

++

++

++

++

++

Minimal; < 4 min

pedsFACIT-F

++

++

++

++

5–10 min

CHQ

0

++

++

0

0

0

5–15 min

> 72 languages

ASCQ-Me

++

++

++

~ < 10 min

SF-36 short-form medical outcome, WHOQOL-BREF WHO Quality of Life-BREF, FACT-An functional assessment of cancer therapy-anemia, EORTC QLQ C30 European Organisation for the Research and Treatment of Cancer Quality of Life Questionnaire Core 30, FROM-16 Family-Reported Outcome Measure-16, PedsQL Pediatric Quality of Life Inventory, MFS Multidimensional Fatigue Scale, SCD sickle cell disease, pedsFACIT-F pediatric Functional Assessment of Chronic Illness Therapy-Fatigue, CHQ Children’s Health Questionnaire, ASCQ-Me adult sickle cell quality of life measurement information system, ++: adequate data available from PK-d analogous population, +: data are available from PK-d analogous population, although inadequate, 0: data available in other patient populations, but not from PK-d analogous populations, −: data not reported in any patient population

aFor the FACT-G and FACT-F subscales

Appraisal of HRQoL measures

Adults

Four most relevant QoL/HRQoL measures used in patient populations analogous to PKD patients, the WHOQOL-BREF, the SF-36, the EORTC QLQ C30, and the FACT-An were compared (Table 4). Of these four, the EORTC QLQ C30 addressed all impact as well as symptom concepts except for “self-image” (Table 2). The WHOQOL-BREF, although based on a more broad definition of health, has psychometric evidence that supports using this scale for adults with chronic conditions and addresses the individualized nature of HRQOL, so is appropriate for this purpose [55]. The SF-36 and the FACT-An (particularly the FACT-G subscale) address similar impact-related concepts, except for one concept “perceived general health/HRQoL” which is assessed in the SF-36 only. The WHOQOL-BREF and the SF-36 address most domains with greater detail in comparison with the FACT-An, i.e., with more items addressing a wider range of issues. For example, whereas the FACT-An contains 2 items directly related to work, the SF-36 has 7 items/2 subscales to represent this issue. The FACT-An covers four of the five key symptom concepts, excluding “depression”; the SF-36 addresses three symptom concepts, while the WHOQOL-BREF covers two only. The “shortness of breath” and “sleep disturbances” concepts have been excluded from the WHOQOL and SF-36.

Table 4

Appraisal of health-related quality of life measures

Attribute

Advantages

Disadvantages

SF-36

• The most widely used PRO measure in general in PKD analogous conditions

• Floor/ceiling effects for some subscales (e.g., PH) in SCD

• Published dimensional structure not supported in SCD (a 3 component structure was reported in a Jamaican population)

• Content validity in SCD not demonstrated

WHOQOL-BREF

• Cross-culturally developed

• Measures relatively broader and subjective QOL domains, which may be affected by other factors unrelated to PK-d treatment

• Limited regulatory experience

FACT-An

• Extensive regulatory experience based on clinical trials of ESAs in cancer patients

• Integrates generic HRQOL aspects as well as impacts/symptoms specific to anemia

 

EORTC QLQ C30

• Has best coverage of impacts/symptom-related concepts for the psychosocial model

• Extensive regulatory experience based on clinical trials of ESAs in cancer patients

• Integrates generic HRQOL aspects relevant to anemia

 

CHQ

 

• Domain structure is different between healthy children and those with a chronic disease

• Use of multiple response scaling and recall periods has potential to be confusing

PedsQL

• Has the most extensive psychometric evidence in pediatric SCD

• Has been used to support a PRO labeling claim for Soliris in PNH in Europe

• Recall period and response scaling appropriate for pediatrics

 

PedsQL MFS

• One of the most validated and widely used pediatric fatigue measures in SCD

• Has been used to support a PRO labeling claim for Soliris in PNH in Europe

 

PedsQL SCD

• Content is underpinned by extensive qualitative research across children 5–8 years, parents, and experts in SCD

• Provides comprehensive measurement capabilities in combination with other PedsQL scales (i.e., generic, fatigue)

• As the measure is relatively new, current psychometric evidence is not extensive

• Some scales could not discriminate between mild-severe disease for child report

pedsFACIT-F

• Superior measurement attributes—its development applied modern test theory

• May permit possible comparison with scores from adult-version FACIT-F

• Some key psychometric properties have not been reported, e.g., responsiveness and test–retest reliability

• Clinical trials using the measure were scarce

• Regulatory uncertainty with use of PROs based on item banks in drug development

ASCQ-Me

• The only item bank for ASCQ-Me

• High precision/strong measurement capabilities as a result of use of modern test theory as well as classical test theory

• Regulatory uncertainty with use of PROs based on item banks in drug development

FROM-16

• Rigorous development and validation across at least ten medical specialties

• High precision/strong measurement capabilities as a result of use of modern test theory as well as classical test theory

• As the measure is quite new, use in clinical trials remains limited

Extensive psychometric evidence including construct validity, reliability, and responsiveness is available supporting the validity of the EORTC QLQ C30, SF-36v2, and FACT-An measures in PKD analogous conditions such as sickle cell and cancer-treatment-related anemia.

Pediatric/adolescents

Comparison of HRQoL measures used in pediatric patients considered the Children’s Health Questionnaire (CHQ) and the PedsQL 4.0 Generic Core Scales (Table 4). The CHQ appeared to have better coverage of impact-related concepts in comparison with the PedsQL 4.0, in particular the “impact on self-image,” “perceived general health/HRQOL,” and “family impacts.” For symptom-related concepts, the CHQ addresses one concept, while the PedsQL 4.0 Generic score includes none. Such an omission in the PedsQL Generic Core Scale may be due to the availability of a SCD-specific PedsQL module, which seemed to be the most validated SCD-specific instrument in the pediatric population (Table 4).

Although most subscales of the CHQ apply a recall period of 4 weeks, one subscale (change in health) uses a recall period of 1 year. Long recall periods are generally not recommended due to concerns about accuracy of recall and potential biases. On the other hand, as assessment of HRQoL occurs at a certain interval within a study, assessing differences between such “actual” snapshots of the patients’ conditions provides the most reliable way of assessing change over time.

For the assessment of fatigue, two measures, the PedsQL MFS and the pedsFACIT-F, were compared (Table 4). Both measures address multiple dimensions of fatigue including physical as well as psychological aspects, although the PedsQL MFS additionally includes sleep aspects. More psychometric evidence is available for the PedsQL MFS relative to the pedsFACIT-F; specifically evidence of the pedsFACIT was not available.

For family life impacts, the Family-Reported Outcome Measure (FROM-16) was the only measure considered across the pediatric, adolescent, as well as adult patient populations (Table 4). This measure has been validated for use across all disease areas, and in all patient populations. There is to date limited use of this measure in clinical trials, reflecting the novelty of the conceptualization of family life impact as a broad and generic concept. Other measures such as the Family Dermatology Life Quality Index (FDLQI) or the PedsQL Family Impact Module were not reviewed, as they were considered to be too narrowly focused on a specific disease/patient population.

Discussion

The PRO instruments potentially useful in PKD trials have been identified and their psychometric properties, namely strengths and weaknesses, were compared. On this basis, recommendations for the choice of PRO tools for use in future clinical trials are provided here. The relevance of PRO measures for use in PKD trials has been evaluated based on the understanding that clinical trials may include adult patient populations as well as pediatrics and adolescents. These conclusions therefore address the outcome measurement’s unique requirements of these groups.

Relevant patient-reported outcomes (PROs) in PKD encompass disease-specific symptoms, generalized impacts on multiple HRQoL domains and impacts on the wider family; this is the case for pediatrics, adolescents, as well as adult patient populations. Therefore, all these areas of impact should be measured in Phase II and III trials. Thus, measures addressing generic HRQoL, disease-related symptoms (particularly anemia-related symptoms such as fatigue), and family impacts should be considered.

In the adult population, the uses of EORTC QLQ C30 and SF-36v2 are recommended, the former being the basic minimum, particularly because this measure covers all key impacts as well as symptom concepts in the hypothesized physico-psychosocial model, including generic HRQoL, as well as core symptoms such as fatigue. It would be beneficial to use SF-36v2 in a study to validate results from EORTC QLQ C30, as well as to facilitate further comparative research in PKD, with other similar/related diseases. The SF-36v2 would be a better additional generic PRO measure than WHOQOL-BREF based on extensive experience with the SF-36 in clinical trials across PKD analogous conditions and a lack of such evidence with the WHOQOL-BREF.

In the pediatric population, the uses of PedsQL Generic Core Scale and the PedsQL MFS scale are recommended. The PedsQL Generic Core Scale is recommended as a measure of HRQoL, over the CHQ. The PedsQL MFS scale is recommended as a measure of fatigue over the pedsFACIT.

For pediatrics, adolescents, as well as the adult patient populations, the FROM-16 is recommended for the measurement of family burden associated with PKD. At present, this is considered to be the only generic measure of family life impact of illness appropriate for use across different patient populations. For example, the PedsQL has a family life impact module; however, this may not be of use in an adult population. Inclusion of this measure may be more appropriate for Phase III trials.

As currently no PRO measure has been validated in PKD, we recommend exploratory qualitative research to generate required content validity evidence (in a form of a pilot study) to support the above-recommended measures in PKD. Furthermore, we recommend that the blinded data from the phase II trial be utilized to assess/confirm the validity, reliability, responsiveness, and MCID of the recommended measures in PKD.

Thalassemia and paroxysmal nocturnal hemoglobinuria are clearly analogous to PKD, but sickle cell disease is a much more complicated disorder. The pathophysiology of sickle cell disease, in particular vaso-occlusive crises, is very different from what occurs in PKD. However, the selected PRO tools measure the impact of the disease and treatments from patient’s perspectives, and the analogous diseases chosen have symptoms that will produce similar patient experiences, even if the pathophysiology of the disease is somewhat different and some symptoms are different.

A major limitation to the above recommendations is that it is not possible to determine which symptoms/life impacts seen in the PKD analogous conditions may actually be observed in patients with PKD. Similarly, it is likely that some symptoms/life impacts in PKD are unique to the disease, and are not observable in analogous conditions, for example aplastic crises, osteopenia/bone fragility, extramedullary hematopoiesis, post-splenectomy sepsis, pulmonary hypertension, and leg ulcers [56]. Therefore, it would be ideal, at the minimum, to obtain the views of therapeutic experts on key symptoms and life impacts. A patient diary may then be developed within a short-time framework, to address those symptoms/life impact not covered in the current PRO measures, based on expert input. Social networking communities of patients with PKD may also be used to rapidly develop and validate such a diary.

The developed Physico-Psychosocial model for PKD may be helpful in future studies with a PRO component in patients with PKD. It may also form the basis for a hypothesized conceptual framework to be used for the development of a disease-specific PRO tool to capture/measure the QoL and symptom impact of PKD and its treatments in clinical trials.

At this stage, it is not possible to use empirical testing, e.g., structural equation modeling (SEM)/path analysis on this Physico-Psychosocial model for PKD. As there is currently no tool in this population, the data that we used came from variety of non-PKD tools for analogous diseases, applied in different settings. Therefore, this model should be a starting point to guide the next step to develop a specific Pyruvate Kinase Deficiency tool. Then, through data collection using this tool in a PKD population, it will be possible to test and either confirm or refute this Physico-Psychosocial Model.

Additionally, as the data used for this review came from a range of tools for analogous diseases, with a high level of heterogeneity among the different studies, it does not lend itself to meta-analysis and determination of effect size. Although the Pyruvate Kinase Deficiency Natural History Study (PKD NHS, https://clinicaltrials.gov, NCT02053480), a longitudinal, multicenter, international patient registry, is collecting retrospective and current clinical information and patient-reported outcome measures at enrolment and annually (expected study completion Dec 2020), there are limited data from PRO studies on PKD and quality of life. Until more data are available, effect size remains unknown.

Notes

Author contributions

MSS developed the protocol, wrote the first draft of manuscript, carried out part of the search, and designed Fig. 2. JRJ carried out the systematic literature search and appraised the PRO measures. TI helped with the appraisal of the PRO measures and reviewed the first draft of the manuscript. ENO helped with the appraisal of the PRO measures and reviewed the first draft of the manuscript.

Compliance with ethical standards

Conflict of interest

The authors declare that there are no commercial or financial conflicts of interest.

Glossary

Allosteric activator

Binding of one ligand enhances the attraction between substrate molecules and other binding sites

Aplastic crises

Temporary cessation of red cell production

Autosomal recessive trait

Ways that a trait, disorder, or disease can be passed down through families whereby two copies of an abnormal gene must be present in order for the disease or trait to develop

β-Thalassemia

A genetic blood disorder that reduces the production of hemoglobin

Chelation Therapy

A form of complementary therapy involving circulating a chelating solution in the bloodstream to bind toxins

Chronic non-spherocytic hemolytic anemia

A group of rare, genetically transmitted blood disorders characterized by the premature destruction of red blood cells

Cytopenia

Reduction in the level of red blood cells

Desferoxamine

A medication that binds iron used to treat iron overload

Extramedullary hematopoiesis

Hematopoiesis (differentiation processes that lead to the formation of all blood cells from hematopoietic stem cells) occurring in organs outside of the bone marrow

Hematopoietic stem cell transplantation

Is the transplantation of multipotent hematopoietic stem cells

Hemolytic anemia

A form of anemia due to hemolysis, the abnormal breakdown of red blood cells

Myelodysplastic syndromes

A group of cancers in which immature blood cells in the bone marrow do not mature and therefore do not become healthy blood cells

Neonatal hemolysis

Hemolytic disease of the newborn

Neonatal jaundice

A yellowish discoloration of the white part of the eyes and skin in a newborn baby due to high bilirubin levels

Paroxysmal nocturnal hemoglobinuria

A rare blood condition where blood cells are prone to be attacked by part of the body’s immune system

Pathophysiology

Functional changes associated with or resulting from disease or injury

Post-splenectomy sepsis

Body’s response to infection causing injury to its own tissues and organs following removal of the spleen

Pulmonary hypertension

Increased blood pressure within the arteries of the lungs

Pyruvate kinase

The enzyme that catalyzes the final step of glycolysis

Structural equation modeling

A diverse set of mathematical models, computer algorithms, and statistical methods that fit networks of constructs to data and includes confirmatory factor analysis, path analysis, and partial least squares

Symptomatic hypersplenism

Exhibiting an overactive spleen, removing too many blood cells, including healthy ones

Path analysis

An extension of multiple regression used to provide estimates of the magnitude and significance of hypothesized causal connections between sets of variables and includes multiple regression analysis, factor analysis, and discriminant analysis

Vaso-occlusive crises

A common painful complication of sickle cell anemia where circulation of blood vessels is obstructed by sickled red blood cells, causing ischemic injuries

Supplementary material

11136_2018_2025_MOESM1_ESM.pdf (282 kb)
Supplementary material 1 (PDF 282 KB)

References

  1. 1.
    Grace, R., Zanella, A., Neufeld, E., et al. (2015). Erythrocyte pyruvate kinase deficiency: 2015 status report. American Journal of Hematology, 90(9), 825–830.Google Scholar
  2. 2.
    Beutler, E., & Gelbart, T. (2000). Estimating the prevalence of pyruvate kinase deficiency from the gene frequency in the general white population. Blood, 95(11), 3585–3588.Google Scholar
  3. 3.
    Zanella, A., Bianchi, P., & Fermo, E. (2006). Red cell enzyme deficiencies: Molecular and clinical aspects. Haematologica Reports, 2(10), 96–102.Google Scholar
  4. 4.
    Grace, R., Barcellini, W., Eber, S., et al. (2015). Categorization of clinical severity in pyruvate kinase deficiency (PKD) in an international, observational cohort. In Paper presented at presented at the 20th congress of the European Hematology Association; 11–14 June 2015, Vienna.Google Scholar
  5. 5.
    Gregg, X., & Prchal, J. (2016). Red blood cell enzymopathies. In R. B. E. Hoffman (Ed.), Hematology: Basic principles and practice (7th ed.). London: Elsevier.Google Scholar
  6. 6.
    van Wijk, R. (2015). Erythrocyte enzyme disorders. In K. Kaushansky, M. Lichtman, J. Prchal, et al. (Eds.), Williams hematology (9th ed., p. 689). New York: McGraw-Hill.Google Scholar
  7. 7.
    Zanella, A., & Bianchi, P. (2000). Red cell pyruvate kinase deficiency: From genetics to clinical manifestations. Best Practice & Research: Clinical Haematology, 13(1), 57–81.Google Scholar
  8. 8.
    Pérez-Albert, P., Guillén, M., Prudencio, M., & Sevilla, J. (2017). Haematopoietic stem cell transplantation in pyruvate kinase deficiency: When is it indicated? Anales de Pediatría, 88(2), 106–107.Google Scholar
  9. 9.
    Morton, D., Knoll, C., Rothman, J., et al. (2015). The clinical features and treatment of iron overload in pyruvate kinase deficiency (PKD): Data from the PKD Natural History Study (NHS). In Paper presented at the 20th congress of the European Hematology Association; 11–14 June 2015, Vienna.Google Scholar
  10. 10.
    Kung, C., Hixon, J., Kosinski, P., et al. (2017). AG-348 enhances pyruvate kinase activity in red blood cells from patients with pyruvate kinase deficiency. Blood, 130(11), 1347–1356.Google Scholar
  11. 11.
    Garate, Z., Quintana-Bustamante, O., & Crane, A. (2015). Generation of a high number of healthy erythroid cells from gene-edited pyruvate kinase deficiency patient-specific induced pluripotent stem cells. Stem Cell Reports, 5(6), 1053–1066.Google Scholar
  12. 12.
    Garcia-Gomez, M., Calabria, A., & Garcia-Bravo, M. (2016). Safe and efficient gene therapy for pyruvate kinase deficiency. Molecular Therapy, 24(7), 1187–1198.Google Scholar
  13. 13.
    Meza, N., Alonso, M., Navarro, S., et al. (2007). Development of efficient gene therapy for the treatment of erythrocyte pyruvate kinase deficiency. Blood, 110(11), 2584.Google Scholar
  14. 14.
    Meza, N., Alonso-Ferrero, M., Navarro, S., et al. (2009). Rescue of pyruvate kinase deficiency in mice by gene therapy using the human isoenzyme. Molecular Therapy, 17(12), 2000–2009.Google Scholar
  15. 15.
    Cavazzana-Calvo, M., Payen, E., Negre, O., Wang, G., Hehir, K., & Fusil, F. (2011). Transfusion independence and HMGA2 activation after gene therapy of human β-thalassaemia. Nature, 467, 318–322.Google Scholar
  16. 16.
    Fucharoen, S., Ketvichit, P., Pootrakul, P., Siritanaratkul, N., Piankijagum, A., & Wasi, P. (2000). Clinical manifestation of beta-thalassemia/hemoglobin E disease. Journal of Pediatric Hematology/Oncology, 22(6), 552–557.Google Scholar
  17. 17.
    Belfield, K., & Tichy, E. (2018). Review and drug therapy implications of glucose-6-phosphate dehydrogenase deficiency. American Journal of Health-System Pharmacy, 75(3), 97–104.Google Scholar
  18. 18.
    Grace, R., Cohen, J., Egan, S., et al. (2018). The burden of disease in pyruvate kinase deficiency: Patients’ perception of the impact on health-related quality of life. European Journal of Haematology.Google Scholar
  19. 19.
    Worsham, C., Martin, S., Nouraie, S., Cohen, R., & Klings, E. (2017). Clinical and laboratory findings associated with sleep disordered breathing in sickle cell disease. Annals of Hematology, 92(12), E649–E651.Google Scholar
  20. 20.
    Raghunathan, V., Whitesell, P., & Lim, S. (2018). Sleep-disordered breathing in patients with sickle cell disease. Annals of Hematology, 97(5), 755–762.Google Scholar
  21. 21.
    Cappellini, M., Musallam, K., & Taher, A. (2009). Insight onto the pathophysiology and clinical complications of thalassemia intermedia. Hemoglobin, 33(Suppl 1), S145–S159.Google Scholar
  22. 22.
    Ballas, S., Zeidan, A., Duong, V., DeVeaux, M., & Heeney, M. (2018). The effect of iron chelation therapy on overall survival in sickle cell disease and β-thalassemia: A systematic review. American Journal of Hematology, 93(7), 943–952.Google Scholar
  23. 23.
    Hilgard, P., & Gerken, G. (2005). Liver cirrhosis as a consequence of iron overload caused by hereditary nonspherocytic hemolytic anemia. World Journal of Gastroenterology, 28(11), 1241–1244.Google Scholar
  24. 24.
    Rossi, F., Perrotta, S., Bellini, G., et al. (2014). Iron overload causes osteoporosis in thalassemia major patients through interaction with transient receptor potential vanilloid type 1 (TRPV1) channels. Haematologica, 99(12), 1876–1884.Google Scholar
  25. 25.
    Wang, Y., Liu, W., Masuyama, R., et al. (2012). Pyruvate dehydrogenase kinase 4 induces bone loss at unloading by promoting osteoclastogenesis. Bone, 50(1), 409–419.Google Scholar
  26. 26.
    Matta, B., As, O., Maakaron, J., Koussa, S., Daderian, R., & Taher, A. (2014). Leg ulcers in patients with β-thalassaemia intermedia: A single centre’s experience. Journal of the European Academy of Dermatology and Venereology, 28(9), 1245–1250.Google Scholar
  27. 27.
    Vogiatzi, M., Macklin, E., & Fung, E. (2009). et a. Bone disease in thalassemia: A frequent and still unresolved problem. Journal of Bone and Mineral Research, 24(3), 543–557.Google Scholar
  28. 28.
    Martí-Carvajal, A., Solà, I., & Agreda-Pérez, L. (2016). Treatment for avascular necrosis of bone in people with sickle cell disease. The Cochrane Database of Systematic Reviews, 8, CD004344.Google Scholar
  29. 29.
    Grace, R. (2015). Pyruvate Kinase Deficiency Natural History Study (PKD NHS) (NCT02053480). Retrieved January 8, 2018, from https://clinicaltrials.gov/ct2/show/record/NCT02053480.
  30. 30.
    Stegenga, K., Ward-Smith, P., Hinds, P., Routhieaux, J., & Woods, G. (2004). Quality of life among children with sickle cell disease receiving chronic transfusion therapy. Journal of Pediatric Oncology Nursing, 21(4), 207–213.Google Scholar
  31. 31.
    Thomas, V., & Taylor, L. (2002). The psychosocial experience of people with sickle cell disease and its impact on quality of life: Qualitative findings from focus groups. British Journal of Health Psychology, 7(3), 345–363.Google Scholar
  32. 32.
    Abetz, L., Baladi, J., Jones, P., & Rofail, D. (2006). The impact of iron overload and its treatment on quality of life: Results from a literature review. Health and Quality of Life Outcomes, 4, 73–78.Google Scholar
  33. 33.
    Lasch, K., Evans, C., & Schatell, D. (2009). A qualitative analysis of patient-reported symptoms of anemia. Nephrology Nursing Journal, 36(6), 621–624.Google Scholar
  34. 34.
    McClish, D., Penberthy, L., Bovbjerg, V., et al. (2005). Health related quality of life in sickle cell patients: The PiSCES project. Health and Quality of Life Outcomes, 29(3), 50–56.Google Scholar
  35. 35.
    Dampier, C., Lieff, S., LeBeau, P., et al. (2010). Health-related quality of life in children with sickle cell disease: A report from the Comprehensive Sickle Cell Centers Clinical Trial Consortium. Pediatric Blood & Cancer, 55(3), 485–494.Google Scholar
  36. 36.
    Ameringer, S., Elswick, R., & Smith, W. (2014). Fatigue in adolescents and young adults with sickle cell disease: Biological and behavioral correlates and health-related quality of life. Journal of Pediatric Oncology Nursing, 31(1), 6–17.Google Scholar
  37. 37.
    Sobota, A., Yamashita, R., Xu, Y., et al. (2011). Quality of life in thalassemia: A comparison of SF-36 results from the thalassemia longitudinal cohort to reported literature and the US norms. American Journal Of Hematology, 86(1), 92–95.Google Scholar
  38. 38.
    Payne, K., Desrosiers, M., Caro, J., et al. (2007). Clinical and economic burden of infused iron chelation therapy in the United States. Transfusion, 47(10), 1820–1829.Google Scholar
  39. 39.
    Payne, K., Rofail, D., Baladi, J., et al. (2008). Iron chelation therapy: Clinical effectiveness, economic burden and quality of life in patients with iron overload. Advances in Therapy, 25(8), 725–742.Google Scholar
  40. 40.
    Schrezenmeier, H., Muus, P., Socié, G., et al. (2014). Baseline characteristics and disease burden in patients in the International Paroxysmal Nocturnal Hemoglobinuria Registry. Haematologica, 99(5), 922–929.Google Scholar
  41. 41.
    USFDA. (2009). US food and drug administration (FDA) guidance for industry -patient-reported outcome measures: Use in medical product development to support labeling claims. Maryland: USFDA.Google Scholar
  42. 42.
    Skevington, S., Lotfy, M., & O’Connell, K. (2004). The World Health Organization’s WHOQOL-BREF quality of life assessment: Psychometric properties and results of the international field trial. A report from the WHOQOL group. Quality of Life Research, 13(2), 299–310.Google Scholar
  43. 43.
    McDowell, I. (2006). Measuring health: A guide to rating scales and questionnaires (3rd ed.). Oxford: Oxford University Press.Google Scholar
  44. 44.
    Aaronson, N., Ahmedzai, S., Bergman, B., et al. (1993). The European Organisation for Research and Treatment of Cancer QLQ-C30: A quality-of-life instrument for use in international clinical trials in oncology. Journal of the National Cancer Institute, 85, 365–376.Google Scholar
  45. 45.
    Luckett, T., King, M., Butow, P., et al. (2011). Choosing between the EORTC QLQ-C30 and FACT-G for measuring health-related quality of life in cancer clinical research: Issues, evidence and recommendations. Annals of Oncology, 22(10), 2179–2190.Google Scholar
  46. 46.
    Hullmann, S., Ryan, J., Ramsey, R., Chaney, J., Mullin, & sL (2011). Measures of general pediatric quality of life: Child Health Questionnaire (CHQ), DISABKIDS Chronic Generic Measure (DCGM), KINDL-R, Pediatric Quality of My Life Questionnaire (QoML). Arthritis Care & Research (Hoboken), 63(Supp11), S420–S430.Google Scholar
  47. 47.
    Panepinto, J., O’Mahar, K., DeBaun, M., Rennie, K., & Scott, J. (2004). Validity of the child health questionnaire for use in children with sickle cell disease. Journal of Pediatric Hematology, 26(9), 574–578.Google Scholar
  48. 48.
    Panepinto, J., Pajewski, N., Foerster, L., & Hoffmann, R. (2008). The performance of the PedsQL generic core scales in children with sickle cell disease. Journal of Pediatric Hematology, 30(9), 666–673.Google Scholar
  49. 49.
    Panepinto, J., Torres, S., Bendo, C., et al. (2014). PedsQLTM Multidimensional Fatigue Scale in sickle cell disease: Feasibility, reliability, and validity. Pediatric Blood & Cancer, 61(1), 171–177.Google Scholar
  50. 50.
    Tomlinson, D., Hinds, P., Ethier, M., Ness, K., Zupanec, S., & Sung, L. (2013). Psychometric properties of instruments used to measure fatigue in children and adolescents with cancer: A systematic review. Journal of Pain and Symptom Management, 45(1), 83–91.Google Scholar
  51. 51.
    Lai, J., Cella, D., Kupst, M., et al. (2007). Measuring fatigue for children with cancer: Development and validation of the pediatric Functional Assessment of Chronic Illness Therapy-Fatigue (pedsFACIT-F). Journal of Pediatric Hematology, 29(7), 471–479.Google Scholar
  52. 52.
    Panepinto, J., Torres, S., & Varni, J. (2012). Development of the PedsQLTM Sickle Cell Disease Module items: Qualitative methods. Quality of Life Research, 21(2), 341–357.Google Scholar
  53. 53.
    Keller, S., Yang, M., Treadwell, M., Werner, E., & Hassell, K. (2014). Patient reports of health outcome for adults living with sickle cell disease: Development and testing of the ASCQ-Me item banks. Health and Quality of Life Outcomes, 12, 125–135.Google Scholar
  54. 54.
    Golics, C., Basra, M., Finlay, A., & Salek, S. (2013). The development and validation of the Family Reported Outcome Measure (FROM-16)(©) to assess the impact of disease on the partner or family member. Quality of Life Research, 23(1), 317–326.Google Scholar
  55. 55.
    Hand, C. (2016). Measuring health-related quality of life in adults with chronic conditions in primary care settings. Critical review of concepts and 3 tools. Canadian Family Physician, 62(7), e375–e383.Google Scholar
  56. 56.
    Grace, R., Bianchi, P., van Beers, E., et al. (2018). Clinical spectrum of pyruvate kinase deficiency: Data from the Pyruvate Kinase Deficiency Natural History Study. Blood, 131(20), 2183–2192.Google Scholar

Copyright information

© The Author(s) 2018

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.School of Life and Medical SciencesUniversity of HertfordshireHatfieldUK
  2. 2.University Clinic St. Petersburg State University and Multinational Centre for Quality of Life ResearchSt. PetersburgRussia
  3. 3.Institute for Medicines DevelopmentCardiffUK
  4. 4.Haematology UnitGrande Ospedale Metropolitano Bianchi Melacrino MorelliReggio CalabriaItaly

Personalised recommendations