Developing and pretesting a new patient reported outcome measure for paediatric Chronic Fatigue Syndrome/ Myalgic Encephalopathy (CFS/ME): cognitive interviews with children
There is a lack of patient derived, child specific outcome measures to capture what health outcomes are important to children with Chronic Fatigue Syndrome/ Myalgic Encephalopathy (CFS/ME). We developed a new Patient Reported Outcome Measure (PROM) for paediatric CFS/ME through qualitative research with children. This study aimed to pre-test the new measure through cognitive interviews with children with CFS/ME.
Cognitive interviews were undertaken in children’s homes or over Skype. The Three-Step Test-Interview (TSTI) method was used to assess the quality of the draft PROM with children with CFS/ME to identify problems with initial content and design and test modifications over subsequent interview rounds. Children were purposively sampled from a single specialist paediatric CFS/ME service in England.
Twenty-four children and their parents took part. They felt the new measure captured issues relevant to their condition and preferred it to the generic measures they completed in clinical assessment. Changes were made to item content and phrasing, timeframe and response options and tested through three rounds of interviews.
Cognitive interviews identified problems with the draft PROM, enabling us to make changes and then confirm acceptability in children aged 11–18. Further cognitive interviews are required with children 8–10 years old to examine the acceptability and content validity and provide evidence for age related cut offs of the new PROM to meet FDA standards. This study demonstrates the content validity of the new measure as relevant and acceptable for children with CFS/ME. The next stage is to undertake a psychometric evaluation to support the reduction of items, confirm the structure of the PROM and provide evidence of the data quality, reliability and validity.
KeywordsChronic fatigue syndrome/ Myalgic encephalopathy (CFS/ME) Children Patient reported outcome measure (PROM) Qualitative Cognitive interviews Three step test interview (TSTI)
Chronic Fatigue Syndrome
Health-related quality of life
Patient-reported outcome measure
Young persons advisory group
Health Related Quality of Life (HRQoL) in children with Chronic Fatigue Syndrome/Myalgic Encephalopathy (CFS/ME) is lower than children with asthma, type 1 diabetes mellitus or junior idiopathic arthritis [1, 2, 3]. Children with CFS/ME experience a range of physical and cognitive symptoms alongside extreme fatigue [4, 5, 6] that impact on their physical function , social functioning at school and with friends [3, 8, 9] as well as psychological wellbeing, increasing depression and anxiety [10, 11, 12]. Measuring HRQoL through Patient Reported Outcome Measures (PROMs) is important to describe the impact of an illness on a patient’s daily functioning. PROMs provide evidence about the benefits of treatment in clinical and research settings and improve clinical decision making [13, 14, 15, 16, 17].
The first key stage in PROM development [17, 18, 19] is the content validity phase, during which qualitative methods are used to provide evidence that domains measured in the final instrument are important to patients [17, 19, 20]. Traditionally, child-specific PROMs have failed to incorporate children’s perspectives, instead relying on input from health professionals or parents [21, 22, 23]. The resultant measures may therefore miss outcomes that are important to children, risking content validity and measure responsiveness [24, 25]. The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) taskforce advocates children as “effective content experts” . There is currently no validated paediatric CFS/ME-specific PROM  informed by children’s views.
We sought to address this gap by developing a PROM through extensive qualitative work with 46 children with CFS/ME [28, 29] and specialist paediatric CFS/ME clinicians . A child-specific conceptual framework of HRQoL was developed and provided a basis for the operationalisation of questionnaire items based on the most important outcomes to children with CFS/ME (reference currently under review).
A second key stage in PROM development is pretesting a new measure to ensure the final PROM is acceptable and easily understood by patients [19, 20, 31]. Guidance recommends the use of cognitive interviews to ensure “vocabulary level, item content, recall period, response options, instructions and comprehensiveness are appropriate for the target age group” . Children may not interpret items the way adults intend  and there is a need to examine if a child can read, comprehend and respond to questionnaire items [26, 33]. This is particularly important for children with CFS/ME who often experience cognitive impairment [34, 35, 36, 37]. Cognitive interviewing focuses on the cognitive processes that respondents use to answer questions , with the aim of improving the design of instruments to avoid respondent misunderstanding, minimize errors and reduce missing data . The aim of the current study was to pre-test the new CFS/ME-specific PROM for children using cognitive interview methods.
Development of the new Paediatric CFS/ME PROM
The questionnaire was developed through a systematic approach. A conceptual framework of ‘living with paediatric CFS/ME’ developed from qualitative research (reference currently under review) formed the basis of the new measure with quotes taken directly from children used to construct age appropriate items for each domain of the framework . Draft items (n = 95) were then reviewed by 22 paediatric CFS/ME health professionals from the U. K and the Netherlands in a day-long meeting. Health professionals were asked to review the items for content and wording, consider the comprehensiveness of domains and any missing outcomes, and ensure that items were clinically relevant and measured the range from mild to moderate severity. Healthy children as well as children with CFS/ME were consulted on their views on the structure and formatting of existing generic child PROMs (e.g. question tense, recall period, response options) with the new measure designed around their preferences.
A final 67 items were selected to form the basis of a draft questionnaire ready for pre-testing in cognitive interviews. The preliminary paediatric CFS/ME PROM consisted of 67 items grouped into 4 domains (symptoms, physical, social & psychological) and 11 subdomains: sleep (1 item), tiredness/fatigue (9 items), cognitive difficulties (4 items), individual symptoms (7 items), fluctuation and payback (3 items), physical function (daily activities and mobility) (11 items), participation in school life (3 items) participation in social life (7 items), mood (10 items), anxiety (7 items) and self-esteem (5 items). Items on physical function and participation were developed to capture the range of severity described by children from mild (problems with outdoor, sustained activities and sports but attending full time school) to moderate (significant physical disability, problems washing, can only do indoor activities and attending part time school) to optimise the relevance and coverage of the PROM [19, 39]. Negatively phrased items (“I had problems remembering things”) were used to assess ill-being (problems/symptoms)  whereas items on social participant and self-esteem were positively phrased (“I felt good about myself”) to balance the questionnaire . The questionnaire was developed as a self-report instrument for children 8–18 years old to report how they have been feeling ‘over the past week’ using five point Likert response scales that children have found easier to complete [26, 42] targeting: severity (Not at all- Very much), frequency (Never-Always) and interference (With no difficulty- Not able to do).
Pretesting the new Paediatric CFS/ME PROM (cognitive interviews)
Three rounds of cognitive interviews were undertaken with changes made to the questionnaire after each round and tested in an iterative process with different children in subsequent rounds until saturation (Round 1 n = 10. Round 2 n = 9; Round 3 n = 5). Changes made to the questionnaire in round 1 were tested in round 2 and then a further 5 children were recruited for a final round to check the final changes made in round 1 and 2 were acceptable. No significant findings emerged in round 3. The Three-Step Test-Interview (TSTI) method was used to assess the quality of the new self-complete questionnaire . The TSTI combines think aloud and verbal probing/debriefing in a sequence of stages: 1) Think aloud (verbalizing thoughts whilst completing a measure) by participants and observation of response behaviour by the interviewer to collect primary data on any problems with the measure (e.g. skipping questions; hesitation, changing response options); 2) follow-up probing by the interviewer to explore any observations e.g. “Did I hear you say.. .? ”; and 3) debriefing by the interviewer aimed at eliciting overall experiences and opinions of participants.
Children aged 8–18 years, diagnosed with mild to moderate CFS/ME (not housebound) , were recruited from a specialist paediatric chronic fatigue service in South West England. Sampling of participants was purposeful  and guided by participant characteristics (age, gender, and disease severity). Further children were recruited in subsequent rounds to check changes made to the questionnaire based on earlier findings were appropriate and understandable.
Cognitive interviews were undertaken face to face in participants’ own homes or over Skype. While we aimed to interview children alone, parents were given the option to remain. If parents helped during the interviews, how they helped was noted. Interviews were audio-recorded using an encrypted digital recorder and written notes were made.
Step1: Participants completed the PROM as they would do normally, whilst encouraged to verbalize what they were thinking and why they selected their responses. Children were given an example of think aloud at the start to help demonstrate what was required. The interviewer intervened as little as possible and recorded the processes that participants used in arriving at an answer as well as any difficulties (reading difficulty, pausing, flipping pages).
Step 2: The interviewer followed up on observations they noted down that were unclear using spontaneous probes (e.g. “Did I hear you say?”).
Extract of cognitive probes from topic guide
Cognitive/ Questionnaire Component
Follow up on observations
Why did you pause on this question?
What does [item content] mean to you?
What did you remember when you read this question?
Describe your experiences with [concept] over the (timeframe).
How did you select your [response option?]
Are there things that we forgot to ask about that you think are important?
What do you think about the questionnaire?
Audio recordings were reviewed in detail by RP, alongside the written notes, and data inserted on a structured excel form that had rows for each questionnaire item and individual participants and columns for participant responses to each element of the questionnaire (e.g. item wording, timeframe, response options etc. …). This ensured that for each questionnaire item, we recorded information from observation of questionnaire completion, participants’ feedback on item meaning, and any difficulties and suggestions for change. For each participant, we also noted feedback on the appropriateness of the timeframe and response options, if anything was missing and general impressions of the questionnaire. A summary of each item was generated including recommended changes. Quotes were transcribed verbatim to illustrate findings or reasons for change.
Reasons for changing items during pre-testing (Adapted from )
Reasons for change or deletion
Clarity of relevance
• Not relevant to a large number of the participants
• Large amounts of requests for clarification from participants
• Participants interpret items in a different way than intended by the conceptual framework
• High number of participants response at extreme ends of the scale
• Participants feedback that the none of the response options apply to them
• All participants give the same answer
• Participants do not give different responses even when important differences are known
• Item duplicates information collected from other items
Participants taking part in cognitive interviews
Age at interview
Interviewed alone/parent present
Interviewed at home or Skype
Sub domains covered
Overall, all children preferred the new PROM to current generic measures and felt that it captured issues that are relevant and not just ‘fatigue’, “It’s a lot more relevant than the other ones [generic] were” (CI22, female, aged 14). Children liked the format with instructions in orange and alternating shades as well as the different sections covering the various aspects of health; “One of the easiest ones I’ve ever seen. Broken down which is nice.” (CI2, female, aged 17). Younger children reported more problems with specific words or longer items. Problems were identified in the following areas: content and item phrasing, timeframe and response options. Changes were made and tested in subsequent interviews (Additional file 1).
Content and item phrasing
Items about payback (i.e. tired after activity) resulted in a ceiling effect as use of the term ‘high energy activities’ resulted in all children selecting that they would always get payback. In the final round, ‘active day’ was used and this worked well to capture the payback that can be caused when children have a busy day or usually a day out:
"You might be physically there but mentally you may not actually be there … you are not always able to fully concentrate … whether you contribute to the lesson" (CI8, male, aged 16)
“I went swimming on the weekend at my Dads house for 45 minutes and after I felt really bad “(CI23, female, aged 11)
Minor revisions were made to the ‘participation’ and ‘emotional wellbeing’ domains. Examples were modified to make them more relevant (e.g. ‘board games’ replaced with ‘playing and going on the computer’). ‘Sad’ was amalgamated with ‘feeling down’. The item “I felt like I’m being left behind” was felt to be ambiguous thus was amended to: “I worried about being able to do what other people my age can do”. In subsequent rounds, it demonstrated its applicability across the age range as younger children referred to ‘not being able to do physical activities with friends’ and older children thought about life stages such as university.
Some children felt the examples given of social activities were not applicable to them, therefore, the content of the school and social participation domains were adjusted and different versions developed for 8–15 & 16–18 year olds and tested in later interviews. For example, ‘leisure activities’ was used for 16–18 years olds rather than ‘afterschool clubs’ for 8–15 year olds. Items were also edited to refer to “school, college or work/apprenticeship” for 16–18 year olds.
“Maybe two weeks, people can remember that and it gives you the longer timespan to remember things and give a lot more solid answers”. (CI2, female, aged 17)
“Int: So if I asked about 2 weeks do you think that would be better. Do you think you would remember?
Int: What if it asked about how you are at the moment?
CI6: you would think how you were that day to be honest. So you might have a good day and you might have a bad day." (CI6, female, aged 12)
“I can do it. I would struggle, I usually push through things” (CI6, female, aged 12).
“I don’t see it as ‘being able to do it’ as such, if I was able to do it I’d do it and not crash and burn, that’s the difficult thing about answering these, I could probably play a board game but the effect would be a few days afterwards.” (CI2, female, aged 17)
Parents of younger children also commented that their child could do almost anything but would require breaks and pacing. Comments- this appears bold on the printed PDF. Please unbold.
"I love football and cricket and swimming and I won’t be able to do that for as long as I want" (CI23, female, aged 11)
The ‘frequency’ response options were changed to ‘interference’ (with no difficulty- with a lot of difficulty) options for the participant domain and these worked well in the last round of cognitive interviewing:
"CIP7: He can do anything he wants to do but it’s the amount he has to restrict. Anything is possible, but it depends on the chunks, he can do it but he can only do a few minutes a time" (CIP7, mother of male aged 11)
"I think these are quite good [difficulty response options] because they have the word 'difficulty' in, so it’s like 'how difficult do you find this to do', not just 'can you do this or not'" (CI14, female, aged 17)
"I: When you think about 'difficulty' what are you thinking about?
CI6: Just struggling or thinking 'I'm shattered' while doing it". (CI6, female, aged 12)
School response options: days changed to hours
The response options were edited to hours with prompts. This worked well across the older and younger age groups in round 2 and 3:
"By full time you mean 5 days but for me full time is 6 hours each day". (CI5, female, aged 16)
Cognitive testing and review by the YPAG resulted in the final measure ready for psychometric testing: 63 items grouped into 4 domains, 11 subdomains: sleep (1 item), tiredness/fatigue (6 items), cognitive difficulties (4 items), individual symptoms (8 items), fluctuation and payback (3 items), physical function (daily activities and mobility) (8 items), participation in school life (6 items) participation in social life (7 items), mood (7 items), anxiety (8 items) and self-esteem (5 items). It was developed as a self-report instrument with patients reporting how they have been feeling ‘over the past two weeks’ on 5 point response scales: severity (Not at all- Very much), frequency (Never-Always) and interference (With no difficulty- Not able to do).
"I like how you've put 'one day a week', because it doesn’t make sense if you just put 1-7 hours" (CI22, female, aged 14)
The cognitive interviews were valuable to identify problems with the initial content and design of the draft PROM including: 1) repetitive items for tiredness and missing items on school participation, 2) items on payback requiring rewording and age appropriate examples required for participation items, 3) unsuitable timeframe with most children preferring 2 weeks and 4) the value of the interference response options. Rounds of interviews enabled us to make these changes and confirm the acceptability of the questionnaire in children aged 11–18.
Strengths and weaknesses
Children’s (n = 46) verbatim quotes were used to craft items, grounding the new PROM on children’s expressed concerns and preserving children’s speech for strong internal and external validity . To ensure optimal content validity, the perspectives of children and health professionals were used to generate, format and refine the draft PROM, ensuring items were included that mattered to patients but were also important clinically . A range of paediatric CFS/ME health professionals were recruited from around the U.K as well as from the Netherlands to review the draft PROM. This increases the applicability of the new PROM and ensures that perspectives from different specialist paediatric CFS/ME services were included.
The cognitive interviews have been an extremely valuable step in developing our new PROM and optimising the sensitivity of the instrument. For example, as a result of feedback from children, the response options in the participation domain were changed to interference (with no difficulty- not able to do) to capture that children with CFS/ME can often participate in ‘normal’ activities but they will then experience payback or increase in their symptoms and difficulty as a result of taking part. We believe this is one of the first studies to use the Three Step Test Interview (TSTI) in children. There are studies available in the literature that have used the TSTI for adult PROMs [50, 51, 52, 53]. The TSTI includes the traditional methods of think aloud and verbal probing in sequential steps and this worked well as if a child was capable of ‘think aloud’ less verbal probing was required and vice versa. The cognitive debriefing/probing method has traditionally been used in studies developing new PROMs in children: DISABKIDS , Haemo-QoL , EQ. 5D-Y  and PedsQL disease specific modules  as well as the ‘think aloud’ technique [33, 58, 59, 60, 61]. Some younger children found both think aloud and debriefing methods difficult. In these cases, asking them to look at the questions and physically mark with a pen/pencil the words or items they found difficult worked more effectively. Two age appropriate topic guides may be needed to draw on the different cognitive interview methods for different ages. Further empirical evidence is required to explore the TSTI in children.
The majority of children (n = 19) reviewed the whole questionnaire, however, four got tired or lost concentration. The interviews took an average 43 min and we feel this is a much longer process, than asking a child to complete a questionnaire normally without incorporating think aloud and probing. Therefore, future research should divide long questionnaires up between participants and recruit a larger sample size to ensure the interviews are more manageable for children, particularly those with cognitive impairment. We ended interviews after 24 participants had been recruited as no new significant findings were emerging, fulfilling the 10–15 interviews per item recommended sample size by some researchers .
The ISPOR task force recommend an adequate sample size “at the upper and lower bounds of the target age range” . We had problems recruiting younger children (< 11 years old) as fewer younger children attend the specialist paediatric CFS/ME clinic. CFS/ME is more common in adolescents and girls [62, 63, 64, 65, 66, 67, 68, 69, 70] and therefore, there is good evidence for content validity of the new PROM in children who more commonly get CFS/ME. However, ensuring that the new PROM is age appropriate across the 8–18 age range was a key challenge in this study. The cognitive abilities of an 8 year old can differ substantially from a 10 year old . Thus, further cognitive interviews with younger children (< 11 years old) are required to provide robust evidence for age related cut offs of the new PROM to meet FDA standards [17, 26]. Following this, the next stage of psychometric testing should ensure children younger than 11 years old are sufficiently represented. The literacy level of participants was not recorded and this may have been useful ensure the items were understandable to a wide range of respondents [19, 72].
Results in context with previous literature
The results from this study are consistent with other cognitive interview studies for children’s PROM development. We found children had difficulty understanding ambiguous terms such as “activities”. In cognitive interviews involving 77 children (8–17 years of age) to edit items of the PROMIS paediatric item bank , problematic terms included ‘social activities’. Our study revealed that children with CFS/ME found interference response options (With no difficulty- Not able to do) particularly helpful to describe how they are able to do an activity but this may be ‘difficult’ and result in payback (e.g. tired after activity) which is a key feature of CFS/ME. These response options may be applicable in other fatigue measures.
As a result of the cognitive interviews, the recall period of the new measure was changed from 1 week to 2 weeks. Children 8 years and older are thought to be able to recall a 4 week period . This is consistent with well-known child-report measures such as the Child Health Illness Profile , Child Health Questionnaire  and the PedsQL  with a recall period of the past month and are widely used in clinical trials . Shorter recall periods may fail to capture symptoms or events that occur outside the specified period or may misrepresent a particular experience . This was clear in the cognitive interviews as CFS/ME fluctuates and children felt a shorter period did not represent how they were generally feeling. Interventions in CFS/ME usually occur over months [79, 80, 81] and children are expected to make small changes gradually. Whilst fluctuation is important (and captured in items within the PROM), health professionals are interested in sustained change (either worse or better) rather than the experience of the last few days and therefore it is important to capture long term changes to patient status rather than short term fluctuation. Although the FDA do not specify an optimal recall period, shorter recall periods are preferred in children . Therefore, this 2-week recall may be problematic and requires further consideration in the future development of the measure.
We developed two different versions of the questionnaire for 8–15 & 16–18 year olds. School or college attendance (including missing lessons, taking breaks and problems concentrating) is one of the most important outcomes for children with CFS/ME (reference currently under review). Minor modifications were made to the school and social participation domains based on feedback from the cognitive interviews with differences between those in school (8–15) versus those more likely to be in college or work (16–18). Further work needs to be done with younger children (< 11 years old) to decide if a further version is needed.. This is consistent with the well-known generic child measure PedsQL, which has the same overall domains but different formatting for the various age groups 5–7, 8–12, and 13–18 years . A recent systematic review of generic multi-dimensional PROMS used for children up to age 18  demonstrated that the majority (29 out of 35) of child PROMs have the same version across the age range [84, 85]. However, some condition specific child measures include multiple versions: the Childhood Asthma Questionnaire (4–7, 8–11 & 12–16 years of age)  and the Paediatric Advanced Cancer-Quality of Life (PAC-QoL) scale has two versions (8–12 & 13–18) to account for developmental differences . Other generic child measures such as the PROMIS paediatric item bank  and KIDSCREEN [74, 89, 90] have the same forms for ages 8–18 years, and the Oxford Foot and Ankle PROM  has the same version for ages 5–15. This allows for the longitudinal use of the questionnaire and comparison of results across age groups .
This study described the careful development of a long list of items for the draft child specific CFS/ME PROM grounded in a child specific conceptual framework. Items were reviewed and refined through clinical expert review as well as healthy children and paediatric CFS/ME patient groups. The measure was then tested through cognitive interviews which illuminated problems, allowing changes and confirming acceptability in children aged 11–18. We were unable to confirm acceptability in younger children (< 11 years old) and further cognitive interviews are required with children 8–10 years old to examine the acceptability and content validity and provide evidence for age related cut offs of the new PROM to meet FDA standards. The long form questionnaire is currently too long to be readily completed in routine practice or research settings. The next essential step is a psychometric evaluation in the target population. This will support the reduction of items, confirm the structure of the PROM and provide initial essential evidence of the data quality, reliability and validity.
We would like to thank all the participants who took part in the study. We are grateful for the support of the health professionals and CFS/ME YPAG for their views and opinions to help refine the draft PROM.
RP, EC, AS and KLH designed the study. RP collected and analysed the data. RP, EC, AS and KLH contributed to the interpretation of results. All authors have read and approved the final version of the manuscript.
This work was supported by a University of Bristol PhD Scholarship. EC was funded by the National Institute for Health Research (Senior Research Fellowship, SRF-2013-06-013). The views expressed in this publication are those of the authors and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health.
Ethics approval and consent to participate
Full ethical approval was obtained for the cognitive interviews from the West of Scotland Research Ethics Service (25/01/2016, ref. 16/WS/0011). The relevant R&D approval was obtained (25/01/2016, ref-RBB 472).
Full ethical approval was obtained for the consensus meeting with health professionals in the development of the questionnaire from the NRES Committee North West (08/04/2014, ref. 14/NW/0170/14/NW/0770).
Consent for publication
EC was the medical advisor for the Association for Young people with ME (AYME) until 2017. The authors declare that they have no other conflict of interest.
- 4.RCPCH. (2004). Evidence Based Guideline for the Management of CFS/ME (Chronic Fatigue Syndrome/Myalgic Encephalopathy) in Children and Young People. London: Royal College of Paediatrics and Child Health.Google Scholar
- 5.NICE. (2007). Chronic fatigue syndrome/myalgic encephalomyelitis (or encephalopathy) Diagnosis and management of CFS/ME in adults and children. London: National Institute for Health and Care Excellence.Google Scholar
- 6.Huang, Y., et al. (2010). Postinfectious fatigue in adolescents and physical activity. Archives of Pediatrics & Adolescent Medicine, 164(9), 803–809.Google Scholar
- 17.Food and Drug Administration. (2009). Guidance for Industry Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims (pp. 65132–65133). Silver Spring: Federal Register.Google Scholar
- 19.Patrick, D. L., et al. (2011). Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: Part 2--assessing respondent understanding. Value in Health, 14(8), 978–988.PubMedCrossRefPubMedCentralGoogle Scholar
- 20.Patrick, D. L., et al. (2011). Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: Part 1--eliciting concepts for a new PRO instrument. Value in Health, 14(8), 967–977.PubMedCrossRefPubMedCentralGoogle Scholar
- 27.Haywood, K. L., Collin, S. M., & Crawley, E. (2014). Assessing severity of illness and outcomes of treatment in children with Chronic Fatigue Syndrome/Myalgic Encephalomyelitis (CFS/ME): a systematic review of patient-reported outcome measures (PROMs) (pp. n/a–n/a). Child: Care, Health and Development. 40 806-824. https://doi.org/10.1111/cch.12135 Google Scholar
- 29.Parslow, R. M. (2016). Developing a Patient Reported Outcome Measure (PROM) for Children with Chronic Fatigue Syndrome/Myalgic Encephalomyelitis (CFS/ME), in Centre for Child and Adolescent Health. Bristol: Univesity of Bristol.Google Scholar
- 38.Willis, G. B. (2004). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks: Sage Publications.Google Scholar
- 40.Petersson, C., et al. (2013). Comparing children’s self-report instruments for health-related quality of life using the international classification of functioning, disability and health for children and youth (ICF-CY). Health and Quality of Life Outcomes, 11(75), 1–10.Google Scholar
- 43.Hak, T., & Van der Veer, K. (2008). The Three-Step Test-Interview (TSTI): An observation-based method for pretesting self-completion questionnaires. In Survey Research Methods.Google Scholar
- 44.Ritchie, J., et al. (2014). Qualitative research practice: A guide for social science students and researchers. Second Edition. London: Sage Publication Ltd.Google Scholar
- 45.Tourangeau, R. (1984). Cognitive sciences and survey methods. In T. Jabine et al. (Eds.), Cognitive aspects of survey methodology: Building a bridge between disciplines (pp. 73–100). Washington, DC: National Academy Press.Google Scholar
- 46.Gorecki, C., et al. (2013). Development and validation of a new patient-reported outcome measure for patients with pressure ulcers: The PU-QOL instrument. Health and Quality of Life Outcomes, 11(95), 1–12.Google Scholar
- 60.Cremeens, J., Eiser, C., & Blades, M. (2007). A qualitative investigation of school-aged children’s answers to items from a generic quality of life measure. Child: Care, Health and Development, 33(1), 83–89.Google Scholar
- 72.PROMIS. PROMIS: Instrument Development and Psychometric Evaluation Scientific Standards. PROMIS Standards for Measurement Development 2012; US Department of Health Human Services: Available from: http://www.nihpromis.org/science/publications. Cited 2016
- 80.Crawley, E., et al. (2013). Comparing specialist medical care with specialist medical care plus the lightning process for chronic fatigue syndrome or myalgic encephalomyelitis (CFS/ME): Study protocol for a randomised controlled trial (SMILE trial). Trials, 14, 444.PubMedPubMedCentralCrossRefGoogle Scholar
- 81.Brigden, A., et al. (2016). G73 (P) the MAGENTA protocol: The feasibility and acceptability of conducting a trial investigating the effectiveness and cost effectiveness of graded exercise therapy compared to activity management for paediatric CFS/ME–a feasibility randomised controlled trial. Archives of Disease in Childhood, 101(Suppl 1), A43–A43.CrossRefGoogle Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.