Measurement properties of instruments to assess pain in children and adolescents with cancer: a systematic review protocol
Pain in children and adolescents with cancer has been identified as an area where many healthcare professionals seek guidance. This protocol details a systematic review whose aim is to explore current knowledge regarding measurement instruments to assess pain (and pain-related distress) in children and adolescents with cancer. After completion of the review, the information will be used in the development of a clinical practice guideline.
We will search four electronic databases (MEDLINE via PubMed, CINAHL, PsycINFO and HaPI). Additional relevant studies will be identified by reference checking and expert consultation. All citations will be screened independently by two reviewers in a three-step approach: first selection based on title, second selection based on abstract, third selection based on full-text. Studies in children and adolescents with cancer that aimed to evaluate the clinimetric properties of an existing pain measurement instrument or to develop a new pain measurement instrument and that include at least one relevant outcome (reliability, validity, responsiveness, interpretability, clinical utility) are eligible for inclusion. For all steps of evidence selection, a detailed list with eligibility criteria will be determined a priori. Data extraction and quality assessment of included studies (according to the COnsensus-based Standards for the selection of health Measurement INstruments, COSMIN criteria) will be conducted independently by two authors.
This systematic review will provide an overview of the current literature regarding measurement instruments to assess pain in children and adolescents with cancer. This knowledge synthesis will be used to formulate recommendations for clinical practice. Also, by synthesizing existing evidence, knowledge gaps will be identified.
Systematic review registration
KeywordsSystematic review Protocol Pediatric oncology Pain Measurement instrument
Cumulative Index to Nursing and Allied Health Literature
COnsensus-based Standards for the selection of health Measurement INstruments
Excerpta Medica Database
Grading of Recommendations Assessment, Development and Evaluation
Health and Psychosocial Instruments Database
Medical Literature Analysis and Retrieval System Online
Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols
International prospective register of systematic reviews
Psychological Information Database
Visual analog scale
The survival outcome for children with cancer has improved dramatically over the past 50 years . In developed nations, the cure rates of childhood cancers now exceed 80%, whereas, in the 1960s, cure rates in children with cancer were the opposite: 20% [2, 3]. This remarkable increase can be attributed to a multitude of factors including the introduction of intensive treatment protocols and highly coordinated modern multidisciplinary care. However, these treatment modalities (mainly chemotherapy, radiotherapy, and surgery) also have clear undesirable side effects, both short- and long-term. Of these, pain is among the most prevalent and distressing [4, 5].
Besides improving survival outcome, an important focus in present day pediatric oncology is reducing morbidity and improving quality of life during and after treatment. This is referred to as supportive care, which the Multinational Association of Supportive Care in Cancer defined as “the prevention and management of the adverse effects of cancer and its treatment across the cancer continuum” . As the field of supportive care is extremely broad, healthcare professionals often seek guidance on how to optimally assess and manage the undesirable effects of anti-cancer therapy . To provide this guidance, the project “Towards evidence-based guidelines in childhood cancer” was initiated in 2014 .
As the first step in this project, we surveyed health care providers to determine the topic areas in which they wanted guidance. Pain assessment, evaluation (i.e., interpretation of the assessment to evaluate treatment effect), and management were identified as priorities [8, 9]. Regarding pain assessment, one of the great challenges in pediatric oncology is selecting measurement instruments that are age-appropriate, clinimetrically sound, and facilitate measurement of the various types of pain that children and adolescents with cancer experience.
In previous research, the focus has been placed upon synthesizing knowledge regarding pain measurement tools used with other populations. A systematic review on pain intensity assessment tools for adults concluded that numerical rating scales are applicable for unidimensional assessment of pain intensity in these patients . Another recent systematic review focused on non-responsive adults and found that there are multiple observational pain assessment tools available for this population . However, there are clear differences between children and adults, and there are various types of pain that children with cancer experience (e.g., neuropathic pain) that require specific guidance.
Thus, this systematic review will focus on assessment and evaluation of pain and pain-related distress and is an essential step in the development of a clinical practice guideline regarding pain in children and adolescents with cancer. For this guideline, the international guideline development panel consists of 46 members (including pediatric oncologists, nurses, anesthesiologists, pharmacologists, psychologists, and patient representatives) divided into a core group (leads, coordinators, advisors) and six working groups. Recommendations will be developed for assessment and evaluation of pain, as well as for pharmacological, physical, and psychological management of tumor-, treatment-, and procedure-related pain.
Construct of interest, i.e., pain (and pain-related distress)
Population of interest, i.e., children and adolescents with cancer
Type of measurement instrument, i.e., self-report (using numbers or pictures/faces/colors), observer ratings, behavioral measurement, and multidimensional measurement
Measurement properties, i.e., reliability, validity, responsiveness, interpretability, and clinical utility
The Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) statement has been followed in the development of this systematic review protocol . The completed PRISMA-P checklist can be found in Additional file 1.
To be included in this review, studies need to meet the subjoined inclusion criteria. These are defined according to the COSMIN criteria:
Construct of interest
Studies that evaluate instruments that aim to measure the construct of pain (all types of pain). Distress is also taken into account, when related to pain assessment.
Population of interest
Studies that encompass children and adolescents with cancer, defined as (1) all participants < 25 years old or a median ≤ 16 years old or a mean ≤ 16 years old and (2) at least 75% of participants are diagnosed with cancer.
Type of measurement instrument
Studies that investigate a relevant measurement instrument, which differs per clinical question and is either self-reported (e.g., visual analog scale (VAS)) or a behavioral pain/distress measurement instrument (e.g., the COMFORT scale) [13, 14].
Overview of clinical questions and included outcomes per clinical question. Included outcomes differ per clinical question and are highlighted with a number/color. The number represents the median score of the outcome in the guideline development panel voting process. Outcomes with a score of 7–9 are considered critical to decision-making (highlighted green), outcomes with a score of 4–6 are considered important, but not critical to decision-making (highlighted yellow), and outcomes with a score of 1–3 are considered of low importance to decision-making (none in this case)
Eligible study types
Only primary, quantitative studies will be included. The sample size per study should be at least ten participants. Studies should clearly state their aim is to evaluate the clinimetric properties of an existing measurement instrument or to develop a new measurement instrument (this criterion is in line with COSMIN) . Studies need to be published in a peer-reviewed medical journal, with a full text available in the English language.
Children, with free text terms such as infan* and child*, and MeSH headings such as pediatrics[mh]
Childhood cancer types, with free text terms such as osteosarcom* and meningiom*, and MeSH headings such as (leukemia, lymphocytic, acute[mh])
Pain, with free text terms such as neuropath* and ache, and MeSH headings such as pain[mh]
Search strategy for PubMed/MEDLINE
Results (March 22, 2017)
((leukemia OR leukemi* OR leukaemi* OR (childhood ALL) OR AML OR lymphoma OR lymphom* OR hodgkin OR hodgkin* OR T-cell OR B-cell OR non-hodgkin OR sarcoma OR sarcom* OR sarcoma, Ewing’s OR Ewing* OR osteosarcoma OR osteosarcom* OR wilms tumor OR wilms* OR nephroblastom* OR neuroblastoma OR neuroblastom* OR rhabdomyosarcoma OR rhabdomyosarcom* OR teratoma OR teratom* OR hepatoma OR hepatom* OR hepatoblastoma OR hepatoblastom* OR PNET OR medulloblastoma OR medulloblastom* OR PNET* OR neuroectodermal tumors, primitive OR retinoblastoma OR retinoblastom* OR meningioma OR meningiom* OR glioma OR gliom*) OR (pediatric oncology OR paediatric oncology) OR (childhood cancer OR childhood tumor OR childhood tumors)) OR (brain tumor* OR brain tumour* OR brain neoplasms OR central nervous system neoplasm OR central nervous system neoplasms OR central nervous system tumor* OR central nervous system tumour* OR brain cancer* OR brain neoplasm* OR intracranial neoplasm*) OR (leukemia, lymphocytic, acute[mh]) OR (leukemia, lymphocytic, acute*)
Search filter for childhood cancer (from Cochrane Childhood Cancer)
Infan* OR toddler* OR minors OR minors* OR boy OR boys OR boyfriend OR boyhood OR girl* OR kid OR kids OR child OR child* OR children* OR schoolchild* OR schoolchild OR school child[tiab] OR school child*[tiab] OR adolescen* OR juvenil* OR youth* OR teen* OR under*age* OR pubescen* OR pediatrics[mh] OR pediatric* OR paediatric* OR peadiatric* OR school[tiab] OR school*[tiab]
Search filter for children (from Cochrane Childhood Cancer)
pain[mh] OR acute pain[mh] OR chronic pain[mh] OR pain OR ache OR agony OR hyperalgesia[mh] OR allodynia OR analgesia OR distress OR headache OR hurt[tiab] OR hyperesthesia OR hyperaesthesia OR myalgia OR neuralgia OR neuropath* OR polyneuropathy OR painful[tiab]
Search filter for pain (developed in consultation with the information specialist from the Cochrane Childhood Cancer)
Validation Studies[pt] OR “psychometrics”[MeSH] OR psychometr*[tiab] OR clinimetr*[tw] OR clinometr*[tw] OR reproducibi*[tiab] OR reliabi*[tiab] OR unreliabi*[tiab] OR validi*[tiab] OR valida*[tiab] OR “internal consistency”[tiab] OR precision[tiab] OR imprecision[tiab] OR test–retest[tiab] OR repeatabi*[tiab] OR ((multitrait[tiab] AND scaling[tiab]) AND (analysis[tiab] OR analyses[tiab])) OR item discriminant[tiab] OR sensitivi*[tiab] OR specifici* OR ((minimal[tiab] OR minimally[tiab]) AND (clinical[tiab] OR clinically[tiab]) AND (important[tiab] OR significant[tiab] OR detectable[tiab]) AND (change[tiab] OR difference[tiab])) OR (small* AND (real[tiab] OR detectable[tiab]) AND (change[tiab] OR difference[tiab])) OR pain measurement[mh]
Search filter measurement properties (developed in consultation with the information specialist from the Cochrane Childhood Cancer)
#1 AND #2 AND #3 AND #4
#5, filters: Humans; English
All citations that will be identified in the database searches will be imported in EndNote X8 (Clarivate Analytics, Philadelphia, PA, USA) after which duplicates will be removed. For appraisal of the citations, we will use exported Microsoft Excel files (Microsoft, Redmond, WA, USA).
As we expect a large total number of citations, the first round will consist of selection based on titles only. A recent study that compared screening titles alone versus titles and abstracts concluded that screening on title alone may be more effective . The entire selection will be performed in duplicate by two independent reviewers (with extensive experience in both pediatric oncology and in systematic reviews, 1: EL, 2: WT). This serves to exclude studies that are obviously irrelevant for this review. A conservative approach to inclusion will be employed to ensure relevant titles are not excluded. All citations classified as “include” by at least one reviewer will be included, even if the second reviewer classified the citation as “exclude” (no discussion will be held). It should be noted that this conservative approach is only applied during the title selection phase. During abstract and full-text selection, all discrepancies will be discussed in detail and resolved by consensus.
To pilot the title selection process, two reviewers will appraise the first 250 citations. If there is an agreement below 85% percent, the title selection criteria will be optimized and the pilot will be repeated on the next 250 citations. This continues until the agreement is at least 85%.
The second round will consist of selection based on title and abstract. The entire selection will be performed in duplicate by two independent reviewers (1: EL, 2: divided among JS, KB, MD, KK, MR, MW, and FG). A pilot will be performed with the first 100 citations in a similar fashion as with the title selection pilot. Hereafter, discrepancies will be resolved by consensus, and if this is not reached, a third reviewer will be consulted.
In the final selection round, two independent reviewers (the same as in the second round) will perform the selection of the full texts. Discrepancies will be resolved by consensus, and if this is not reached, a third reviewer will be consulted.
Two reviewers will independently extract data from each included study using a data abstraction form designed for this systematic review (see Additional file 1). This form will be pilot-tested by two reviewers with a representable sample of studies and, if necessary, optimized according to the reviewers’ comments. The data extraction form differs slightly for each clinical question, but includes at minimum (1) general study information (i.e., first author, title, journal, year, funding source), (2) study design characteristics (i.e., study design, setting, duration, sample size), (3) participant characteristics (i.e., age, gender, diagnosis, age at diagnosis, type of pain), (4) measurement instrument characteristics (i.e., instrument used, measurement method/dimensions), (5) outcome characteristics (i.e., included outcomes, outcome values), and (6) additional information (differs per study, at the discretion of the reviewer). Reviewers will discuss the completed forms, and discrepancies will be resolved by consensus. If consensus is not reached, a third reviewer will be consulted. In case of missing information, the authors of the included study will be contacted in an effort to obtain this information.
Two reviewers will independently assess the quality of the included studies, according to the COSMIN checklist for assessing methodological quality of studies on measurement properties [15, 19]. This checklist is purpose-built for the evaluation of measurement properties and takes a three-tier approach. For multi-item measures, it is first determined if the statistical methods are based on the classical test theory (CCT) or on the item response theory (IRT); in the case of the latter, a separate quality checklist is included. Second, a checklist is completed for each measurement property evaluated in each included paper to evaluate if standards for good methodological quality are met. Third and last, a checklist regarding the generalizability of each outcome is completed. Discrepancies between reviewers will be resolved in the same manner as described for data extraction.
The COSMIN 4-point checklist scoring system will result in a score per included outcome for each study that can either be “excellent” (evidence that the methodological quality is adequate), “good” (assumed that the methodological quality is adequate, but relevant information is not reported), “fair” (doubtful whether the methodological quality is adequate), or “poor” (evidence that the methodological quality is not adequate). This serves the purpose of identifying the measurement tool with the strongest clinimetric properties.
Regarding publication bias, we will seek to identify cues that imply this bias, e.g., identification of merely studies that report measurement instruments with strong clinimetric properties. A prospective registration database, as there is for randomized controlled trials, does not exist for measurement properties studies, which makes identifying publication bias particularly hard in this area of research. Also, there is no established methodology for this, e.g., as there is for a meta-analysis with a funnel plot. Therefore, we will comment upon this in a qualitative manner.
Synthesis of results
Firstly, we will prepare a table of characteristics of included studies where information regarding design, sample, measurement instrument(s), and outcome(s) will be presented. Outcome tables will be prepared to cover each measurement instrument (e.g., the Comfort Behavior Scale) and each outcome construct (e.g., self-reported pain intensity) separately. In addition, an outcome table will be prepared which categorizes instruments according to age group, i.e., 0–0.99 years (infants), 1–2.99 years (children not able to speak or read), 3–6.99 years (children able to speak, not able to read), 7–12.99 years (children able to speak and read), and 12–18 years (teenagers). If the identified evidence leads to the conclusion that differently defined age groups are more appropriate, we will reconsider the definition of the age groups.
To provide a comprehensive overview, we will develop a quality matrix. Herein, we will list all included measurement instruments and the identified information on purpose (self- or observer-report, type of pain), number of studies, population/age group, and per outcome COSMIN quality score (four-level). This will allow the reader to quickly identify the appropriate measurement instrument with the strongest clinimetric properties.
In addition, we will present a narrative synthesis discussing our findings.
In this protocol, we describe our approach for a systematic review regarding measurement properties of instruments to assess pain and pain-related distress in children and adolescents with cancer. This overview is much needed for multiple reasons.
First, there is a clear difference between adults and children. Second, besides this difference, there is also a great difference between children across ages (e.g., a 15-year-old and a 3-year-old). Children go through various developmental stages which lead to great variation in this population in terms of abilities in verbal communication, pain experience, and associative thinking, which will determine the appropriateness of individual instruments within specific age groups . Although self-report is often regarded as the ideal method for assessing pain, in young children, this is not always possible. Therefore, these populations have to rely on the people around them (family members but also healthcare professionals) to notice and act upon their pain. This underlines the importance of having clinimetrically sound measurement instruments not just for self-report, but also for proxy report. Third, children and adolescents with cancer have various specific types of pain which are often related to their disease (e.g., bone pain due to metastases) or treatment (e.g., procedural pain, vinca alkaloid-associated neuropathic pain). Fourth, pain severity measurement is the critical first step to determine the need for treatment and to evaluate the success of treatment. When pain is not measured, it is often undertreated . All of these reasons call for age-specific measurement instruments with strong clinimetric properties to assess pain in children and adolescents with cancer.
Currently, there is great interest in reducing the side-effects of cancer treatment in children and adolescents, to reduce morbidity and improve quality of life. Pain is frequently experienced by children and adolescents with cancer and carries with it a great physical and psychological burden . It is known that not acknowledging and treating pain can have a detrimental effect on quality of life . Therefore, it is of utmost importance that healthcare professionals have access to the available evidence and evidence-based recommendations for care.
To address these needs, we have initiated a clinical practice guideline development project that focuses on formulating recommendations on the assessment and treatment of pain in children and adolescents with cancer. This systematic review of the literature describing pain measurement instruments is the first step towards formulating these recommendations.
We thank Edith Leclercq for critically reviewing the search strategy and running the initial searches in order to check the correctness of the strategy and identify necessary changes.
The project “Towards evidence-based guidelines for supportive care in childhood oncology” is supported by the Alpe d’HuZes foundation/Dutch Cancer Society (RUG 2013–6345). The editorial base of Cochrane Childhood Cancer is funded by Stichting Kinderen Kankervrij (KiKa). The funding source had no role in the study design, in the preparation of the manuscript, or in the decision to submit the manuscript for publication.
Availability of data and materials
EL contributed in defining the clinical questions and the conceptualization and design of the review and drafted the manuscript. JS, KB, MD, KK, MR, AF, LD, ED, RM, FC, MW, and FG contributed in defining the clinical questions and the conceptualization and design of the review. WT is the guarantor for the review, contributed in defining the clinical questions and the conceptualization and design of the review, and drafted the manuscript. All authors critically reviewed the protocol and manuscript. All authors read and approved the final manuscript.
Ethics approval and consent to participate
Consent for publication
Erik A.H. Loeffen is an associate editor of Systematic Reviews. All other authors declare no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
- 6.MASCC - Multinational Association of Supportive Care in Cancer [Internet]. 2016 [cited 2017 Jun 27]. Available from: http://www.mascc.org/
- 10.Hjermstad MJ, Fayers PM, Haugen DF, Caraceni A, Hanks GW, Loge JH, et al. Studies comparing numerical rating scales, verbal rating scales, and visual analogue scales for assessment of pain intensity in adults: a systematic literature review. J Pain Symptom Manag. 2011 Jun;41(6):1073–93.CrossRefGoogle Scholar
- 11.Varndell W, Fry M, Elliott D. A systematic review of observational pain assessment instruments for use with nonverbal intubated critically ill adult patients in the emergency department: an assessment of their suitability and psychometric properties. J Clin Nurs. 2017 Jan;26(1–2):7–32.CrossRefGoogle Scholar
- 19.Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010 May 19;19(4):539–49.CrossRefGoogle Scholar
- 21.Srouji R, Ratnapalan S, Schneeweiss S. Pain in children: assessment and nonpharmacological management. Int J Pediatr. 2010;2010:474838.Google Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.