There is a need to measure and improve the quality of paediatric primary care in Europe where major differences in the delivery and outcomes of child health care exist. A collaborative panel of paediatric senior experts developed a Core Set of Indicators for Paediatric Primary Care in Europe by compiling 42 quality indicators in a modified consensus process following the RAND/UCLA appropriateness method. The aim of this study was to explore the feasibility of the quality indicator set in European paediatric primary care practices. Seventy-nine practices from eight countries participated in a detailed online interview. The practices rated the applicability, relevance, reliability and acceptance of the 42 quality indicator as well as the availability, technical feasibility and effort to retrieve the needed data from their medical records. Most quality indicators were considered applicable, available, reliable, acceptable and relevant for monitoring quality of care in paediatric primary care. Respondents rated feasibility and effort to retrieve the data lowest because of difficulties collecting the data from the medical records.
Conclusion: European paediatric primary care practices generally agree with the proposed quality indicator set. They document most of the parameters. However, the collection of specific needed values from available routine patient-data is considered technically difficult and time-consuming.
What is Known?
• Paediatric primary care systems in Europe show striking differences in their performance. Pre-existing sets of quality indicators are predominantly limited to national populations, specific diseases and hospital care.
• A Core Set of 42 quality indicators for paediatric primary care in Europe was developed by European paediatricians using a systematic literature review and a consensus process following a modified RAND/UCLA appropriateness method.
What is New?
• Paediatric primary care providers in Europe agree with the idea to use COSI-PPC-EU to monitor and improve the quality of care. The set was considered applicable, available, reliable, acceptable, and relevant for quality improvement.
• The score for feasibility and effort to retrieve the data was low, because of technical reasons; the electronical or paper-based medical documentation in most cases does not allow convenient access to all necessary data.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Continuing medical education
Core Set of Quality Indicators for Paediatric Primary Care in Europe
German Academic Society for General Paediatrics
European Academy of Paediatrics
European Confederation of Primary Care Paediatricians
Electronic Data Processing
Electronic Medical Records
SPSS Statistics is a software package used for statistical analysis from the IBM Corporation
International Classification of Disease
Models of Child Health Appraised Project
National Institute for Health and Care Excellence, UK
Paediatric Primary Care
RAND/University of California Los Angeles Appropriateness Method
Bennett B, Coventry E, Greenway N, Minchin M (2014) The NICE process for developing quality standards and indicators. Z Für Evidenz Fortbild Qual Im Gesundheitswesen 108:481–486. https://doi.org/10.1016/j.zefq.2014.09.008
Beyer M, Chenot R, Erler A, Gerlach FM (2011) Die Darstellung der hausärztlichen Versorgungsqualität durch Qualitätsindikatoren. Z Für Evidenz Fortbild Qual Im Gesundheitswesen 105:13–20. https://doi.org/10.1016/j.zefq.2010.09.018
Campbell SM, Braspenning J, Hutchinson A, Marshall M (2002) Research methods used in developing and applying quality indicators in primary care. Qual Saf Health Care 11:358–364. https://doi.org/10.1136/qhc.11.4.358
Cheung CRLH, Gray JAM (2013) Unwarranted variation in health care for children and young people. Arch Dis Child 98:60–65. https://doi.org/10.1136/archdischild-2012-302041
de Cruppé W, Kleudgen S, Diel F, Burgdorf F, Geraedts M (2015) Feasibility of 48 quality indicators in ambulatory care in Germany: a cross-sectional observational study. Z Für Evidenz Fortbild Qual Im Gesundheitswesen 109:682–694. https://doi.org/10.1016/j.zefq.2015.02.015
de Cruppé W, Nguyen BH, Weissenrieder N et al (2010) Qualitätsmanagement in kinder- und jugendärztlichen Praxen. Monatsschr Kinderheilkd 159:145–151. https://doi.org/10.1007/s00112-010-2287-0
Donabedian A (1966) Evaluating the quality of medical care. Milbank Mem Fund Q 44:166–206. https://doi.org/10.2307/3348969
Ehrich J, Namazova-Baranova L, Pettoello-Mantovani M (2016) Introduction to “diversity of child health Care in Europe: a study of the European Paediatric association/Union of National European Paediatric Societies and Associations”. J Pediatr 177:S1–S10. https://doi.org/10.1016/j.jpeds.2016.04.036
Engels Y, Dautzenberg M, Campbell S, Broge B, Boffin N, Marshall M, Elwyn G, Vodopivec-Jamsek V, Gerlach FM, Samuelson M, Grol R (2006) Testing a European set of indicators for the evaluation of the management of primary care practices. Fam Pract 23:137–147. https://doi.org/10.1093/fampra/cmi091
van Esso D, del Torso S, Hadjipanayis A, Biver A, Jaeger-Roman E, Wettergren B, Nicholson A, and the members of the Primary-Secondary Working Group (PSWG) of the European Academy of Paediatrics (EAP) (2010) Paediatric primary care in Europe: variation between countries. Arch Dis Child 95:791–795. https://doi.org/10.1136/adc.2009.178459
European Commission, Directorate General for Health & Consumers (2014) The third Health Programme 2014–2020 funding health initiatives. European Commission, Brussels
Ewald DA, Huss G, Auras S, Caceres JRC, Hadjipanayis A, Geraedts M (2018) Development of a core set of quality indicators for paediatric primary care practices in Europe, COSI-PPC-EU. Eur J Pediatr 177:921–933. https://doi.org/10.1007/s00431-018-3140-z
Fitch K (2001) The Rand/UCLA appropriateness method user’s manual. Rand, Santa Monica
Gill PJ, O’Neill B, Rose P, Mant D, Harnden A (2014) Primary care quality indicators for children: measuring quality in UK general practice. Br J Gen Pract 64:e752–e757. https://doi.org/10.3399/bjgp14X682813
Gold R, Angier H, Mangione-Smith R, Gallia C, McIntire PJ, Cowburn S, Tillotson C, DeVoe JE (2012) Feasibility of evaluating the CHIPRA care quality measures in electronic health record data. Pediatrics 130:139–149. https://doi.org/10.1542/peds.2011-3705
Hartveit M, Vanhaecht K, Thorsen O, Biringer E, Haug K, Aslaksen A (2017) Quality indicators for the referral process from primary to specialised mental health care: an explorative study in accordance with the RAND appropriateness method. BMC Health Serv Res 17:4. https://doi.org/10.1186/s12913-016-1941-1
Holmboe ES, Weng W, Arnold GK, Kaplan SH, Normand SL, Greenfield S, Hood S, Lipner RS (2010) The comprehensive care project: measuring physician performance in ambulatory practice. Health Serv Res 45:1912–1933. https://doi.org/10.1111/j.1475-6773.2010.01160.x
Horodecka AM, Wolniak R (2015) Uwarunkowania rynku włoskiego w zakresie implementacji SZJ norm ISO 9000. Probl Jakości 1:22–28. https://doi.org/10.15199/46.2015.12.4
Kötter T, Blozik E, Scherer M (2012) Methods for the guideline-based development of quality indicators--a systematic review. Implement Sci 7:21. https://doi.org/10.1186/1748-5908-7-21
Mainz J (2003) Developing evidence-based clinical indicators: a state of the art methods primer. Int J Qual Health Care 15:5i–11i. https://doi.org/10.1093/intqhc/mzg084
Mainz J (2004) Quality indicators: essential for quality improvement. Int J Qual Health Care 16:i1–i2. https://doi.org/10.1093/intqhc/mzh036
Mangione-Smith R, Roth CP, Britto MT, Chen AY, McGalliard J, Boat TF, Adams JL, McGlynn EA (2017) Development and testing of the pediatric respiratory illness measurement system (PRIMES) quality indicators. Hosp Pediatr 7:125–133. https://doi.org/10.1542/hpeds.2016-0182
Mangione-Smith R, Schiff J, Dougherty D (2011) Identifying children’s health care quality measures for Medicaid and CHIP: an evidence-informed, publicly transparent expert process. Acad Pediatr 11:S11–S21
Marshall M, Roland MO, Campbell SM et al (2003) Measuring general practice. Demonstr. Proj. Dev. Test set prim. Care Clin. Qual. Indic. Lond. Nuffield trust
Marshall MN, Shekelle PG, McGlynn EA, Campbell S, Brook RH, Roland MO (2003) Can health care quality indicators be transferred between countries? Qual Saf Health Care 12:8–12. https://doi.org/10.1136/qhc.12.1.8
Mattke S, Nicolucci A, Greenfield S (2004) Selecting Indicators for the Quality of Diabetes Care at the Health Systems Level in OECD Countries. doi: https://doi.org/10.1787/165531523300
McGlynn EA (1998) Choosing and evaluating clinical performance measures. Jt Comm J Qual Improv 24:470–479. https://doi.org/10.1016/S1070-3241(16)30396-0
Peña A, Virk SS, Shewchuk RM et al (2010) Validity versus feasibility for quality of care indicators: expert panel results from the MI-plus study. Int J Qual Health Care 22:201–209. https://doi.org/10.1093/intqhc/mzq018
Rubin HR, Pronovost P, Diette GB (2001) Methodology matters. From a process of care to a measure: the development and testing of a quality indicator. Int J Qual Health Care 13:489–496. https://doi.org/10.1093/intqhc/13.6.489
Salzer-Muhar U, Pollak A, Aufricht C, Ehrich J, Lenton S (2012) European challenges: cross-border care for children. J Pediatr 161:574–576. https://doi.org/10.1016/j.jpeds.2012.06.052
Stang AS, Straus SE, Crotts J et al (2013) Quality Indicators for High Acuity Pediatric Conditions. Pediatrics peds 132(4):752–762 2013–0854. https://doi.org/10.1542/peds.2013-0854
Thern J, de With K, Strauss R et al (2014) Selection of hospital antimicrobial prescribing quality indicators: a consensus among German antibiotic stewardship (ABS) networkers. Infection 42:351–362. https://doi.org/10.1007/s15010-013-0559-z
To T, Guttmann A, Lougheed MD et al (2010) Evidence-based performance indicators of primary care for asthma: a modified RAND appropriateness method. Int J Qual Health Care 22:476–485. https://doi.org/10.1093/intqhc/mzq061
Wolfe I, Thompson M, Gill P, Tamburlini G, Blair M, van den Bruel A, Ehrich J, Pettoello-Mantovani M, Janson S, Karanikolos M, McKee M (2013) Health services for children in western Europe. Lancet 381:1224–1234. https://doi.org/10.1016/S0140-6736(12)62085-6
Wollersheim H, Hermens R, Hulscher M, Braspenning J, Ouwens M, Schouten J, Marres H, Dijkstra R, Grol R (2007) Clinical indicators: development and applications. Neth J Med 65:15–22
We thank the country coordinators which selected and motivated the study participants: Andreas Werner and Liliane Cret (France), Folkert Fehr (Germany), Peter Altorjaj and Arkos Kovacs (Hungary), Shimon Barak and Jakob Urkin (Israel), Laura Reali and Patrizia Calamita (Italy), Beata Kartousova (Slovakia), Denis Baš and Margareta Seher Zupancic (Slovenia), Juan Ruiz Canela and Angel Saenz Carrasco (Spain), Daniela Kasparek and Wilhelm Sedlack (Austria), Hadjigeorgiou Charis (Cyprus), Gabriela Kubatova (Czech Republic), Pirkko Keronen (Finland), Iveta Skurvydiene (Lithuania) and Mario Schumacher (Switzerland). We thank all 79 participating paediatricians for their invested time and engagement, as well as all other country delegates and members from EAP, ECPCP and German Academic Society for General Paediatrics (DGAAP), who supported the study.
The scientific evaluation of the COSI project by Max Geraedts and Rike A. Kraska was financed by a grant from the “Foundation Child and Youth”, Cologne, and by European Confederation of Primary Care Paediatricians (ECPCP) and German Association of Primary Care Paediatrics (DGAAP).
Conflict of interest
The authors declare that they have no conflict of interests.
All authors revised the manuscript for important intellectual content and approved the final version before submission.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Communicated by Mario Bianchetti
Electronic supplementary material
About this article
Cite this article
Ewald, D.A., Huss, G., Kraska, R.A. et al. Feasibility testing of the Core set of quality Indicators for Paediatric Primary Care in Europe, COSI-PPC-EU. Eur J Pediatr 178, 707–719 (2019). https://doi.org/10.1007/s00431-019-03344-5
- Quality indicator
- Quality improvement
- Paediatric primary care
- Community child care
- Feasibility study