A Theoretical Framework and Competency-Based Approach to Training in Guideline Development

  • Shahnaz Sultan
  • Rebecca L. Morgan
  • M. Hassan MuradEmail author
  • Yngve Falck-Ytter
  • Philipp Dahm
  • Holger J. Schünemann
  • Reem A. Mustafa
Original Research



There is increasing requirement to develop guidelines using transparent, standardized, and rigorous methods. Consequently, a better understanding of the knowledge, skills, and expertise necessary for guideline development is needed. The aim of this manuscript is to describe a theoretical framework of knowledge and skills that are required for individuals to serve on a guideline panel in varying capacities.


Based on an iterative process and review of published manuscripts focused on guideline development, we identified competencies, subcompetencies, and milestones.


Using a competency-based approach to training and the Dreyfus model of skill acquisition, we identified three core competencies: (1) facilitate the development of guideline structure and setup, (2) make judgments about the quality or certainty of the evidence, and (3) transform evidence to a recommendation. Level 1 focuses on recognizing and acknowledging the importance of a specific skill or behavior. Levels 2 and 3 require learners to demonstrate progressive acquisition of knowledge and application to specific behaviors. Level 4 represents the individual who has acquired the requisite knowledge and can function independently, while level 5 represents the mastery/aspirational level.


We propose a preliminary competency-based education framework that will (1) help standardize the qualifications needed for individuals to serve on guideline panels in varying capacities or (2) help with curricula development for teaching and training of guideline panel members. This framework can also help enable guideline-producing organizations to identify guideline methodologists with the relevant and appropriate level of knowledge and skills to lead guidelines. Validation of the framework and further refinement of the competencies and milestones will be required before widespread adoption.


clinical practice guideline methodologist competency-based training guideline panel 



The authors acknowledge the participation and contributions of Ms. Madelin Seidler and Dr. Raj Shah in the development of this manuscript.


Conception and design: SS, PD, YFY, RLM, MHM, and RAM; development of the framework: SS, PD, YFY, RLM, MHM, and RAM; analysis and interpretation: SS, PD, YFY, RLM, MHM, and RAM; drafting of the article: SS; and critical revisions of the article for important intellectual content: PD, YFY, RLM, MHM, RAM, and HJS.

Compliance with Ethical Standards

Conflict of Interest

The authors do not have relevant financial conflicts of interest. All the authors are members of the GRADE Working Group and the U.S. GRADE Network. The authors provide educational activities and workshops on guideline development.


  1. 1.
    Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924-6.CrossRefGoogle Scholar
  2. 2.
    Murad MH. Clinical practice guidelines: a primer on development and dissemination. Mayo Clin Proc 2017;92(3):423-33.CrossRefGoogle Scholar
  3. 3.
    Institute of Medicine. Clinical practice guidelines we can trust. Washinton, DC: National Academies Press; 2011; pp. 1-300.Google Scholar
  4. 4.
    Rosenfeld RM, Wyer PC. Stakeholder-driven quality improvement: a compelling force for clinical practice guidelines. Otolaryngol Head Neck Surg 2018;158(1):16-20.CrossRefGoogle Scholar
  5. 5.
    Ioannidis JPA. Professional societies should abstain from authorship of guidelines and disease definition statements. Circ Cardiovasc Qual Outcomes 2018;11(10):e004889.PubMedGoogle Scholar
  6. 6.
    Ponce OJ, Alvarez-Villalobos N, Shah R, Mohammed K, Morgan RL, Sultan S, et al. What does expert opinion in guidelines mean? A meta-epidemiological study. Evid Based Med 2017;22(5):164-9.CrossRefGoogle Scholar
  7. 7.
    Guyatt G, Akl EA, Hirsh J, Kearon C, Crowther M, Gutterman D, et al. The vexing problem of guidelines and conflict of interest: a potential solution. Ann Intern Med 2010;152(11):738-41.CrossRefGoogle Scholar
  8. 8.
    Norris SL, Meerpohl JJ, Akl EA, Schunemann HJ, Gartlehner G, Chen Y, et al. The skills and experience of GRADE methodologists can be assessed with a simple tool. J Clin Epidemiol 2016;79:150-8 e1.CrossRefGoogle Scholar
  9. 9.
    Zhang Y, Coello PA, Guyatt GH, Yepes-Nunez JJ, Akl EA, Hazlewood G, et al. GRADE guidelines: 20. Assessing the certainty of evidence in the importance of outcomes or values and preferences-inconsistency, imprecision, and other domains. J Clin Epidemiol 2018.Google Scholar
  10. 10.
    Zhang Y, Alonso-Coello P, Guyatt GH, Yepes-Nunez JJ, Akl EA, Hazlewood G, et al. GRADE guidelines: 19. Assessing the certainty of evidence in the importance of outcomes or values and preferences—risk of bias and indirectness. J Clin Epidemiol 2018.Google Scholar
  11. 11.
    Schunemann HJ, Cuello C, Akl EA, Mustafa RA, Meerpohl JJ, Thayer K, et al. GRADE guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence. J Clin Epidemiol 2018.Google Scholar
  12. 12.
    Schunemann HJ, Mustafa R, Brozek J, Santesso N, Alonso-Coello P, Guyatt G, et al. GRADE guidelines: 16. GRADE evidence to decision frameworks for tests in clinical practice and public health. J Clin Epidemiol 2016;76:89-98.CrossRefGoogle Scholar
  13. 13.
    Andrews JC, Schunemann HJ, Oxman AD, Pottie K, Meerpohl JJ, Coello PA, et al. GRADE guidelines: 15. Going from evidence to recommendation—determinants of a recommendation’s direction and strength. J Clin Epidemiol 2013;66(7):726-35.CrossRefGoogle Scholar
  14. 14.
    Andrews J, Guyatt G, Oxman AD, Alderson P, Dahm P, Falck-Ytter Y, et al. GRADE guidelines: 14. Going from evidence to recommendations: the significance and presentation of recommendations. J Clin Epidemiol 2013;66(7):719-25.CrossRefGoogle Scholar
  15. 15.
    Guyatt GH, Thorlund K, Oxman AD, Walter SD, Patrick D, Furukawa TA, et al. GRADE guidelines: 13. Preparing summary of findings tables and evidence profiles—continuous outcomes. J Clin Epidemiol 2013;66(2):173-83.CrossRefGoogle Scholar
  16. 16.
    Guyatt GH, Oxman AD, Santesso N, Helfand M, Vist G, Kunz R, et al. GRADE guidelines: 12. Preparing summary of findings tables—binary outcomes. J Clin Epidemiol 2013;66(2):158-72.CrossRefGoogle Scholar
  17. 17.
    Guyatt G, Oxman AD, Sultan S, Brozek J, Glasziou P, Alonso-Coello P, et al. GRADE guidelines: 11. Making an overall rating of confidence in effect estimates for a single outcome and for all outcomes. J Clin Epidemiol 2013;66(2):151-7.CrossRefGoogle Scholar
  18. 18.
    Brunetti M, Shemilt I, Pregno S, Vale L, Oxman AD, Lord J, et al. GRADE guidelines: 10. Considering resource use and rating the quality of economic evidence. J Clin Epidemiol 2013;66(2):140-50.CrossRefGoogle Scholar
  19. 19.
    Guyatt GH, Oxman AD, Sultan S, Glasziou P, Akl EA, Alonso-Coello P, et al. GRADE guidelines: 9. Rating up the quality of evidence. J Clin Epidemiol 2011;64(12):1311-6.CrossRefGoogle Scholar
  20. 20.
    Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: 8. Rating the quality of evidence—indirectness. J Clin Epidemiol 2011;64(12):1303-10.CrossRefGoogle Scholar
  21. 21.
    Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: 7. Rating the quality of evidence—inconsistency. J Clin Epidemiol 2011;64(12):1294-302.CrossRefGoogle Scholar
  22. 22.
    Guyatt GH, Oxman AD, Montori V, Vist G, Kunz R, Brozek J, et al. GRADE guidelines: 5. Rating the quality of evidence—publication bias. J Clin Epidemiol 2011;64(12):1277-82.CrossRefGoogle Scholar
  23. 23.
    Guyatt GH, Oxman AD, Vist G, Kunz R, Brozek J, Alonso-Coello P, et al. GRADE guidelines: 4. Rating the quality of evidence—study limitations (risk of bias). J Clin Epidemiol 2011;64(4):407-15.CrossRefGoogle Scholar
  24. 24.
    Balshem H, Helfand M, Schunemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol 2011;64(4):401-6.CrossRefGoogle Scholar
  25. 25.
    Guyatt GH, Oxman AD, Kunz R, Atkins D, Brozek J, Vist G, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin Epidemiol 2011;64(4):395-400.CrossRefGoogle Scholar
  26. 26.
    Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables. J Clin Epidemiol 2011;64(4):383-94.CrossRefGoogle Scholar
  27. 27.
    Guyatt GH, Ebrahim S, Alonso-Coello P, Johnston BC, Mathioudakis AG, Briel M, et al. GRADE guidelines 17: assessing the risk of bias associated with missing participant outcome data in a body of evidence. J Clin Epidemiol 2017;87:14-22.CrossRefGoogle Scholar
  28. 28.
    Guyatt GH, Oxman AD, Kunz R, Brozek J, Alonso-Coello P, Rind D, et al. GRADE guidelines 6. Rating the quality of evidence—imprecision. J Clin Epidemiol 2011;64(12):1283-93.CrossRefGoogle Scholar
  29. 29.
    Frank JR, Snell LS, Cate OT, Holmboe ES, Carraccio C, Swing SR, et al. Competency-based medical education: theory to practice. Med Teach 2010;32(8):638-45.CrossRefGoogle Scholar
  30. 30.
    Carraccio C, Englander R, Van Melle E, Ten Cate O, Lockyer J, Chan MK, et al. Advancing competency-based medical education: a charter for clinician-educators. Acad Med 2016;91(5):645-9.CrossRefGoogle Scholar
  31. 31.
    Weinberger SE, Pereira AG, Iobst WF, Mechaber AJ, Bronze MS, Alliance for Academic Internal Medicine Education Redesign Task F, II. Competency-based education and training in internal medicine. Ann Intern Med 2010;153(11):751-6.CrossRefGoogle Scholar
  32. 32.
    Long DM. Competency-based residency training: the next advance in graduate medical education. Acad Med 2000;75(12):1178-83.CrossRefGoogle Scholar
  33. 33.
    Holmboe ES, Call S, Ficalora RD. Milestones and competency-based medical education in internal medicine. JAMA Intern Med 2016;176(11):1601-2.CrossRefGoogle Scholar
  34. 34.
    Accreditation Council of Graduate Medical Education website. Accessed January 2, 2019
  35. 35.
    Dreyfus H, Dreyfus SE. Five Steps From Novice to Expert: Mind Over Machine. New York: Free Press; 1988.Google Scholar

Copyright information

© Society of General Internal Medicine 2019

Authors and Affiliations

  • Shahnaz Sultan
    • 1
    • 2
  • Rebecca L. Morgan
    • 3
  • M. Hassan Murad
    • 4
    Email author
  • Yngve Falck-Ytter
    • 5
  • Philipp Dahm
    • 2
    • 6
  • Holger J. Schünemann
    • 7
  • Reem A. Mustafa
    • 8
  1. 1.Department of Medicine, Division of Gastroenterology, Hepatology, and NutritionUniversity of MinnesotaMinneapolisUSA
  2. 2.Minneapolis Veterans Affairs Healthcare SystemMinneapolisUSA
  3. 3.Department of Health Research Methods, Evidence, and Impact, Health Sciences CentreMcMaster UniversityHamiltonCanada
  4. 4.Evidence-based Practice Center Mayo ClinicRochesterUSA
  5. 5.Case Western Reserve University, University Hospitals and VA Northeast Ohio Healthcare SystemClevelandUSA
  6. 6.Department of UrologyUniversity of MinnesotaMinneapolisUSA
  7. 7.Department of Health Research Methods, Evidence and Impact, Department of MedicineMcMaster UniversityHamiltonCanada
  8. 8.Department of MedicineUniversity of Kansas Health SystemKansas CityUSA

Personalised recommendations