Advertisement

The Correlation Between Students’ Progress Testing Scores and Their Performance in a Residency Selection Process

  • Pedro Tadao Hamamoto FilhoEmail author
  • Pedro Luiz Toledo de Arruda Lourenção
  • Adriana Polachini do Valle
  • Joélcio Francisco Abbade
  • Angélica Maria Bicudo
Original Research
  • 1 Downloads

Abstract

Brazil is currently seeing an increased number of medical schools, leading to high competition for medical residency vacancies. Public managers have thus considered Progress Testing scores potentially useful as part of the final decision in the medical residency selection process. We analyzed whether there is a correlation between students’ Progress Testing scores and their performances in medical residency selection. We examined four subsequent cohorts of students who attempted Progress Testing yearly and compared their accumulated scores with their medical residency selection scores from Botucatu Medical School, Universidade Estadual Paulista. We included 212 students who finished the 6-year medical course in 2013, 2014, 2015, and 2016. The comparison between the area under the Progress Testing curve and the medical residency selection score was performed using a Pearson correlation, with a p value set at < 0.05. We found a positive association between the two scores (p < 0.05 for the 4 years). Next, the students were grouped according to their performance in Progress Testing: above one, within one, and below one standard deviation. A chi-square test was used to compare the rates of approval with the second step of the medical residency selection process. Approval rates were 91.7%, 69.2%, and 42.1%, respectively (p < 0.05). We conclude that, in fact, there is a correlation between students’ performance on these measures. This is partially explained by the fact that both instruments measure cognitive competencies and knowledge. These data may support national policy changes for medical residency selection.

Keywords

Progress testing Medical residency selection Undergraduate medical education Assessment 

Notes

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

Institutional Review Board at Botucatu Medical School

Informed Consent

NA

References

  1. 1.
    Schuwirth LWT, van der Vleuten CPM. The use of progress testing. Perspect Med Educ. 2012;1:24–30.CrossRefGoogle Scholar
  2. 2.
    Arnold L, Willoughby TL. The quarterly profile examination. Acad Med. 1990;65(8):515–6.CrossRefGoogle Scholar
  3. 3.
    Van der Vleuten CPM, Verwijnen GM, Wijnen WHFW. Fifteen years of experience with progress testing in a problem-based learning curriculum. Med Teach. 1996;18:103–9.CrossRefGoogle Scholar
  4. 4.
    Freeman A, van der Vleuten CPM, Nouns Z, Ricketts C. Progress testing internationally. Med Teach. 2010;32:451–5.CrossRefGoogle Scholar
  5. 5.
    Albanese M, Case SM. Progress testing: critical analysis and suggested practices. Adv Health Sci Educ. 2016;21:221–34.CrossRefGoogle Scholar
  6. 6.
    Albano MG, Cavallo F, Hoogenboom R, Magni F, Majoor G, Manenti F, et al. An international comparison of knowledge levels of medical students: the Maastricht Progress Test. Med Educ. 1996;30:239–45.CrossRefGoogle Scholar
  7. 7.
    Tomic ER, Martins MA, Lotufo PA, Benseñor IM. Progress testing: evaluation of four years of application in the School of Medicine, University of São Paulo. Clinics. 2005;60:389–96.CrossRefGoogle Scholar
  8. 8.
    Nouns ZM, Georg W. Progress testing in German speaking countries. Med Teach. 2010;32:467–70.CrossRefGoogle Scholar
  9. 9.
    Finucane P, Flannery D, Keane D, Norman G. Cross-institutional progress testing: feasibility and value to a new medical school. Med Educ. 2010;44:184–6.CrossRefGoogle Scholar
  10. 10.
    World Federation of Medical Education, Foundation for Advancement of International Medical Education and Research. World directory of medical schools. 2017.Google Scholar
  11. 11.
    Amaral JLG, Pêgo-Fernandes PM, Bibas BJ. More doctors: better attention to the population’s health? Sao Paulo Med J. 2012;130:139–49.CrossRefGoogle Scholar
  12. 12.
    Santos LMP, Oliveira A, Trindade JS, Barreto ICHC, Palmeira PA, Comes Y, et al. Implementation research: towards universal health coverage with more doctors in Brazil. Bull World Health Organ. 2017;95:103–12.CrossRefGoogle Scholar
  13. 13.
    Ferreira EAB, Rasslan S. Surgical education in Brazil. World J Surg. 2010;34:880–3.CrossRefGoogle Scholar
  14. 14.
    Brazil. Ministry of Education. Resolution CNRM no. 003/2011, 11/16/2011. Dispõe sobre o processo de seleção pública dos candidatos aos Programas de Residência Médica. Available at: em: http://portal.mec.gov.br/index.php?option=com_docman&view=download&alias=8950-res-cnrm-03-2011&category_slug=setembro-2011-pdf&Itemid=30192. Accessed 25 July 2019.
  15. 15.
    Goldwasser RS. Selection of medical residents based on hands-on skills. Rev Bras Educ Méd. 2006;30:115–24.Google Scholar
  16. 16.
    Bicudo AM, Hamamoto Filho PT, Abbade JF, Hafner MLMB, Maffei CM. Teste de Progresso em consóricos para todas as escolas médicas do Brasil. [Consortia of cross-institutional Progress Testing for all medical schools in Brazil]. Rev Bras Educ Méd. 2019; Ahead of print.Google Scholar
  17. 17.
    Cecilio-Fernandes D, Kerdijk W, Jaarsma AD, Tio RA. Development of cognitive processing and judgments of knowledge in medical students: analysis of progress test results. Med Teach. 2016;38:1125–9.CrossRefGoogle Scholar
  18. 18.
    Plessas A. Validity of progress testing in healthcare education. Int J Human Soc Sci Educ. 2015;2:23–33.Google Scholar
  19. 19.
    Tio R, Schutte B, Meiboom AA, Greidanus J, Dubois EA, Bremers AJ. The progress test of medicine: the Dutch experience. Perspect Med Educ. 2016;5:51–5.CrossRefGoogle Scholar
  20. 20.
    Karay Y, Schauber SK. A validity argument for progress testing: examining the relation between growth trajectories obtained by progress tests and national licensing examinations using a latent growth curve approach. Med Teach. 2018;40:1123–9.CrossRefGoogle Scholar
  21. 21.
    Willoughby TL, Dimond EG, Smull NW. Correlation of quarterly profile examination and National Board of Medical Examiner Scores. Educ Psychol Meas. 1977;37:445–9.CrossRefGoogle Scholar
  22. 22.
    Blake JM, Norman GR, Keane DR, Mueller B, Cunnington J, Didyk N. Introducing progress testing in McMaster University’s problem-based curriculum: psychometric properties and effect of learning. Acad Med. 1996;71:1002–7.CrossRefGoogle Scholar
  23. 23.
    Neeley SM, Ulman CA, Sydelko BS, Borges NJ. The value of progress testing in undergraduate medical education: a systematic review of the literature. Med Sci Educ. 2016;26:617–22.CrossRefGoogle Scholar
  24. 24.
    Santos IS, Vieira JE, Nunes MPT. Length of internship influences performance on medical residency selection. Rev Assoc Med Bras. 2009;55:744–8.CrossRefGoogle Scholar
  25. 25.
    Pugh D, Touchie C, Humphrey-Murto S, Wood TJ. The OSCE progress test – measuring clinical skill development over residency training. Med Teach. 2016;38:168–73.CrossRefGoogle Scholar
  26. 26.
    Pugh D, Bhanji F, Cole G, Dupre J, Hatala R, Humphrey-Murto S, et al. Do OSCE progress test score predict performance in a national high-stake examination? Med Educ. 2016;50:351–8.CrossRefGoogle Scholar
  27. 27.
    Coombes L, Ricketts C, Freeman A, Stratford J. Beyond assessment: feedback for individuals and institutions based on the progress test. Med Teach. 2010;32:486–90.CrossRefGoogle Scholar
  28. 28.
    Norman GR, Neville A, Blake JM, Mueller B. Assessment steers learning down the right road: impact of progress testing on licensing examination performance. Med Teach. 2010;32:496–9.CrossRefGoogle Scholar
  29. 29.
    Mota A, Carvalho B, Candido L, Lomanto R, Maia T. The São Paulo state regional medical council exam as an indicator of quality in medical education. Rev Bras Educ Méd. 2014;38:150–9.CrossRefGoogle Scholar
  30. 30.
    Hamamoto Filho PT, Zeferino AMB. Preparatory courses for medical residency: reflections on possible causes and consequences. Rev Bras Educ Méd. 2011;35:550–6.CrossRefGoogle Scholar
  31. 31.
    Van der Vleuten C, Freeman A, Collares CF. Progress test utopia. Perspect Med Educ. 2018;7:136–8.CrossRefGoogle Scholar

Copyright information

© International Association of Medical Science Educators 2019

Authors and Affiliations

  1. 1.Botucatu Medical School, Department of Neurology, Psychology and PsychiatryUNESP – Univ Estadual PaulistaBotucatuBrazil
  2. 2.Botucatu Medical School, Department of Surgery an OrthopedicsUNESP – Univ Estadual PaulistaBotucatuBrazil
  3. 3.Botucatu Medical School, Department of Internal MedicineUNESP – Univ Estadual PaulistaBotucatuBrazil
  4. 4.Botucatu Medical School, Department of Gynecology and ObstetricsUNESP – Univ Estadual PaulistaBotucatuBrazil
  5. 5.School of Medical Sciences, Department of PediatricsUNICAMP – Univ Estadual de CampinasCampinasBrazil

Personalised recommendations