Advertisement

Canadian Journal of Anaesthesia

, Volume 47, Issue 9, pp 914–918 | Cite as

Performance on ABA-ASA in-training examination predicts success for RCPSC certification

  • Ramona A. Kearney
  • Patrick Sullivan
  • Ernest Skakun
Education

Abstract

Purpose: Most Canadian University Departments of Anesthesia require residents to take the American Board of Anesthesiology-American Society of Anesthesiologists (ABA-ASA) in-training examination (ITE). The result is expressed as a percentile relative to all examinees at similar levels of training. Its value as a predictor of performance in the Royal College of Physicians and Surgeons of Canada (RCPSC) certification examinations is not known.

Methods: All English speaking Canadian residency programs in Anesthesia were surveyed. Results of the ABA-ASA ITE of former residents who had completed RCPSC certification were collected as percentile scores according to level of training. Level of training was based on the number of months of anesthesia training and classified according to American residency program nomenclature. The ABA-ASA ITE scores were correlated with success on the RCPSC written and oral examinations. The probability of success on the RCPSC examinations was determined by calculating the cutoff score with the best sensitivity and specificity as determined by Receiver Operating Characteristic (ROC) curves for each year in which the examination was taken and for both the written and oral examinations.

Results: Nine residency programs provided information on 165 residents. A weak positive correlation was found between scores on each year of the ABA-ASA exam. Scores >50th percentile for any year were highly predictive of success in the written component (>60th percentile for the oral component). Scores<20th percentile were predictive of failure on both the written and oral components of the RCPSC examination.

Conclusion: The ABA-ASA ITE is a useful tool in predicting performance on the RCPSC examination.

Keywords

Residency Program Oral Examination Pretest Probability Percentile Score Certification Examination 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Résumé

Objectif: La plupart des départements d’anesthésie des universités canadiennes exigent de leurs résidents qu’ils se présentent à l’examen américain des résidents en cours de formation de l7rsAmerican Board of Anesthesiology-American Society of Anesthesiologists (ABA-ASA) in-training examination (ITE). Le résultat est exprimé en percentile relatif au nombre total de candidats au même niveau de formation. Sa valeur prédictive de performance à l’examen de certification du Collège royal des médecins et chirurgiens du Canada (CRMCC) n’est pas connue.

Méthode: Tous les programmes canadiens anglais de résidence en anesthésie ont été étudiés. Les résultats de l’ABA-ASA ITE des anciens résidents qui se sont présentés à l’examen du CRMCC ont été recueillis et cotés selon le niveau de formation. Le niveau de formation a été fondé sur le nombre de mois de formation en anesthésie et classé selon la nomenclature du programme américan de résidence. La corrélation des scores d’ABA-ASA ITE avec les examens écrits et oraux du CRMCC a été un succès. La probabilité de réussite des examens du CRMCC a été déterminée par le calcul de la cote minimale ou maximale du meilleur taux de sensibilité et de spécificité établis au moyen de l’analyse par les courbes ROC pour chaque année où l’examen a été fait et pour les examens oraux et écrits.

Résultats: Neuf programmes de résidence ont fourni des renseignements sur 165 résidents. Une faible corrélation positive a été découverte entre les scores de chaque année d’examen de l’ABA-ASA. Les scores >50e percentile, quelle que soit l’année, ont été fortement prédictifs de succès pour la composante écrite de l’examen et les scores >60e percentile, pour la composante orale. Les scores <20e percentile ont été prédictifs d’échecs aux examens oraux et écrits du CRMCC.

Conclusion: L’ABA-ASA ITE est utile pour prédire la performance à l’examen du CRMCC.

References

  1. 1.
    General Standards of Accreditation, The Royal College Of Physicians And Surgeons of Canada, September 1997.Google Scholar
  2. 2.
    Holmboe ES, Hawkins RE. Methods for evaluating the clinical competence of residents in Internal Medicine: a review. Ann Intern Med 1998; 129: 42–8.PubMedGoogle Scholar
  3. 3.
    Waxman H, Braunstein G, Dantzker D, et al. Performance on the Internal Medicine second-year residency In-training Examination predicts the outcome of the ABIM certifying examination. J Gen Intern Med 1994; 9: 692–4.PubMedCrossRefGoogle Scholar
  4. 4.
    Grossman RS, Fincher R-ME, Layne RD, Seelig CB, Berkowitz LR, Levine MA. Validity of the In-Training Examination for predicting American Board of Internal Medicine certifying examination scores. J Gen Intern Med 1992; 7: 63–7.PubMedCrossRefGoogle Scholar
  5. 5.
    Leigh TM, Johnson TP, Pisacano NJ. Predictive validity of the American Board of Family Practice In- Training Examination. Acad Med 1990; 65: 454–7.PubMedCrossRefGoogle Scholar
  6. 6.
    Shetler PL. Observations on the American Board of Surgery In-Training Examination, board results, and conference attendance. Am J Surg 1982; 144: 292–4.PubMedCrossRefGoogle Scholar
  7. 7.
    Webb LC, Juul D, Reynolds CF III,et al. How well does the Psychiatry residency In-Training Examination predict performance on the American Board of Psychiatry and Neurology Part I examination? Am J Psychiatr 1996; 153: 831–2.PubMedGoogle Scholar
  8. 8.
    Streiner DL, Norman GR. Health Measurement Scales, 2nd ed. New York: Oxford University Press Inc., 1995.Google Scholar
  9. 9.
    Butzin DW, Guerin RO, Oliver TK, Stockman JA, Da Dalt L, Perilongo G Administering the American Board Of Pediatrics In-Training Examination in a European Pediatrics residency (Letter). Acad Med 1996; 71: 393–4.PubMedCrossRefGoogle Scholar

Copyright information

© Canadian Anesthesiologists 2000

Authors and Affiliations

  • Ramona A. Kearney
    • 1
  • Patrick Sullivan
    • 1
    • 2
  • Ernest Skakun
    • 1
    • 2
  1. 1.From the Departments of Anesthesiology and Pain Medicine, Division of Studies in Medical EducationUniversity of AlbertaCanada
  2. 2.University of OttawaCanada

Personalised recommendations