Angiographic core laboratory reproducibility analyses: implications for planning clinical trials using coronary angiography and left ventriculography end-points

  • Terje K. Steigen
  • Cheryl Claudio
  • David Abbott
  • Michael Schulzer
  • Jeff Burton
  • Wayne Tymchak
  • Christopher E. Buller
  • G. B. John Mancini
Original Paper


Objectives To assess reproducibility of core laboratory performance and impact on sample size calculations. Background Little information exists about overall reproducibility of core laboratories in contradistinction to performance of individual technicians. Also, qualitative parameters are being adjudicated increasingly as either primary or secondary end-points. The comparative impact of using diverse indexes on sample sizes has not been previously reported. Methods We compared initial and repeat assessments of five quantitative parameters [e.g., minimum lumen diameter (MLD), ejection fraction (EF), etc.] and six qualitative parameters [e.g., TIMI myocardial perfusion grade (TMPG) or thrombus grade (TTG), etc.], as performed by differing technicians and separated by a year or more. Sample sizes were calculated from these results. TMPG and TTG were also adjudicated by a second core laboratory. Results MLD and EF were the most reproducible, yielding the smallest sample size calculations, whereas percent diameter stenosis and centerline wall motion require substantially larger trials. Of the qualitative parameters, all except TIMI flow grade gave reproducibility characteristics yielding sample sizes of many 100’s of patients. Reproducibility of TMPG and TTG was only moderately good both within and between core laboratories, underscoring an intrinsic difficulty in assessing these. Conclusions Core laboratories can be shown to provide reproducibility performance that is comparable to performance commonly ascribed to individual technicians. The differences in reproducibility yield huge differences in sample size when comparing quantitative and qualitative parameters. TMPG and TTG are intrinsically difficult to assess and conclusions based on these parameters should arise only from very large trials.


Angiographic core laboratory Clinical trials Reproducibility Sample size Coronary angiography Left ventriculography 


  1. 1.
    Bolson EL, Kilman S, Sheehan FH, Dodge HT (1980) Left ventricular segmental wall motion: a new method using local direction information. Proceedings of computers in cardiology, IEEE Press, pp 245–248Google Scholar
  2. 2.
    Mancini GBJ, Simon SB, McGillem MJ, Lefree MT, Friedman HZ, Vogel RA (1987) Automated quantitative coronary arteriography: morphologic and physiologic validation in vivo of a rapid digital angiographic method. Circulation 75:452–460PubMedGoogle Scholar
  3. 3.
    Mancini GB, Yeoh E, Kamimura C, Abbott D (2001) A comparison of quantitative coronary angiography systems using a unique set of in vivo coronary stenosis images. Can J Cardiol 17:785–791PubMedGoogle Scholar
  4. 4.
    Gibson CM, Cannon CP, Daley WL, Dodge JT Jr, Alexander B Jr, Marble SJ, McCabe CH, Raymond L, Fortin T, Poole WK, Braunwald E (1996) TIMI frame count: a quantitative method of assessing coronary artery flow. Circulation 93:879–888PubMedGoogle Scholar
  5. 5.
    Kalbfleisch SJ, McGillem MJ, Pinto IM, Kavanaugh KM, DeBoe SF, Mancini GB (1990) Comparison of automated quantitative coronary angiography with caliper measurements of percent diameter stenosis. Am J Cardiol 65:1181–1184PubMedCrossRefGoogle Scholar
  6. 6.
    Baim DS, Grossman W (2000) Cardiac catheterization, angiography, and intervention, 6th edn. Lippincott Williams & Wilkins, Philadelphia, pp 766–767 Google Scholar
  7. 7.
    Bashore TM (ed) (1990) Invasive cardiology, principles and techniques. BC Decker Inc, TorontoGoogle Scholar
  8. 8.
    The Thrombolysis in Myocardial Infarction (TIMI) trial (1985) Phase I findings. TIMI Study Group. N Engl J Med 312:932–936Google Scholar
  9. 9.
    Gibson CM, Ryan K, Sparano A, Moynihan JL, Rizzo M, Kelley M, Marble SJ, Laham R, Simons M, McClusky TR, Dodge JT Jr (1999) Angiographic methods to assess human coronary angiogenesis. Am Heart J 137:169–179PubMedCrossRefGoogle Scholar
  10. 10.
    Gibson CM, Cannon CP, Murphy SA, Ryan KA, Mesley R, Marble SJ, McCabe CH, Van De Werf F, Braunwald E (2000) Relationship of TIMI myocardial perfusion grade to mortality after administration of thrombolytic drugs. Circulation 101:125–130PubMedGoogle Scholar
  11. 11.
    Gibson CM, de Lemos JA, Murphy SA, Marble SJ, McCabe CH, Cannon CP, Antman EM, Braunwald E (2001) Combination therapy with Abciximab reduces angiographically evident thrombus in acute myocardial infarction: A TIMI 14 substudy. Circulation 103:2550–2554PubMedGoogle Scholar
  12. 12.
    Bland JM, Altman DG (1986) Comparing methods of measurement: why plotting difference against standard method is misleading. Lancet 346:1085–1087CrossRefGoogle Scholar
  13. 13.
    Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20:37–46CrossRefGoogle Scholar
  14. 14.
    Sim J, Wright CC (2005) The kappa statistic in reliability studies. Use, interpretation, and sample size requirements. Phys Ther 85:257–268PubMedGoogle Scholar
  15. 15.
    Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174. (in Sim J, Wright CC. The Kappa statistic in reliability studies: use, interpretation, and sample size requirements)PubMedCrossRefGoogle Scholar
  16. 16.
    Winer BJ, Brown DR, Michels KM (1991) Statistical principles in experimental design. 3rd edn. McGraw Hill, Boston, pp 92–100Google Scholar
  17. 17.
    Fleiss JL (1999) The design and analysis of clinical experiments. Wiley, NewYorkGoogle Scholar
  18. 18.
    Machin D, Campbell M, Fayers P, Pinol A (1997) Sample size tables for clinical studies, 2nd edn. Blackwell Science, OxfordGoogle Scholar
  19. 19.
    International Organization for Standardization (1993) Guide to the expression of uncertainty in measurement, 1st edn. ISO/TAG/WG3; ISO: Geneva, SwitzerlandGoogle Scholar
  20. 20.
    Moer R, Van Weert AWM, Myreng Y, Mølstad P (2003) Variability of quantitative coronary angiography: an evaluation of on-site versus core laboratory analysis. Int J Cardiovasc Imaging 19(6):457–464PubMedCrossRefGoogle Scholar
  21. 21.
    Popma JJ, Lansky AJ, Yeh W, Kennard ED, Keller MB, Merritt AJ, DeFalco RA, Desai A, Pacera JH, Schnabel JF, Niedermeyer V, Baim DS, Detre KM (1997) Reliability of the quantitative angiographic measurements in the new approaches to coronary intervention (NACI) registry: a comparison of clinical site and repeated angiographic core laboratory readings. Am J Cardiol 80(10A):19K–25KPubMedCrossRefGoogle Scholar
  22. 22.
    Brown BG, Hillger LA, Lewis C, Zhao X-Q, Sacco D, Bisson B, Fisher L (1993) Quantitative imaging, risk factors, prevalence, and change: chairman’s discussion of session 2: a maximum confidence approach for measuring progression and regression of coronary artery disease in clinical trials. Circulation 87(3S):II66–II73PubMedGoogle Scholar
  23. 23.
    Reiber JHC, Serruys PW, Kooijman CJ, Wijns W, Slager CJ, Gerbrands JJ, Schuurbiers JCH, den Boer A, Hugenholtz PG (1985) Assessment of short-, medium-, and long-term variations in arterial dimensions from computer-assisted quantitation of coronary cineangiograms. Circulation 71:280–288PubMedGoogle Scholar
  24. 24.
    Herrman JP, Azar A, Umans VA, Boersma E, von Es GA, Serruys PW (1996) Inter- and intra-observer variability in the qualitative categorization of coronary angiograms. Int J Cardiovasc Imaging 12(1):21–30CrossRefGoogle Scholar
  25. 25.
    Beauman GJ, Vogel RA (1990) Accuracy of individual and panel visual interpretations of coronary arteriograms: implications for clinical decision making. J Am Coll Cardiol 16(1):108–113PubMedCrossRefGoogle Scholar
  26. 26.
    van ‘t Hof AW, Liem A, Suryapranata H, Hoorntje JC, de Boer MJ, Zijlstra F (1998) Angiographic assessment of myocardial reperfusion in patients treated with primary angioplasty for acute myocardial infarction: myocardial blush grade. Zwolle Myocardial Infarction Study Group. Circulation 97(23):2302–2306PubMedGoogle Scholar
  27. 27.
    Stone GW, Peterson MA, Lansky AJ, Dangas G, Mehran R, Leon MB (2002) Impact of normalized myocardial perfusion after successful angioplasty in acute myocardial infarction. J Am Coll Cardiol 39(4):591–597PubMedCrossRefGoogle Scholar
  28. 28.
    Costantini CO, Stone GW, Mehran R, Aymong E, Grines CL, Cox DA, Stuckey T, Turco M, Gersh BJ, Tcheng JE, Garcia E, Griffin JJ, Guagliumi G, Leon MB, Lansky AJ (2004) Frequency, correlates, and clinical implications of myocardial perfusion after primary angioplasty and stenting, with and without glycoprotein IIb/IIIa inhibition, in acute myocardial infarction. J Am Coll Cardiol 44(2):305–312PubMedCrossRefGoogle Scholar
  29. 29.
    Gibson CM, Cannon CP, Murphy SA, Ryan KA, Mesley R, Marble SJ, McCabe CH, Van De Werf F, Braunwald E (2000) Relationship of TIMI myocardial perfusion grade to mortality after administration of thrombolytic drugs. Circulation 101(2):125–130PubMedGoogle Scholar
  30. 30.
    Kirtane AJ, Bui A, Murphy SA, Karmpaliotis D, Kosmidou I, Boundy K, Rahman A, Pinto DS, Aroesty JM, Giugliano RP, Cannon CP, Antman EM, Gibson CM (2004) TIMI Study Group. Association of epicardial and tissue-level reperfusion with left ventricular end-diastolic pressures in ST-elevation myocardial infarction. J Thromb Thrombolysis 17(3):177–184PubMedCrossRefGoogle Scholar
  31. 31.
    Gibson CM, Cannon CP, Murphy SA, Marble SJ, Barron HV, Braunwald E (2002) TIMI Study Group. Relationship of the TIMI myocardial perfusion grades, flow grades, frame count, and percutaneous coronary intervention to long-term outcomes after thrombolytic administration in acute myocardial infarction. Circulation 105(16):1909–1913PubMedCrossRefGoogle Scholar
  32. 32.
    Gibson CM, Schomig A (2004) Coronary and myocardial angiography: angiographic assessment of both epicardial and myocardial perfusion. Circulation 109(25):3096–3105PubMedCrossRefGoogle Scholar
  33. 33.
    Gibson CM, Murphy SA, Morrow DA, Aroesty JM, Gibbons RJ, Gourlay SG, Barron HV, Giugliano RP, Antman EM, Braunwald E (2004) Angiographic perfusion score: an angiographic parameter that integrates both epicardial and tissue level perfusion before and after facilitated percutaneous coronary intervention in acute myocardial infarction. Am Heart J 148(2):336–340PubMedCrossRefGoogle Scholar
  34. 34.
    Gibson CM, Kirtane AJ, Boundy K, Ly H, Karmpaliotis D, Murphy SA, Giugliano RP, Cannon CP, Antman EM, Braunwald E (2005) TIMI Study Group. Association of a negative residual stenosis following rescue/adjunctive percutaneous coronary intervention with impaired myocardial perfusion and adverse outcomes among ST-segment elevation myocardial infarction patients. J Am Coll Cardiol 45(3):357–362PubMedCrossRefGoogle Scholar
  35. 35.
    Gibson CM, de Lemos JA, Murphy SA, Marble SJ, Dauterman KW, Michaels A, Barron HV, Antman EM (2002) TIMI Study Group. Methodologic and clinical validation of the TIMI myocardial perfusion grade in acute myocardial infarction. J Thromb Thrombolysis 14(3):233–237PubMedCrossRefGoogle Scholar
  36. 36.
    Tuinenburg JC, Koning G, Hekking E, Desjardins C, Harel F, Bilodeau L, van Weert AW, Lesperance J, Reiber JH (2002) One core laboratory at two international sites, is that feasible? An inter-core laboratory and intra-observer variability study. Catheter Cardiovasc Interv 56(3):333–340PubMedCrossRefGoogle Scholar
  37. 37.
    Beauman GJ, Reiber JH, Koning G, Vogel RA (1996) Comparisons of angiographic core laboratory analyses of phantom and clinical images: interlaboratory variability. Cathet Cardiovasc Diagn 37(1):24–31PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2007

Authors and Affiliations

  • Terje K. Steigen
    • 1
    • 2
  • Cheryl Claudio
    • 1
  • David Abbott
    • 1
  • Michael Schulzer
    • 3
  • Jeff Burton
    • 4
  • Wayne Tymchak
    • 4
  • Christopher E. Buller
    • 1
  • G. B. John Mancini
    • 1
  1. 1.Cardiovascular Imaging Research Core Laboratory, Division of CardiologyVancouver HospitalVancouverCanada
  2. 2.University of TromsoTromsoNorway
  3. 3.Department of MedicineUniversity of British Columbia, Vancouver HospitalVancouverCanada
  4. 4.Division of CardiologyUniversity of AlbertaEdmontonCanada

Personalised recommendations