Does No Difference Really Mean No Difference?

  • Carola F. van Eck
  • Marcio Bottene Villa Albers
  • Andrew J. Sheean
  • Freddie H. FuEmail author


The most common statistical error in orthopedic research is a type II (or “beta”) error. This occurs when a study incorrectly concludes that there is “no difference” between the test and control treatment. Finding no difference does not always mean there is no difference. This chapter will review the different causes of type II errors and provide tips for avoiding these errors when conducting research and interpreting the literature. The prevalence of type II errors in orthopedic research underscores the necessity of more objective outcome measurements and randomized studies with large numbers of patients.


  1. 1.
    Ahlden M, Araujo P, Hoshino Y, et al. Clinical grading of the pivot shift test correlates best with tibial acceleration. Knee Surg Sports Traumatol Arthrosc. 2012;20(4):708–12.CrossRefGoogle Scholar
  2. 2.
    Andriacchi TP, Briant PL, Bevill SL, Koo S. Rotational changes at the knee after ACL injury cause cartilage thinning. Clin Orthop Relat Res. 2006;442:39–44.CrossRefGoogle Scholar
  3. 3.
    Barber-Westin SD, Noyes FR, McCloskey JW. Rigorous statistical reliability, validity, and responsiveness testing of the Cincinnati knee rating system in 350 subjects with uninjured, injured, or anterior cruciate ligament-reconstructed knees. Am J Sports Med. 1999;27(4):402–16.CrossRefGoogle Scholar
  4. 4.
    Bollen S, Seedhom BB. A comparison of the Lysholm and Cincinnati knee scoring questionnaires. Am J Sports Med. 1991;19(2):189–90.CrossRefGoogle Scholar
  5. 5.
    Carey JL, Dunn WR, Dahm DL, Zeger SL, Spindler KP. A systematic review of anterior cruciate ligament reconstruction with autograft compared with allograft. J Bone Joint Surg Am. 2009;91(9):2242–50.CrossRefGoogle Scholar
  6. 6.
    Chavez I, Dorbecker N, Celis A. Direct intracardiac angiocardiography; its diagnostic value. Am Heart J. 1947;33(5):560–93.CrossRefGoogle Scholar
  7. 7.
    Chu CR, Williams AA, West RV, et al. Quantitative magnetic resonance imaging UTE-T2* mapping of cartilage and meniscus healing after anatomic anterior cruciate ligament reconstruction. Am J Sports Med. 2014;42(8):1847–56. Scholar
  8. 8.
    Desai N, Alentorn-Geli E, van Eck CF, et al. A systematic review of single-versus double-bundle ACL reconstruction using the anatomic anterior cruciate ligament reconstruction scoring checklist. Knee Surg Sports Traumatol Arthrosc. 2016;24(3):862–72.CrossRefGoogle Scholar
  9. 9.
    Eastlack ME, Axe MJ, Snyder-Mackler L. Laxity, instability, and functional outcome after ACL injury: copers versus noncopers. Med Sci Sports Exerc. 1999;31(2):210–5.CrossRefGoogle Scholar
  10. 10.
    Forsythe B, Kopf S, Wong AK, et al. The location of femoral and tibial tunnels in anatomic double-bundle anterior cruciate ligament reconstruction analyzed by three-dimensional computed tomography models. J Bone Joint Surg Am. 2010;92(6):1418–26.CrossRefGoogle Scholar
  11. 11.
    Foster TE, Wolfe BL, Ryan S, Silvestri L, Kaye EK. Does the graft source really matter in the outcome of patients undergoing anterior cruciate ligament reconstruction? An evaluation of autograft versus allograft reconstruction results: a systematic review. Am J Sports Med. 2010;38(1):189–99.CrossRefGoogle Scholar
  12. 12.
    Frobell RB, Roos EM, Roos HP, Ranstam J, Lohmander LS. A randomized trial of treatment for acute anterior cruciate ligament tears. N Engl J Med. 2010;363(4):331–42.CrossRefGoogle Scholar
  13. 13.
    Fujimoto E, Sumen Y, Deie M, Yasumoto M, Kobayashi K, Ochi M. Anterior cruciate ligament graft impingement against the posterior cruciate ligament: diagnosis using MRI plus three-dimensional reconstruction software. Magn Reson Imaging. 2004;22(8):1125–9.CrossRefGoogle Scholar
  14. 14.
    Gabriel MT, Wong EK, Woo SL, Yagi M, Debski RE. Distribution of in situ forces in the anterior cruciate ligament in response to rotatory loads. J Orthop Res. 2004;22(1):85–9.CrossRefGoogle Scholar
  15. 15.
    Gillquist J, Messner K. Anterior cruciate ligament reconstruction and the long-term incidence of gonarthrosis. Sports Med. 1999;27(3):143–56.CrossRefGoogle Scholar
  16. 16.
    Harner CD, Irrgang JJ, Paul J, Dearwater S, Fu FH. Loss of motion after anterior cruciate ligament reconstruction. Am J Sports Med. 1992;20(5):499–506.CrossRefGoogle Scholar
  17. 17.
    Harner CD, Marks PH, Fu FH, Irrgang JJ, Silby MB, Mengato R. Anterior cruciate ligament reconstruction: endoscopic versus two-incision technique. Arthroscopy. 1994;10(5):502–12.CrossRefGoogle Scholar
  18. 18.
    Harner CD, Olson E, Irrgang JJ, Silverstein S, Fu FH, Silbey M. Allograft versus autograft anterior cruciate ligament reconstruction: 3- to 5-year outcome. Clin Orthop Relat Res. 1996;324:134–44.CrossRefGoogle Scholar
  19. 19.
    Heckman JD. Are validated questionnaires valid? J Bone Joint Surg Am. 2006;88(2):446.PubMedGoogle Scholar
  20. 20.
    Hefti F, Muller W. [Current state of evaluation of knee ligament lesions. The new IKDC knee evaluation form ]. Orthopade. 1993;22(6):351–62.Google Scholar
  21. 21.
    Hoher J, Munster A, Klein J, Eypasch E, Tiling T. Validation and application of a subjective knee questionnaire. Knee Surg Sports Traumatol Arthrosc. 1995;3(1):26–33.CrossRefGoogle Scholar
  22. 22.
    Holm I, Oiestad BE, Risberg MA, Aune AK. No difference in knee function or prevalence of osteoarthritis after reconstruction of the anterior cruciate ligament with 4-strand hamstring autograft versus patellar tendon-bone autograft: a randomized study with 10-year follow-up. Am J Sports Med. 2010;38(3):448–54.CrossRefGoogle Scholar
  23. 23.
    Illingworth KD, Hensler D, Working ZM, Macalena JA, Tashman S, Fu FH. A simple evaluation of anterior cruciate ligament femoral tunnel position: the inclination angle and femoral tunnel angle. Am J Sports Med. 2011;39(12):2611–8.CrossRefGoogle Scholar
  24. 24.
    Iriuchishima T, Tajima G, Ingham SJ, et al. Intercondylar roof impingement pressure after anterior cruciate ligament reconstruction in a porcine model. Knee Surg Sports Traumatol Arthrosc. 2009;17(6):590–4.CrossRefGoogle Scholar
  25. 25.
    Iriuchishima T, Tajima G, Ingham SJ, Shen W, Smolinski P, Fu FH. Impingement pressure in the anatomical and nonanatomical anterior cruciate ligament reconstruction: a cadaver study. Am J Sports Med. 2010;38(8):1611–7.CrossRefGoogle Scholar
  26. 26.
    Irrgang JJ, Anderson AF. Development and validation of health-related quality of life measures for the knee. Clin Orthop Relat Res. 2002;402:95–109.CrossRefGoogle Scholar
  27. 27.
    Irrgang JJ, Anderson AF, Boland AL, et al. Responsiveness of the International Knee Documentation Committee Subjective Knee Form. Am J Sports Med. 2006;34(10):1567–73.CrossRefGoogle Scholar
  28. 28.
    Irrgang JJ, Bost JE, Fu FH. Re: outcome of single-bundle versus double-bundle reconstruction of the anterior cruciate ligament: a meta-analysis. Am J Sports Med. 2009;37(2):421–2; author reply 422.CrossRefGoogle Scholar
  29. 29.
    Johnson DL, Swenson TM, Irrgang JJ, Fu FH, Harner CD. Revision anterior cruciate ligament surgery: experience from Pittsburgh. Clin Orthop Relat Res. 1996;325:100–9.CrossRefGoogle Scholar
  30. 30.
    Keays SL, Newcombe PA, Bullock-Saxton JE, Bullock MI, Keays AC. Factors involved in the development of osteoarthritis after anterior cruciate ligament surgery. Am J Sports Med. 2010;38(3):455–63.CrossRefGoogle Scholar
  31. 31.
    Kocher MS, Steadman JR, Briggs K, Zurakowski D, Sterett WI, Hawkins RJ. Determinants of patient satisfaction with outcome after anterior cruciate ligament reconstruction. J Bone Joint Surg Am. 2002;84-A(9):1560–72.CrossRefGoogle Scholar
  32. 32.
    Kopf S, Forsythe B, Wong AK, et al. Nonanatomic tunnel position in traditional transtibial single-bundle anterior cruciate ligament reconstruction evaluated by three-dimensional computed tomography. J Bone Joint Surg Am. 2010;92(6):1427–31.CrossRefGoogle Scholar
  33. 33.
    Kopf S, Forsythe B, Wong AK, Tashman S, Irrgang JJ, Fu FH. Transtibial ACL reconstruction technique fails to position drill tunnels anatomically in vivo 3D CT study. Knee Surg Sports Traumatol Arthrosc. 2012;20(11):2200–7.CrossRefGoogle Scholar
  34. 34.
    Kopf S, Musahl V, Perka C, Kauert R, Hoburg A, Becker R. The influence of applied internal and external rotation on the pivot shift phenomenon. Knee Surg Sports Traumatol Arthrosc. 2017;25(4):1106–10.CrossRefGoogle Scholar
  35. 35.
    Lukianov AV, Gillquist J, Grana WA, DeHaven KE. An anterior cruciate ligament (ACL) evaluation format for assessment of artificial or autologous anterior cruciate reconstruction results. Clin Orthop Relat Res. 1987;218:167–80.Google Scholar
  36. 36.
    Lysholm J, Gillquist J. Evaluation of knee ligament surgery results with special emphasis on use of a scoring scale. Am J Sports Med. 1982;10(3):150–4.CrossRefGoogle Scholar
  37. 37.
    Markolf KL, Feeley BT, Jackson SR, McAllister DR. Biomechanical studies of double-bundle posterior cruciate ligament reconstructions. J Bone Joint Surg Am. 2006;88(8):1788–94.CrossRefGoogle Scholar
  38. 38.
    Markolf KL, Park S, Jackson SR, McAllister DR. Simulated pivot-shift testing with single and double-bundle anterior cruciate ligament reconstructions. J Bone Joint Surg Am. 2008;90(8):1681–9.CrossRefGoogle Scholar
  39. 39.
    Marx RG. Knee rating scales. Arthroscopy. 2003;19(10):1103–8.CrossRefGoogle Scholar
  40. 40.
    Marx RG, Jones EC, Allen AA, et al. Reliability, validity, and responsiveness of four knee outcome scales for athletic patients. J Bone Joint Surg Am. 2001;83-A(10):1459–69.CrossRefGoogle Scholar
  41. 41.
    Marx RG, Stump TJ, Jones EC, Wickiewicz TL, Warren RF. Development and evaluation of an activity rating scale for disorders of the knee. Am J Sports Med. 2001;29(2):213–8.CrossRefGoogle Scholar
  42. 42.
    Mayo Robson AW. Ruptured crucial ligaments and their repair by operation. Ann Surg. 1903;37(5):716–8.Google Scholar
  43. 43.
    Meredick RB, Vance KJ, Appleby D, Lubowitz JH. Outcome of single-bundle versus double-bundle reconstruction of the anterior cruciate ligament: a meta-analysis. Am J Sports Med. 2008;36(7):1414–21.CrossRefGoogle Scholar
  44. 44.
    Miyawaki M, Hensler D, Illingworth KD, Irrgang JJ, Fu FH. Signal intensity on magnetic resonance imaging after allograft double-bundle anterior cruciate ligament reconstruction. Knee Surg Sports Traumatol Arthrosc. 2014;22(5):1002–8.CrossRefGoogle Scholar
  45. 45.
    Mohtadi N. Development and validation of the quality of life outcome measure (questionnaire) for chronic anterior cruciate ligament deficiency. Am J Sports Med. 1998;26(3):350–9.CrossRefGoogle Scholar
  46. 46.
    Morimoto Y, Ferretti M, Ekdahl M, Smolinski P, Fu FH. Tibiofemoral joint contact area and pressure after single- and double-bundle anterior cruciate ligament reconstruction. Arthroscopy. 2009;25(1):62–9.CrossRefGoogle Scholar
  47. 47.
    Musahl V, Griffith C, Irrgang JJ, et al. Validation of quantitative measures of rotatory knee laxity. Am J Sports Med. 2016;44(9):2393–8.CrossRefGoogle Scholar
  48. 48.
    Musahl V, Hoshino Y, Ahlden M, et al. The pivot shift: a global user guide. Knee Surg Sports Traumatol Arthrosc. 2012;20(4):724–31.CrossRefGoogle Scholar
  49. 49.
    Musahl V, Hoshino Y, Becker R, Karlsson J. Rotatory knee laxity and the pivot shift. Knee Surg Sports Traumatol Arthrosc. 2012;20(4):601–2.CrossRefGoogle Scholar
  50. 50.
    Musahl V, Rahnemai-Azar AA, Costello J, et al. The influence of meniscal and anterolateral capsular injury on knee laxity in patients with anterior cruciate ligament injuries. Am J Sports Med. 2016;44(12):3126–31.CrossRefGoogle Scholar
  51. 51.
    Neeb TB, Aufdemkampe G, Wagener JH, Mastenbroek L. Assessing anterior cruciate ligament injuries: the association and differential value of questionnaires, clinical tests, and functional tests. J Orthop Sports Phys Ther. 1997;26(6):324–31.CrossRefGoogle Scholar
  52. 52.
    Nishimori M, Sumen Y, Sakaridani K, Nakamura M. An evaluation of reconstructed ACL impingement on PCL using MRI. Magn Reson Imaging. 2007;25(5):722–6.CrossRefGoogle Scholar
  53. 53.
    Passler HH. The history of the cruciate ligaments: some forgotten (or unknown) facts from Europe. Knee Surg Sports Traumatol Arthrosc. 1993;1(1):13–6.CrossRefGoogle Scholar
  54. 54.
    Powell WJ Jr, Wittenberg J, Dinsmore RE, Miller SW, Maturi RA. Definition of cardiac structures using computerized tomography in isolated arrested and beating canine hearts. Am J Cardiol. 1977;39(5):690–6.CrossRefGoogle Scholar
  55. 55.
    Risberg MA, Holm I, Steen H, Beynnon BD. Sensitivity to changes over time for the IKDC form, the Lysholm score, and the Cincinnati knee score. A prospective study of 120 ACL reconstructed patients with a 2-year follow-up. Knee Surg Sports Traumatol Arthrosc. 1999;7(3):152–9.CrossRefGoogle Scholar
  56. 56.
    Roos EM, Ostenberg A, Roos H, Ekdahl C, Lohmander LS. Long-term outcome of meniscectomy: symptoms, function, and performance tests in patients with or without radiographic osteoarthritis compared to matched controls. Osteoarthr Cartil. 2001;9(4):316–24.CrossRefGoogle Scholar
  57. 57.
    Roos EM, Roos HP, Lohmander LS, Ekdahl C, Beynnon BD. Knee Injury and Osteoarthritis Outcome Score (KOOS)—development of a self-administered outcome measure. J Orthop Sports Phys Ther. 1998;28(2):88–96.CrossRefGoogle Scholar
  58. 58.
    Roos EM, Roos HP, Ryd L, Lohmander LS. Substantial disability 3 months after arthroscopic partial meniscectomy: a prospective study of patient-relevant outcomes. Arthroscopy. 2000;16(6):619–26.CrossRefGoogle Scholar
  59. 59.
    Rosenberg T. Techniques for ACL reconstruction with Multi-Trac drill guide. Mansfield: Accufex Microsurgical Inc.; 1994.Google Scholar
  60. 60.
    Samuelsson K, Andersson D, Karlsson J. Treatment of anterior cruciate ligament injuries with special reference to graft type and surgical technique: an assessment of randomized controlled trials. Arthroscopy. 2009;25(10):1139–74.CrossRefGoogle Scholar
  61. 61.
    Sernert N, Kartus J, Kohler K, et al. Analysis of subjective, objective and functional examination tests after anterior cruciate ligament reconstruction. A follow-up of 527 patients. Knee Surg Sports Traumatol Arthrosc. 1999;7(3):160–5.CrossRefGoogle Scholar
  62. 62.
    Sgaglione NA. Revision ACL reconstruction. Presented at Orthopedics Today Hawaii 2011. Jan 16–19 Koloa, Hawaii; 2011.Google Scholar
  63. 63.
    Sgaglione NA, Del Pizzo W, Fox JM, Friedman MJ. Critical analysis of knee ligament rating systems. Am J Sports Med. 1995;23(6):660–7.CrossRefGoogle Scholar
  64. 64.
    Snook GA. A short history of the anterior cruciate ligament and the treatment of tears. Clin Orthop Relat Res. 1983;172:11–3.Google Scholar
  65. 65.
    Tashman S, Anderst W. In-vivo measurement of dynamic joint motion using high speed biplane radiography and CT: application to canine ACL deficiency. J Biomech Eng. 2003;125(2):238–45.CrossRefGoogle Scholar
  66. 66.
    Tashman S, Collon D, Anderson K, Kolowich P, Anderst W. Abnormal rotational knee motion during running after anterior cruciate ligament reconstruction. Am J Sports Med. 2004;32(4):975–83.CrossRefGoogle Scholar
  67. 67.
    Tashman S, Kolowich P, Collon D, Anderson K, Anderst W. Dynamic function of the ACL-reconstructed knee during running. Clin Orthop Relat Res. 2007;454:66–73.CrossRefGoogle Scholar
  68. 68.
    Tegner Y, Lysholm J. Rating systems in the evaluation of knee ligament injuries. Clin Orthop Relat Res. 1985;198:43–9.Google Scholar
  69. 69.
    van Eck CF, Gravare-Silbernagel K, Samuelsson K, et al. Evidence to support the interpretation and use of the anatomic anterior cruciate ligament reconstruction checklist. J Bone Joint Surg Am. 2013;95(20):e1531–9.Google Scholar
  70. 70.
    van Eck CF, Loopik M, van den Bekerom MP, Fu FH, Kerkhoffs GM. Methods to diagnose acute anterior cruciate ligament rupture: a meta-analysis of instrumented knee laxity tests. Knee Surg Sports Traumatol Arthrosc. 2013;21(9):1989–97.CrossRefGoogle Scholar
  71. 71.
    van Eck CF, Schkrohowsky JG, Working ZM, Irrgang JJ, Fu FH. Prospective analysis of failure rate and predictors of failure after anatomic anterior cruciate ligament reconstruction with allograft. Am J Sports Med. 2012;40(4):800–7.CrossRefGoogle Scholar
  72. 72.
    van Eck CF, van den Bekerom MP, Fu FH, Poolman RW, Kerkhoffs GM. Methods to diagnose acute anterior cruciate ligament rupture: a meta-analysis of physical examinations with and without anaesthesia. Knee Surg Sports Traumatol Arthrosc. 2013;21(8):1895–903.CrossRefGoogle Scholar
  73. 73.
    Vavken P. Rationale for and methods of superiority, noninferiority, or equivalence designs in orthopaedic, controlled trials. Clin Orthop Relat Res. 2011;469(9):2645–53.CrossRefGoogle Scholar
  74. 74.
    W-Dahl A, Toksvig-Larsen S, Roos EM. A 2-year prospective study of patient-relevant outcomes in patients operated on for knee osteoarthritis with tibial osteotomy. BMC Musculoskelet Disord. 2005;6:18.CrossRefGoogle Scholar
  75. 75.
    Williams GN, Taylor DC, Gangel TJ, Uhorchak JM, Arciero RA. Comparison of the single assessment numeric evaluation method and the Lysholm score. Clin Orthop Relat Res. 2000;373:184–92.CrossRefGoogle Scholar
  76. 76.
    Wilson FN, Johnston FD, Hill IGW, Macleod AG, Barker PS. The significance of electrocardiograms characterized by an abnormally long QRS interval and by broad S deflections in Lead I. Am Heart J. 1934;9:459.CrossRefGoogle Scholar
  77. 77.
    Wood EH, Ritman EL, Robb RA, Harris LD, Ruegsegger P. Noninvasive numerical vivisection of anatomic structure and function of the intact circulatory system using high temporal resolution cylindrical scanning computerized tomography. Med Instrum. 1977;11(3):153–9.PubMedGoogle Scholar
  78. 78.
    Wright RW. Knee injury outcomes measures. J Am Acad Orthop Surg. 2009;17(1):31–9.CrossRefGoogle Scholar

Copyright information

© ISAKOS 2019

Authors and Affiliations

  • Carola F. van Eck
    • 1
  • Marcio Bottene Villa Albers
    • 1
  • Andrew J. Sheean
    • 1
  • Freddie H. Fu
    • 1
    Email author
  1. 1.Department of Orthopaedic SurgeryUniversity of PittsburghPittsburghUSA

Personalised recommendations