Skip to main content
SpringerLink
Log in

Measurement Reliability in Periodontal Research

  • Chapter
  • First Online:
Understanding Periodontal Research

  • 57 Accesses

Abstract

This chapter focuses primarily on the periodontal measurement sources of variability and on practical issues related to training, calibrating, and monitoring the individuals involved in conducting a clinical trial. It is also provided on overview of statistical methods used to perform evaluation of intra-examiner (comparisons may be made in which the same investigator examines the same subjects two or more times) and inter-examiner reliability (in which different investigators examine the same subjects). Details are also given regarding the reliability studies performed in periodontal research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  • Aardal E, Holm AC. Cortisol in saliva-reference ranges and relation to cortisol in serum. Eur J Clin Chem Clin Biochem. 1995;33:927–32.

    PubMed  Google Scholar 

  • Abbas F, Hart AA, Oosting J, van der Velden U. Effect of training and probing force on the reproducibility of pocket depth measurements. J Periodontal Res. 1982;17:226–34.

    PubMed  Google Scholar 

  • Abrahamsson I, Soldini C. Probe penetration in periodontal and peri-implant tissues. An experimental study in the beagle dog. Clin Oral Implants Res. 2006;17:601–5.

    PubMed  Google Scholar 

  • Albandar JM, Kingman A. Gingival recession, gingival bleeding, and dental calculus in adults 30 years of age and older in the United States, 1988–1994. J Periodontol. 1999;70:30–43.

    PubMed  Google Scholar 

  • Albandar JM, Brunelle JA, Kingman A. Destructive periodontal disease in adults 30 years of age and older in the United States, 1988–1994. J Periodontol. 1999;70:13–29.

    PubMed  Google Scholar 

  • Axtelius B, Edwardsson S, Theodorsson E, Svensäter G, Attström R. Presence of cortisol in gingival crevicular fluid. A pilot study. J Clin Periodontol. 1998;25:929–32.

    PubMed  Google Scholar 

  • Barendregt DS, Van der Velden U, Reiker J, Loos BG. Clinical evaluation of tine shape of 3 periodontal probes using 2 probing forces. J Clin Periodontol. 1996;23:397–402.

    PubMed  Google Scholar 

  • Bartko JJ. Measurement and reliability: statistical thinking considerations. Schizophr Bull. 1991;17:483–9.

    PubMed  Google Scholar 

  • Bergmann A, Deinzer R. Daytime variations of interleukin-1beta in gingival crevicular fluid. Eur J Oral Sci. 2008;116:18–22.

    PubMed  Google Scholar 

  • Bessa Rebelo MA, Adriana Corra de Queiroz A. Gingival indices: state of art. In: Panagakos FS, Davies RM, editors. Gingival diseases – their aetiology, prevention and treatment. InTech, São Paulo. 2011; p. 41–54.

    Google Scholar 

  • Birek P, McCulloch CA, Hardy V. Gingival attachment level measurements with an automated periodontal probe. J Clin Periodontol. 1987;14:472–7.

    PubMed  Google Scholar 

  • Birkeland JM, Jorkjend L. The influence of examination of the assessment of the intra-examiner error by using the plaque and gingival index systems. Community Dent Oral Epidemiol. 1975;3:214–6.

    PubMed  Google Scholar 

  • Blieden TM, Caton JG, Proskin HM, Stein SH, Wagener CJ. Examiner reliability for an invasive gingival bleeding index. J Clin Periodontol. 1992;19:262–7.

    PubMed  Google Scholar 

  • Buduneli E, Aksoy O, Köse T, Atilla G. Accuracy and reproducibility of two manual periodontal probes. An in vitro study. J Clin Periodontol. 2004;31:815–9.

    PubMed  Google Scholar 

  • Bulthuis HM, Barendregt DS, Timmerman MF, Loos BG, van der Velden U. Probe penetration in relation to the connective tissue attachment level: influence of tine shape and probing force. J Clin Periodontol. 1998;25:417–23.

    PubMed  Google Scholar 

  • Caton J, Greenstein G, Polson AM. Depth of periodontal probe penetration related to clinical and histologic signs of gingival inflammation. J Periodontol. 1981;52:626–9.

    PubMed  Google Scholar 

  • Chamberlain AD, Renvert S, Garrett S, Nilvéus R, Egelberg J. Significance of probing force for evaluation of healing following periodontal therapy. J Clin Periodontol. 1985;12:306–11.

    PubMed  Google Scholar 

  • Cicchetti DV, Heavens JR. A computer program for determining the significance of the difference between pairs of independently derived values of kappa or weighted kappa. Educ Psychol Meas. 1981;41:189–93.

    Google Scholar 

  • Clark WB, Magnusson I, Namgung YY, Yang MCK. The strategy and advantage in use of an electronic probe for attachment measurement. Adv Dent Res. 1993;7:152–7.

    PubMed  Google Scholar 

  • Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20:37–46.

    Google Scholar 

  • Cohen J. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull. 1968;70:213–20.

    PubMed  Google Scholar 

  • Daly C, Mitchell D, Grossberg D, Highfield J, Stewart D. Bacteraemia caused by periodontal probing. Aust Dent J. 1997;42:77–80.

    PubMed  Google Scholar 

  • Daly CG, Mitchell DH, Highfield JE, Grossberg DE, Stewart D. Bacteremia due to periodontal probing: a clinical and microbiological investigation. J Periodontol. 2001;72:210–4.

    PubMed  Google Scholar 

  • Deas DE, Moritz AJ, Mealey BL, McDonnell HT, Powell CA. Clinical reliability of the “furcation arrow” as a diagnostic marker. J Periodontol. 2006;77:1436–41.

    PubMed  Google Scholar 

  • Deinzer R, Mossanen BS, Herforth A. Methodological considerations in the assessment of gingival crevicular fluid volume. J Clin Periodontol. 2000;27:481–8.

    PubMed  Google Scholar 

  • Dombret B, Matthijs S, Moradi Sabzevar M. Interexaminer reproducibility of ordinal and interval-scaled plaque indices. J Clin Periodontol. 2003;30:630–5.

    PubMed  Google Scholar 

  • Eaton KA, Rimini FM, Zak E, Brookman DJ, Newman HN. The achievement and maintenance of inter-­examiner consistency in the assessment of plaque and gingivitis during a multicentre study based in general dental practices. J Clin Periodontol. 1997;24:183–8.

    PubMed  Google Scholar 

  • Egelberg J. Oral hygiene methods the scientific way (synopses of clinical studies). 1st ed. Sweden: OdontoScience; 1999. p. 271–8.

    Google Scholar 

  • Eickholz P, Steinbrenner H, Lenhard M, Marquardt M, Holle R. Interexaminer reliability of the assessment of clinical furcation parameters as related to different probes. Eur J Oral Sci. 1999;107:2–8.

    PubMed  Google Scholar 

  • Feldman RS, Douglass CW, Loftus ER, Kapur KK, Chauncey HH. Interexaminer agreement in the measurement of periodontal disease. J Periodontal Res. 1982;17:80–9.

    PubMed  Google Scholar 

  • Fleiss JL. Measuring nominal scale agreement among many raters. Psychol Bull. 1971;76:378–82.

    Google Scholar 

  • Fleiss JL, Chilton NW. The measurement of interexaminer agreement on periodontal disease. J Periodontal Res. 1983;18:601–6.

    PubMed  Google Scholar 

  • Fleiss JL, Gordon J. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psychol Meas. 1973;33:613–9.

    Google Scholar 

  • Fleiss JL, Cohen J, Everitt BS. Large sample standard errors of kappa and weighted kappa. Psychol Bull. 1969;72:323–7.

    Google Scholar 

  • Fleiss JL, Nee JCM, Landis JR. Large sample variance of kappa in the case of different sets of raters. Psychol Bull. 1979;86:974–7.

    Google Scholar 

  • Fleiss JL, Mann J, Paik M, Goultchin J, Chilton NW. A study of inter- and intra-examiner reliability of pocket depth and attachment level. J Periodontal Res. 1991;26:122–8.

    PubMed  Google Scholar 

  • Gibbs CH, Hirschfeld JW, Lee JG, Low SB, Magnusson I, Thousand RR, Yerneni P, Clark WB. Description and clinical evaluation of a new computerized periodontal probe – the Florida probe. J Clin Periodontol. 1988;15:137–44.

    PubMed  Google Scholar 

  • Goodson JM, Kondon N. Periodontal pocket depth measurements by fiber optic technology. J Clin Dent. 1988;1:35–8.

    PubMed  Google Scholar 

  • Goulet JP, Clark GT. Clinical TMJ examination methods. J Calif Dent Assoc. 1990;18:25–33.

    PubMed  Google Scholar 

  • Gröndahl K, Sundén S, Gröndahl HG. Inter- and intraobserver variability in radiographic bone level assessment at Brånemark fixtures. Clin Oral Implants Res. 1998;9:243–50.

    PubMed  Google Scholar 

  • Grossi SG, Dunford RG, Ho A, Koch G, Machtei EE, Genco RJ. Sources of error for periodontal probing measurements. J Periodontal Res. 1996;31:330–6.

    PubMed  Google Scholar 

  • Hansen AM, Garde AH, Persson R. Sources of biological and methodological variation in salivary cortisol and their impact on measurement among healthy adults: a review. Scand J Clin Lab Invest. 2008;68:448–58.

    PubMed  Google Scholar 

  • Hausmann E. Radiographic and digital imaging in periodontal practice. J Periodontol. 2000;71:497–503.

    PubMed  Google Scholar 

  • Hefti AF. Periodontal probing. Crit Rev Oral Biol Med. 1997;8(3):336–56.

    PubMed  Google Scholar 

  • Hill EG, Slate EH, Wiegand RE, Grossi SG, Salinas CF. Study design for calibration of clinical examiners measuring periodontal parameters. J Periodontol. 2006;77:1129–41.

    PubMed  Google Scholar 

  • Hou GL, Hung CC, Yang YS, Shieh TY, Tsai CC. Radiographic alveolar bone loss in untreated Taiwan Chinese subjects with adult periodontitis measured by the digital scanning radiographic image analysis method. Dentomaxillofac Radiol. 2003;32:104–8.

    PubMed  Google Scholar 

  • http://en.wikipedia.org/wiki/Fleiss%27_kappa. Fleiss’ kappa.Accessed on 22 Feb 2010.

  • Jeffcoat MK. Current concepts in periodontal disease testing. J Am Dent Assoc. 1994;125:1071–8.

    PubMed  Google Scholar 

  • Jeffcoat MK, Jeffcoat RL, Jens SC, Captain K. A new periodontal probe with automated cemento-enamel junction detection. J Clin Periodontol. 1986;13:276–80.

    PubMed  Google Scholar 

  • Kawamura M, Fukuda S, Inoue C, Sasahara H, Iwamoto Y. The validity and reproducibility of an oral rating index as a measurement of gingival health care and oral hygiene level in adults. J Clin Periodontol. 2000;27:411–6.

    PubMed  Google Scholar 

  • Khocht A, Chang KM. Clinical evaluation of electronic and manual constant force probes. J Periodontol. 1998;69:19–25.

    PubMed  Google Scholar 

  • Kida IA, Astrøm AN, Strand GV, Masalu JR. Clinical and socio-behavioral correlates of tooth loss: a study of older adults in Tanzania. BMC Oral Health. 2006;6:5.

    PubMed  Google Scholar 

  • Kinane DF, Riggio MP, Walker KF, MacKenzie D, Shearer B. Bacteraemia following periodontal procedures. J Clin Periodontol. 2005;32:708–13.

    PubMed  Google Scholar 

  • Kingman A, Albandar JM. Methodological aspects of epidemiological studies of periodontal diseases. Periodontol 2000. 2002;29:11–30.

    PubMed  Google Scholar 

  • Koo KT, Polimeni G, Albandar JM, Wikesjö UM. Periodontal repair in dogs: analysis of histometric assessments in the supraalveolar periodontal defect model. J Periodontol. 2004;75(12):1688–93.

    PubMed  Google Scholar 

  • Kudielka BM, Hellhammer DH, Wüst S. Why do we respond so differently? Reviewing determinants of human salivary cortisol responses to challenge. Psychoneuroendocrinology. 2009;34:2–18.

    PubMed  Google Scholar 

  • Lafzi A, Mohammadi AS, Eskandari A, Pourkhamneh S. Assessment of intra- and inter-examiner reproducibility of probing depth measurements with a manual periodontal probe. JODDD. 2007;1:19–25.

    Google Scholar 

  • Lang NP, Cullinan MP, Holborow DW, Heitz-Mayfield LJA. Examiner training: standardization and calibration in periodontal studies. In: Giannobile WV, Burt BA, Genco RJ, editors. Clinical research in oral health. 1st ed. Hoboken: Wiley-Blackwell; 2010. p. 159–75.

    Google Scholar 

  • Larsen C, Barendregt DS, Slot DE, Van der Velden U, Van der Weijden F. Probing pressure, a highly undervalued unit of measure in periodontal probing: a systematic review on its effect on probing pocket depth. J Clin Periodontol. 2009;36:315–22.

    PubMed  Google Scholar 

  • Löe H, Silness J. Periodontal disease in pregnancy. 1. Prevalence and severity. Acta Odontol Scand. 1963;21:533–51.

    PubMed  Google Scholar 

  • Magnusson I, Fuller WW, Heins PJ, Rau CF, Gibbs CH, Marks RG, Clark WB. Correlation between electronic and visual readings of pocket depths with a newly developed constant force probe. J Clin Periodontol. 1988;15:180–4.

    PubMed  Google Scholar 

  • Mariath AA, Bressani AE, Haas AN, Araujo FB, Rösing CK. Professional flossing as a diagnostic method for gingivitis in the primary dentition. Braz Oral Res. 2008;22:316–21.

    PubMed  Google Scholar 

  • Marks RG, Magnusson I, Taylor M, Clouser B, Maruniak J, Clark WB. Evaluation of reliability and reproducibility of dental indices. J Clin Periodontol. 1993;20:54–8.

    PubMed  Google Scholar 

  • Mayfield L, Bratthall G, Attström R. Periodontal probe precision using 4 different periodontal probes. J Clin Periodontol. 1996;23:76–82.

    PubMed  Google Scholar 

  • McClanahan SF, Bartizek RD, Biesbrock AR. Identification and consequences of distinct Löe-Silness gingival index examiner styles for the clinical assessment of gingivitis. J Periodontol. 2001;72:383–92.

    PubMed  Google Scholar 

  • Mojon P, Chung J-P, Favre P, Budtz-Jorgensen E. Examiner agreement on periodontal indices during dental surveys of elders. J Clin Periodontol. 1996;23:56–9.

    PubMed  Google Scholar 

  • Mombelli A, Mühle T, Brägger U, Lang NP, Bürgin WB. Comparison of periodontal and peri-implant probing by depth-force pattern analysis. Clin Oral Implants Res. 1997;8:448–54.

    PubMed  Google Scholar 

  • Müller HP, Ulbrich M. Alveolar bone levels in adults as assessed on panoramic radiographs. (I) Prevalence, extent, and severity of even and angular bone loss. Clin Oral Investig. 2005;9(2):98–104.

    PubMed  Google Scholar 

  • Müller HP, Barrieshi-Nusair KM. Gingival bleeding on repeat probing after different time intervals in plaque-induced gingivitis. Clin Oral Investig. 2005;9:278–83.

    PubMed  Google Scholar 

  • Namgung YY, Yang MC. Outlier reduction by an option-3 measurement scheme. Biometrics. 1994;50(1):173–82.

    PubMed  Google Scholar 

  • Neto JB, Filho GR, Tramontina VA, Sallum EA, Nociti Jr FH, Sallum AW. Millimeter marks and probe tip ­diameter standardisation from commercially available periodontal probes. A comparative study. J Int Acad Periodontol. 2001;3:57–60.

    PubMed  Google Scholar 

  • Nield-Gehrig JS. Fundamentals of periodontal instrumentation and advanced root instrumentation. Philadelphia: Lippincott Williams & Wilkins; 2007.

    Google Scholar 

  • Oakley C, Brunette DM. The use of diagnostic data in clinical dental practice. Dent Clin North Am. 2002;46:87–115.

    PubMed  Google Scholar 

  • Osborn J, Stoltenberg J, Huso B, Aeppli D, Pihlstrom B. Comparison of measurement variability using a standard and constant force periodontal probe. J Periodontol. 1990;61:497–503.

    PubMed  Google Scholar 

  • Osborn JB, Stoltenberg JL, Huso BA, Aeppli DM, Pihlstrom BL. Comparison of measurement variability in subjects with moderate periodontitis using a conventional and constant force periodontal probe. J Periodontol. 1992;63:283–9.

    PubMed  Google Scholar 

  • Pecoraro M, Azadivatan-le N, Janal M, Khocht A. Comparison of observer reliability in assessing alveolar bone height on direct digital and conventional radiographs. Dentomaxillofac Radiol. 2005;34:279–84.

    PubMed  Google Scholar 

  • Pierro VS, de Souza IP, Luiz RR, Barcelos R, Moraes RS. Reliability of two methods for measurement of alveolar bone level in children. Dentomaxillofac Radiol. 2008;37:34–9.

    PubMed  Google Scholar 

  • Pihlstrom BL. Measurement of attachment level in clinical trials: probing methods. J Periodontol. 1992;63:1072–7.

    PubMed  Google Scholar 

  • Polson AM. The research team, calibration, and quality assurance in clinical trials in periodontics. Ann Periodontol. 1997;2:75–82.

    PubMed  Google Scholar 

  • Preshaw PM, Kupp L, Hefti AF, Mariotti A. Measurement of clinical attachment levels using a constant force periodontal probe modified to detect the cementoenamel junction. J Clin Periodontol. 1999;26:434–40.

    PubMed  Google Scholar 

  • Quirynen M, Callens A, van Steenberghe D, Nys M. Clinical evaluation of a constant force electronic probe. J Periodontol. 1993;64:35–9.

    PubMed  Google Scholar 

  • Rae G. The equivalence of multiple rater kappa statistics and intraclass correlation coefficients. Educ Psychol Meas. 1988;48:367–74.

    Google Scholar 

  • Ramfjord SP. Indices for prevalence and incidence of periodontal disease. J Periodontol. 1959;30:51–9.

    Google Scholar 

  • Reddy MS, Palcanis KG, Gears NC. A comparison of manual and controlled-force attachment-level measurements. J Clin Periodontol. 1997;24:920–6.

    PubMed  Google Scholar 

  • Rise J, Tollefsen T. Reliability of plaque and periodontal measurements estimated by the internal consistency method. Acta Odontol Scand. 1984;42:293–6.

    PubMed  Google Scholar 

  • Roberts GJ. Dentists are innocent! “Everyday” bacteremia is the real culprit: a review and assessment of the evidence that dental surgical procedures are a principal cause of bacterial endocarditis in children. Pediatr Cardiol. 1999;20:317–25.

    PubMed  Google Scholar 

  • Sakka S, Al-ani Z, Kasioumis T, Worthington H, Coulthard P. Inter-examiner and intra-examiner reliability of the measurement of marginal bone loss around oral implants. Implant Dent. 2005;14:386–8.

    PubMed  Google Scholar 

  • Seabra RC, Costa FO, Costa JE, Van Dyke T, Soares RV. Impact of clinical experience on the accuracy of probing depth measurements. Quintessence Int. 2008;39:559–65.

    PubMed  Google Scholar 

  • Shaw L, Murray JJ. Diagnostic reproducibility of periodontal indices. J Periodontal Res. 1977;12:141–7.

    PubMed  Google Scholar 

  • Shi XQ, Eklund I, Tronje G, Welander U, Stamatakis HC, Engström PE, Engström GN. Comparison of observer reliability in assessing alveolar bone changes from color-coded with subtraction radiographs. Dentomaxillofac Radiol. 1999;28:31–6.

    PubMed  Google Scholar 

  • Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–8.

    PubMed  Google Scholar 

  • Silness J, Löe H. Periodontal disease in pregnancy. 2. Correlation between oral hygiene and periodontal condition. Acta Odontol Scand. 1964;22:121–35.

    PubMed  Google Scholar 

  • Spolsky VW, Gornbein JA. Comparing measures of reliability for indices of gingivitis and plaque. J Periodontol. 1996;67:853–9.

    PubMed  Google Scholar 

  • Suh YI, Lundgren T, Sigurdsson T, Riggs M, Crigger M. Probing bone level measurements for determination of the depths of class II furcation defects. J Periodontol. 2002;73:637–42.

    PubMed  Google Scholar 

  • Teeuw WJ, Coelho L, Silva A, van der Palen CJNM, Lessmann FGJM, van der Velden U, Loos BG. Validation of a dental image analyzer tool to measure alveolar bone loss in periodontitis patients. J Periodontal Res. 2009;44:94–102.

    PubMed  Google Scholar 

  • Valachovic RW, Douglass CW, Berkey CS, McNeil BJ, Chauncey HH. Examiner reliability in dental radiography. J Dent Res. 1986;65:432–6.

    PubMed  Google Scholar 

  • Van der Zee E, Davies EH, Newman HN. Marking width, calibration from tip and tine diameter of periodontal probes. J Clin Periodontol. 1991;18:516–20.

    PubMed  Google Scholar 

  • Verhoeven JW, Cune MS, de Putter C. Reliability of some clinical parameters of evaluation in implant dentistry. J Oral Rehabil. 2000;27:211–6.

    PubMed  Google Scholar 

  • Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam Med. 2005;37:360–3.

    PubMed  Google Scholar 

  • Walsh TF, Saxby MS. Inter- and intra-examiner variability using standard and constant force periodontal probes. J Clin Periodontol. 1989;16:140–3.

    PubMed  Google Scholar 

  • Wilson W, Taubert KA, Gewitz M, Lockhart PB, Baddour LM, Levison M, Bolger A, Cabell CH, Takahashi M, Baltimore RS, Newburger JW, Strom BL, Tani LY, Gerber M, Bonow RO, Pallasch T, Shulman ST, Rowley AH, Burns JC, Ferrieri P, Gardner T, Goff D, Durack DT, American Heart Association Rheumatic Fever, Endocarditis and Kawasaki Disease Committee, Council on Cardiovascular Disease in the Young, Council on Clinical Cardiology, Council on Cardiovascular Surgery and Anesthesia, Quality of Care and Outcomes Research Interdisciplinary Working Group, American Dental Association. Prevention of infective endocarditis: guidelines from the American Heart Association: a guideline from the American Heart Association Rheumatic Fever, Endocarditis and Kawasaki Disease Committee, Council on Cardiovascular Disease in the Young, and the Council on Clinical Cardiology, Council on Cardiovascular Surgery and Anesthesia, and the Quality of Care and Outcomes Research Interdisciplinary Working Group. J Am Dent Assoc. 2007;138(739–45):747–60.

    Google Scholar 

  • Winter AA. Measurement of the millimeter markings of periodontal probes. J Periodontol. 1979;50:483–5.

    PubMed  Google Scholar 

  • Wolf B, Von Bethlenfalvy E, Hassfeld S, Staehle HJ, Eickholz P. Reliability of assessing interproximal bone loss by digital radiography: intrabony defects. J Clin Periodontol. 2001;28:869–78.

    PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Periodontology, University of Tromsø Institute of Clinical Dentistry, Tromsø, Norway

    Alexandrina L. Dumitrescu

  2. Private practice, Bucharest, Romania

    Alexandrina L. Dumitrescu

Authors
  1. Alexandrina L. Dumitrescu
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Dumitrescu, A.L. (2012). Measurement Reliability in Periodontal Research. In: Understanding Periodontal Research. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28923-1_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-28923-1_6

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-28922-4

  • Online ISBN: 978-3-642-28923-1

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics

Search

Navigation