Advertisement

Observer error and its impact on ancestry estimation using dental morphology

  • Marin A. Pilloud
  • Donovan M. Adams
  • Joseph T. Hefner
Original Article

Abstract

Dental morphology is becoming increasingly visible in forensic anthropology as part of the estimation of ancestry. As methods are developed based on these data, it is important to understand the role of observer error in data collection and method application. In this study, 10 observers collected dental morphological data on 19 traits on the same set of nine plaques. Various measures of interrater reliability were calculated to assess observer error. Data were then input into one of three ancestry estimation methods based on dental morphology to understand the role of observer error in these methods. Results show low rater reliability for all dental morphological traits when all 10 observers are compared. Rater reliability increases when only experienced observers are compared and traits are dichotomized. Further, differences in trait scores by observers resulted in disparate estimations of ancestry in each of the methods. While observer error appears to be an issue in dental morphological methods of ancestry estimation, these problems can be addressed. An argument is made for advanced training in dental anthropology in laboratories and in graduate programs. Further, methods need to test for and employ traits with high rater agreement.

Keywords

Forensic anthropology Interobserver error Rater reliability Dental non-metrics Breakpoints ASUDAS 

Notes

Acknowledgments

We thank the Defense POW/MIA Accounting Agency Laboratory for assistance and Heather J.H. Edgar for providing use of the casts from the Economides collection. Also, G. Richard Scott provided insight on the analyses of this manuscript.

References

  1. 1.
    Rhine S (1990) Non-metric skull racing. In: Gill GW, Rhine S (eds) Skeletal attribution of race: methods for forensic anthropology. Maxwell Museum Anthropological Papers No. 4, Albuquerque, N.M., pp 9–20Google Scholar
  2. 2.
    Hinkes MJ (1990) Shovel-shaped incisors in human identification. In: Gill G, Rhine S (eds) Skeletal attribution of race. vol 4. Maxwell Museum Anthropological Papers No. 4, Albuquerque, NM, pp 21–26Google Scholar
  3. 3.
    Birkby WH, Fenton TW, Anderson BE (2008) Identifying Southwest Hispanics using nonmetric traits and the cultural profile. J Forensic Sci 53(1):29–33.  https://doi.org/10.1111/j.1556-4029.2007.00611.x PubMedCrossRefGoogle Scholar
  4. 4.
    Edgar HJH (2005) Prediction of race using characteristics of dental morphology. J Forensic Sci 50(2):269–273PubMedCrossRefGoogle Scholar
  5. 5.
    Edgar HJH (2013) Estimation of ancestry using dental morphological characteristics. J Forensic Sci 58(s1):S3–S8PubMedCrossRefGoogle Scholar
  6. 6.
    Scott GR, Pilloud MA, Navega D, d’Oliveira J, Cunha E, Irish DJ (2018) rASUDAS: a new web-based application for estimating ancestry from tooth morphology. Forensic Anthropol 1:18–31CrossRefGoogle Scholar
  7. 7.
    Turner CG, Nichol CR, Scott GR (1991) Scoring procedures for key morphological traits of the permanent dentition: the Arizona State University Dental Anthropology System. In: Kelley MA, Larsen CS (eds) Advances in dental anthropology. Wiley-Liss, New York, pp 13–31Google Scholar
  8. 8.
    Pilloud MA, Maier C, Scott GR, Edgar HJH (2018) Molar crenulation trait definition and variation in modern human populations. Homo 69(3):77–85.  https://doi.org/10.1016/j.jchb.2018.06.001 PubMedCrossRefGoogle Scholar
  9. 9.
    Scott GR, Irish JD (2017) Human crown and root morphology: the Arizona State University Dental Anthropology System. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  10. 10.
    Pilloud MA (2018) Technical note: the definition of new dental morphological variants related to malocclusion. Dent Anthropol 31(1):10–18CrossRefGoogle Scholar
  11. 11.
    Burnett SE, Hawkey DE, Turner CG (2010) Brief communication: population variation in human maxillary premolar accessory ridges (MxPAR). Am J Phys Anthr 141(2):319–324.  https://doi.org/10.1002/ajpa.21230 CrossRefGoogle Scholar
  12. 12.
    Edgar HJH (2017) Dental morphology: an illustrated manual. Routledge, New YorkCrossRefGoogle Scholar
  13. 13.
    Edgar HJH (2002) Biological distance and the African American dentition. PhD, The Ohio State University, ColumbusGoogle Scholar
  14. 14.
    Pilloud MA (2009) Community structure at Neolithic Çatalhöyük: biological distance analysis of household, neighborhood, and settlement. Unpublished PhD dissertation, The Ohio State University, Columbus, OhioGoogle Scholar
  15. 15.
    Scott GR (1973) Dental morphology: a genetic study of American white families and variation in living Southwest Indians. Unpublished PhD dissertation, Arizona State University, Tempe, AZGoogle Scholar
  16. 16.
    Sofaer JA, Niswander J, Maclean C, Workman P (1972) Population studies on southwestern Indian tribes. V. Tooth morphology as an indicator of biological distance. Am J Phys Anthr 37:357–366CrossRefGoogle Scholar
  17. 17.
    Hsu JW, Tsai PL, Hsiao TH, Chang HP, Liu LM, Liu KM, Yu HS, Ferguson D (1999) Ethnic dental analysis of shovel and Carabelli’s traits in a Chinese population. Aust Dent J 44(1):40–45PubMedCrossRefGoogle Scholar
  18. 18.
    Hasegawa Y, Rogers J, Scriven G, Townsend G (2010) Carabelli trait in Australian twins: reliability and validity of different scoring systems. Dent Anthropol 23:7–14Google Scholar
  19. 19.
    Willermet C, Edgar HJH, Ragsdale C, Aubry BS (2013) Biodistances among Mexica, Maya, Toltec, and Totonac groups of central and coastal Mexico. Chungará 45(3):447–459CrossRefGoogle Scholar
  20. 20.
    Edgar HJH, Lease LR (2007) Correlations between deciduous and permanent tooth morphology in a European American sample. Am J Phys Anthr 133(1):726–734CrossRefGoogle Scholar
  21. 21.
    Nichol CR, Turner CG (1986) Intra- and interobserver concordance in classifying dental morphology. Am J Phys Anthr 69(3):299–315CrossRefGoogle Scholar
  22. 22.
    Edgar HJH, Daneshvari S, Harris EF, Kroth PJ (2011) Inter-observer agreement on subjects’ race and race-informative characteristics. PLoS One 6(8):e23986.  https://doi.org/10.1371/journal.pone.0023986 PubMedPubMedCentralCrossRefGoogle Scholar
  23. 23.
    Scott GR, Turner CG (1997) The anthropology of modern human teeth. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  24. 24.
    Adams DM, George RL Fuzzy inference system (FIS) as a novel statistical method for forensic ancestry estimation. In: American Academy of Forensic Sciences, Seattle, WA, 2018. Proceedings: American Acadmy of Forensic Sciences, p 154Google Scholar
  25. 25.
    Maier C (2017) The combination of cranial morphoscopic and dental morphological methods to improve the forensic estimation of ancestry [PhD dissertation]. University of Nevada, RenoGoogle Scholar
  26. 26.
    Willermet CM, Daniels J, Edgar HJH Seeing RED: a new statistical solution to an old categorical data problem. In: American Association of Physical Anthropologists, Atlanta, Georgia, 2016. American Association of Physical AnthropologistsGoogle Scholar
  27. 27.
    R Core Team (2013) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna. http://www.R-project.org
  28. 28.
    Cohen J (1968) Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Am Psychol Assoc 70(4):213–220.  https://doi.org/10.1037/h0026256 CrossRefGoogle Scholar
  29. 29.
    Shrout PE, Fleiss JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86(2):420–428PubMedPubMedCentralCrossRefGoogle Scholar
  30. 30.
    Koo TK, Li MY (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 15(2):155–163.  https://doi.org/10.1016/j.jcm.2016.02.012 PubMedPubMedCentralCrossRefGoogle Scholar
  31. 31.
    Klales AR, Burns TL (2017) Adapting and applying the Phenice (1969) adult morphological sex estimation technique to subadults. J Forensic Sci 62(3):747–752PubMedCrossRefGoogle Scholar
  32. 32.
    Garvin HM, Klales AR (2018) A validation study of the Langley et al. (2017) decision tree model for sex estimation. J Forensic Sci 63(4):1243–1251PubMedCrossRefGoogle Scholar
  33. 33.
    Shirley NR, Ramirez Montes PA (2015) Age estimation in forensic anthropology: quantification of observer error in phase versus component-based methods. J Forensic Sci 60(1):107–111PubMedCrossRefGoogle Scholar
  34. 34.
    Byrnes JF, Kenyhercz MW, Berg GE (2017) Examining interobserver reliability of metric and morphoscopic characteristics of the mandible. J Forensic Sci 62(4):981–985.  https://doi.org/10.1111/1556-4029.13349 PubMedCrossRefGoogle Scholar
  35. 35.
    Klales AR, Ousley SD, Vollner JM (2012) A revised method of sexing the human innominate using Phenice’s nonmetric traits and statistical methods. Am J Phys Anthr 149(1):104–114CrossRefGoogle Scholar
  36. 36.
    Lesciotto KM, Doershuk LJ (2018) Accuracy and reliability of the Klales et al. (2012) morphoscopic pelvic sexing method. J Forensic Sci 63(1):214–220.  https://doi.org/10.1111/1556-4029.13501 PubMedCrossRefGoogle Scholar
  37. 37.
    Fleiss JL, Cohen J (1973) The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psychol Meas 33:613–619CrossRefGoogle Scholar
  38. 38.
    Feng GC (2015) Mistakes and how to avoid mistakes in using intercoder reliability indices. Methodology 11(1):13–22.  https://doi.org/10.1027/1614-2241/a000086 CrossRefGoogle Scholar
  39. 39.
    Fleiss JL (1971) Measuring nominal scale agreement among many raters. Psychol Bull 76(5):378–382CrossRefGoogle Scholar
  40. 40.
    Conger AJ (1980) Integration and generalization of kappas for multiple raters. Psychol Bull 88(2):322–328CrossRefGoogle Scholar
  41. 41.
    Hayes AF, Krippendorff K (2007) Answering the call for a standard reliability measure for coding data. Commun Methods Meas 1(1):77–89.  https://doi.org/10.1080/19312450709336664 CrossRefGoogle Scholar
  42. 42.
    McHugh ML (2012) Interrater reliability: the kappa statistic. Biochem Med 22(3):276–282CrossRefGoogle Scholar
  43. 43.
    Sim J, Wright CC (2005) The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther 85(3):257–268.  https://doi.org/10.1093/ptj/85.3.257 PubMedCrossRefGoogle Scholar
  44. 44.
    Landis RJ, Koch GG (1977) The measure of observer agreement for categorical data. Biometrics 33:159–174PubMedPubMedCentralCrossRefGoogle Scholar
  45. 45.
    Taylor R (1990) Interpretation of the correlation coefficient: a basic review. J Diagn Med Sonograph 6(1):35–39.  https://doi.org/10.1177/875647939000600106 CrossRefGoogle Scholar
  46. 46.
    Krippendorff K (2004) Reliability in content analysis. Hum Commun Res 30(3):411–433.  https://doi.org/10.1111/j.1468-2958.2004.tb00738.x CrossRefGoogle Scholar
  47. 47.
    Krippendorff K (2004) Content analysis: an introduction to its methodology, 2nd edn. Sage, Thousand OaksGoogle Scholar
  48. 48.
    Scott GR (1977) Classification, sex dimorphism, association, and population variation of the canine distal accessory ridge. Hum Biol 49(3):453–469PubMedGoogle Scholar
  49. 49.
    Irish JD (1993) Biological affinities of late Pleistocene through modern Africal aboriginal populations: the dental evidence. Ph.D., Arizona State University, Tempe, AZGoogle Scholar
  50. 50.
    Scott GR, Turner CG, Townsend G, Martinon-Torres M (2018) Anthropology of modern human teeth: dental morphology and its variation in fossil and recent Homo sapiens. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  51. 51.
    Gwet KL (2008) Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol 61(1):29–48PubMedCrossRefGoogle Scholar
  52. 52.
    Burnett SE, Irish JD, Fong MR (2013) Wear’s the problem? Examining the effect of dental wear on studies of crown morphology. In: Scott GR, Irish JD (eds) Anthropological perspectives on tooth morphology: genetics, evolution, variation. Cambridge University Press, Cambridge, pp 535–554CrossRefGoogle Scholar
  53. 53.
    Stojanowski CM, Johnson KM (2015) Observer error, dental wear, and the inference of new world sundadonty. Am J Phys Anthr 156(3):349–362CrossRefGoogle Scholar
  54. 54.
    Willermet CM, Edgar HJH (2009) Dental morphology and ancestry in Albuquerque, New Mexico Hispanics. HOMO - J Comp Hum Biol 60(3):207–224.  https://doi.org/10.1016/j.jchb.2008.06.008 CrossRefGoogle Scholar
  55. 55.
    Scott GR, Navega D, Coelho J, Cunha E, Irish JD (2016) rASUDAS: a new method for estimating ancestry from tooth crown and root morphology. Am J Phys Anthr 159(S62):285–286Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of AnthropologyUniversity of NevadaRenoUSA
  2. 2.Department of AnthropologyMichigan State UniversityEast LansingUSA

Personalised recommendations