International Journal of Biometeorology

, Volume 62, Issue 8, pp 1421–1435 | Cite as

How training citizen scientists affects the accuracy and precision of phenological data

  • Richard E. FeldmanEmail author
  • Irma Žemaitė
  • Abraham J. Miller-Rushing
Original Paper


Monitoring plant and animal phenology is a critical step to anticipating and predicting changes in species interactions and biodiversity. Because phenology necessarily involves frequent and repeated observations over time, citizen scientists have become a vital part of collecting phenological data. However, there is still concern over the accuracy and precision of citizen science data. It is possible that training citizen scientists can improve data quality though there are few comparisons of trained and untrained citizen scientists in the ability of each to accurately and precisely measure phenology. We assessed how three types of observers—experts, trained citizen scientists that make repeated observations, and untrained citizen scientists making once-per-year observations—differ in quantifying temporal change in flower and fruit abundance of American mountain ash trees (Sorbus americana Marsh.) and arthropods in Acadia National Park, Maine, USA. We found that trained more so than untrained citizen science observers over- or under-estimated abundances leading to precise but inaccurate characterizations of phenological patterns. Our results suggest a new type of bias induced by repeated observations: A type of learning takes place that reduces the independence of observations taken on different trees or different dates. Thus, in this and many other cases, having individuals make one-off observations of marked plants may produce data as good if not better than individuals making repeated observations. For citizen science programs related to phenology, our results underscore the importance of (a) attracting the most number of observers possible even if they only make one observation, (b) producing easy-to-use and informative data sheets, and (c) carefully planning effective training programs that are, perhaps, repeated at different points during the data collection period.


Citizen science Flowers Fruit National Park Phenology Sampling bias Sorbus 



We thank Joellyn Colangelo, Libby Orcutt, Julianne Pekney, Ariel Powers, and Megan Roselli for leading and organizing data collection. We thank Seth Benz and Hannah Webber for help designing and implementing the citizen science monitoring. Our work would not be possible without the numerous observations made by Acadia and Schoodic staff and the citizen scientists visiting the Schoodic Institute campus. R. Feldman was supported by a collaborative grant between the National Park Service and the University of Massachusetts Amherst. I. Žemaitė was supported by Baltic-American Freedom Foundation grant for an internship at the Schoodic Institute.

Supplementary material

484_2018_1540_MOESM1_ESM.docx (220 kb)
ESM 1 (DOCX 219 kb)


  1. Beaubien EG, Freeland HJ (2000) Spring phenology trends in Alberta, Canada: links to ocean temperature. Int J Biometeorol 44:53–59CrossRefGoogle Scholar
  2. Bird TJ, Bates AE, Lefcheck JS, Hill NA, Thomson RJ, Edgar GJ, Stuart-Smith RD, Wotherspoon S, Krkosek M, Stuart-Smith JF, Pecl GT, Barrett N, Frusher S (2014) Statistical solutions for error and bias in global citizen science datasets. Biol Conserv 173:144–154. CrossRefGoogle Scholar
  3. Both C, Bouwhuis S, Lessells CM, Visser ME (2006) Climate change and population declines in a long-distance migratory bird. Nature 441:81–83. CrossRefGoogle Scholar
  4. Both C, van Asch M, Bijlsma RG, van den Burg AB, Visser ME (2009) Climate change and unequal phenological changes across four trophic levels: constraints or adaptations? J Anim Ecol 78:73–83. CrossRefGoogle Scholar
  5. Both C, Van Turnhout CA, Bijlsma RG et al (2010) Avian population consequences of climate change are most severe for long-distance migrants in seasonal habitats. Proc Biol Sci 277:1259–1266. CrossRefGoogle Scholar
  6. Cahill AE, Aiello-lammens ME, Fisher-Reid MC et al (2013) How does climate change cause extinction? Proc Biol Sci 280:20121890CrossRefGoogle Scholar
  7. Chandler M, See L, Copas K, Bonde AMZ, López BC, Danielsen F, Legind JK, Masinde S, Miller-Rushing AJ, Newman G, Rosemartin A, Turak E (2017) Contribution of citizen science towards international biodiversity monitoring. Biol Conserv 213:280–294. CrossRefGoogle Scholar
  8. Chapman DS, Bell S, Helfer S, Roy DB (2015) Unbiased inference of plant flowering phenology from biological recording data. Biol J Linn Soc 115:543–554CrossRefGoogle Scholar
  9. Chmielewski FM (2013) Phenology in agriculture and horticulture. In: Schwartz MD (ed) Phenology: an integrative environmental science. Springer, Netherlands, pp 539–561CrossRefGoogle Scholar
  10. Cohn JP (2008) Citizen science: can volunteers do real research? Bioscience 58:192–197CrossRefGoogle Scholar
  11. Crimmins TM, Crimmins MA, Gerst KL, Rosemartin AH, Weltzin JF (2017) USA National Phenology Network’s volunteer-contributed observations yield predictive models of phenological transitions. PLoS One 12:1–17. CrossRefGoogle Scholar
  12. Crall AW, Newman GJ, Stohlgren TJ et al (2011) Assessing citizen science data quality: an invasive species case study. Conserv Lett 10:1–10Google Scholar
  13. Denny EG, Gerst KL, Miller-Rushing AJ, Tierney GL, Crimmins TM, Enquist CAF, Guertin P, Rosemartin AH, Schwartz MD, Thomas KA, Weltzin JF (2014) Standardized phenology monitoring methods to track plant and animal activity for science and resource management applications. Int J Biometeorol 58:591–601. CrossRefGoogle Scholar
  14. Enquist CAF, Kellermann JL, Gerst KL, Miller-Rushing AJ (2014) Phenology research for natural resource management in the United States. Int J Biometeorol 58:579–589. CrossRefGoogle Scholar
  15. Fisichelli NA, Schuurman G, Monahan WB, Ziesler P (2015) Protected area tourism in a changing climate: will visitation at US national parks warm up or overheat? PLoS One 10:e0128226CrossRefGoogle Scholar
  16. Forrest JR (2015) Plant-pollinator interactions and phenological change: what can we learn about climate impacts from experiments and observations? Oikos 124:4–13. CrossRefGoogle Scholar
  17. Foster-Smith J, Evans SM (2003) The value of marine ecological data collected by volunteers. Biol Conserv 113:199–213CrossRefGoogle Scholar
  18. Fuccillo KK, Crimmins TM, de Rivera CE, Elder TS (2015) Assessing accuracy in citizen science-based plant phenology monitoring. Int J Biometeorol 59:917–926. CrossRefGoogle Scholar
  19. Gardiner MM, Allee LL, Brown PM et al (2012) Lessons from lady beetles: accuracy of monitoring data from US and UK citizen-science programs. Front Ecol Environ 10:471–476CrossRefGoogle Scholar
  20. Gerst KL, Rossington NL, Mazer SJ (2017) Phenological responsiveness to climate differs among four species of Quercus in North America. J Ecol 105:1610–1622. CrossRefGoogle Scholar
  21. Gallinat AS, Primack RB, Wagner DL (2015) Autumn, the neglected season in climate change research. Trends Ecol Evol 30:169–176. CrossRefGoogle Scholar
  22. Galloway AWE, Tudor MT, Haegen WMV (2006) The reliability of citizen science: a case study of Oregon white oak stand surveys. Wildl Soc Bull 34:1425–1429CrossRefGoogle Scholar
  23. Harrington R, Woiwod I, Sparks TH (1999) Climate change and trophic interactions. Trends Ecol Evol 14:146–150CrossRefGoogle Scholar
  24. Haywood BK, Parrish JK, Dolliver J (2016) Place-based and data-rich citizen science as a precursor for conservation action. Conserv Biol 30(3):476–486Google Scholar
  25. Hegland SJ, Nielsen A, Lázaro A, Bjerknes AL, Totland Ø (2009) How does climate warming affect plant-pollinator interactions? Ecol Lett 12:184–195. CrossRefGoogle Scholar
  26. Inouye DW, Barr B, Armitage K, Inouye BD (2000) Climate change is affecting altitudinal migrants and hibernating species. Proc Natl Acad Sci 97:1630–1633CrossRefGoogle Scholar
  27. Isaac NJB, Pocock MJO (2015) Bias and information in biological records. Biol J Linn Soc 115:522–531. CrossRefGoogle Scholar
  28. Isaac NJB, van Strien AJ, August TA, de Zeeuw MP, Roy DB (2014) Statistics for citizen science: extracting signals of change from noisy ecological data. Methods Ecol Evol 5:1052–1060. CrossRefGoogle Scholar
  29. Jiguet F (2009) Method learning caused a first-time observer effect in a newly started breeding bird survey. Bird Study 56:253–258CrossRefGoogle Scholar
  30. Jordan RC, Brooks WR, Howe DW, Ehrenfeld JG (2012) Evaluating the performance of volunteers in mapping invasive plants in public conservation lands. Environ Manag 49:425–434CrossRefGoogle Scholar
  31. Kosmala M, Wiggins A, Swanson A, Simmons B (2016) Assessing data quality in citizen science. Front Ecol Environ 14:551–560. CrossRefGoogle Scholar
  32. Kramer K, Leinonen I, Loustau D (2000) The importance of phenology for the evaluation of impact of climate change on growth of boreal, temperate, and Mediterranean forest ecosystems: an overview. Int J Biometeorol 44:67–75CrossRefGoogle Scholar
  33. Kellermann JL, van Riper C (2015) Detecting mismatches of bird migration stopover and tree phenology in response to changing climate. Oecologia 178:1227–1238. CrossRefGoogle Scholar
  34. Kullman L (2001) 20th century climate warming and tree-limit rise in the southern Scandes of Sweden. Ambio 30:72–80CrossRefGoogle Scholar
  35. Melaas EK, Friedl MA, Richardson AD (2016) Multiscale modeling of spring phenology across deciduous forests in the eastern United States. Glob Chang Biol 22:792–805. CrossRefGoogle Scholar
  36. Mayor SJ, Guralnick RP, Tingley MW, Otegui J, Withey JC, Elmendorf SC, Andrew ME, Leyk S, Pearse IS, Schneider DC (2017) Increasing phenological asynchrony between spring green-up and arrival of migratory birds. Sci Rep 7:1902. CrossRefGoogle Scholar
  37. McDonough MacKenzie C, Murray G, Primack R, Weihrauch D (2017) Lessons from citizen science: assessing volunteer-collected plant phenology data with mountain watch. Biol Conserv 208:121–126. CrossRefGoogle Scholar
  38. Memmott J, Craze PG, Waser NM, Price MV (2007) Global warming and the disruption of plant-pollinator interactions. Ecol Lett 10:710–717. CrossRefGoogle Scholar
  39. Menzel A, Sparks TH, Estrella N et al (2006) European phenological response to climate change matches the warming pattern. Glob Chang Biol 12:1969–1976. CrossRefGoogle Scholar
  40. Miller-Rushing AJ, Primack RB (2008) Global warming and flowering times in Thoreau’s concord: a community perspective. Ecology 89:332–341CrossRefGoogle Scholar
  41. Miller-Rushing AJ, Høye TT, Inouye DW, Post E (2010) The effects of phenological mismatches on demography. Philos Trans R Soc Lond Ser B Biol Sci 365:3177–3186. CrossRefGoogle Scholar
  42. Miller-Rushing AJ, Evenden A, Gross JE et al (2011) Parks use phenology to improve management and communicate climate change. Park Sci 28:65–71Google Scholar
  43. Møller AP, Rubolini D, Lehikoinen E (2008) Populations of migratory bird species that did not show a phenological response to climate change are declining. Proc Natl Acad Sci U S A 105:16195–16200. CrossRefGoogle Scholar
  44. Monahan WB, Rosemartin A, Gerst KL, Fisichelli NA, Ault T, Schwartz MD, Gross JE, Weltzin JF (2016) Climate change is advancing spring onset across the U.S. national park system. Ecosphere 7:1–17. CrossRefGoogle Scholar
  45. Olesen JM, Bascompte J, Dupont YL, Elberling H, Rasmussen C, Jordano P (2010) Missing and forbidden links in mutualistic networks. Proc Biol Sci 20101371:725–732. CrossRefGoogle Scholar
  46. Parrish JD (1997) Patterns of frugivory and energetic condition in Nearctic landbirds during autumn migration. Condor 99:681–697CrossRefGoogle Scholar
  47. R Development Core Team (2016) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna Available at: Google Scholar
  48. Rafferty NE, Ives AR (2012) Pollinator effectiveness varies with experimental shifts in flowering time. Ecology 93:803–814CrossRefGoogle Scholar
  49. Ratnieks FLW, Schrell F, Sheppard RC, Brown E, Bristow OE, Garbuzov M (2016) Data reliability in citizen science: learning curve and the effects of training method, volunteer background and experience on identification accuracy of insects visiting ivy flowers. Methods Ecol Evol 7:1226–1235. CrossRefGoogle Scholar
  50. Samplonius JM, Kappers EF, Brands S, Both C (2016) Phenological mismatch and ontogenetic diet shifts interactively affect offspring condition in a passerine. J Anim Ecol 85:1255–1264. CrossRefGoogle Scholar
  51. Schenk M, Krauss J, Holzschuh A (2017) Desynchronizations in bee-plant interactions cause severe fitness losses in solitary bees. J Anim Ecol 87:1–10. CrossRefGoogle Scholar
  52. Schwartz MD, Ahas R, Aasa A (2006) Onset of spring starting earlier across the northern hemisphere. Glob Chang Biol 12:343–351CrossRefGoogle Scholar
  53. Starr J, Schweik C, Bush N et al (2014) Lights, camera…citizen science: assessing the effectiveness of smartphone-based video training in invasive plant identification. PLoS One 9:e111433. CrossRefGoogle Scholar
  54. Tredick CA, Lewison RL, Deutschman DH, Hunt TANN, Gordon KL, von Hendy P (2017) A rubric to evaluate citizen-science programs for long-term ecological monitoring. Biosci 67:834–844. CrossRefGoogle Scholar
  55. Van der Wal R, Sharma N, Mellish C et al (2016) The role of automated feedback in training and retaining biological recorders for citizen science. Cons Biol 30:550–561. CrossRefGoogle Scholar
  56. Visser ME, Both C (2005) Shifts in phenology due to global climate change: the need for a yardstick. Proc Biol Sci 272:2561–2569. CrossRefGoogle Scholar
  57. Vitasse Y, Delzon S, Dufrêne E, Pontailler JY, Louvet JM, Kremer A, Michalet R (2009) Leaf phenology sensitivity to temperature in European trees: do within-species populations exhibit similar responses? Agric For Meteorol 149:735–744. CrossRefGoogle Scholar
  58. Wolf AA, Zavaleta EA, Selmants PC (2017) Flowering phenology shifts in response to biodiversity loss. Proc Natl Acad Sci 114:3463–3468CrossRefGoogle Scholar
  59. Wolfe D, Schwartz MD, Lakso AN et al (2005) Climate change and shifts in spring phenology of three horticultural woody perennials in northeastern USA. Int J Biometeorol 49:303–309CrossRefGoogle Scholar
  60. Wood S (2008) Fast stable direct fitting and smoothness selection for generalized additive models. J R Stat Soc B 70:495–518CrossRefGoogle Scholar
  61. Wood S (2011) Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models. J R Staistical Soc B 73:3–36CrossRefGoogle Scholar
  62. Winder M, Schindler DE (2004) Climate change uncouples trophic interactions in an aquatic ecosystem. Ecology 85:2100–2106. CrossRefGoogle Scholar
  63. Walther G-R (2010) Community and ecosystem responses to recent climate change. Philos Trans R Soc Lond Ser B Biol Sci 365:2019–2024. CrossRefGoogle Scholar
  64. Yee TW (2015) Vector generalized linear and additive models: with an implementation in R. Springer, New YorkCrossRefGoogle Scholar
  65. Yee TW, Wild CJ (1996) Vector generalized additive models. J R Stat Soc Ser B 58:481–493Google Scholar

Copyright information

© ISB 2018

Authors and Affiliations

  • Richard E. Feldman
    • 1
    • 2
    • 3
    Email author
  • Irma Žemaitė
    • 4
  • Abraham J. Miller-Rushing
    • 5
  1. 1.Unidad de Recursos NaturalesCentro de Investigación Científica de YucatánMéridaMexico
  2. 2.Schoodic Institute at Acadia National ParkWinter HarborUSA
  3. 3.Department of Environmental ConservationUniversity of MassachusettsAmherstUSA
  4. 4.Department of Environmental SciencesVytautas Magnus UniversityKaunasLithuania
  5. 5.US National Park Service, Acadia National ParkBar HarborUSA

Personalised recommendations