Skip to main content
SpringerLink
Log in

Assessing accuracy in citizen science-based plant phenology monitoring

  • Original Paper
  • Published:
International Journal of Biometeorology Aims and scope Submit manuscript

Abstract

In the USA, thousands of volunteers are engaged in tracking plant and animal phenology through a variety of citizen science programs for the purpose of amassing spatially and temporally comprehensive datasets useful to scientists and resource managers. The quality of these observations and their suitability for scientific analysis, however, remains largely unevaluated. We aimed to evaluate the accuracy of plant phenology observations collected by citizen scientist volunteers following protocols designed by the USA National Phenology Network (USA-NPN). Phenology observations made by volunteers receiving several hours of formal training were compared to those collected independently by a professional ecologist. Approximately 11,000 observations were recorded by 28 volunteers over the course of one field season. Volunteers consistently identified phenophases correctly (91 % overall) for the 19 species observed. Volunteers demonstrated greatest overall accuracy identifying unfolded leaves, ripe fruits, and open flowers. Transitional accuracy decreased for some species/phenophase combinations (70 % average), and accuracy varied significantly by phenophase and species (p < 0.0001). Volunteers who submitted fewer observations over the period of study did not exhibit a higher error rate than those who submitted more total observations. Overall, these results suggest that volunteers with limited training can provide reliable observations when following explicit, standardized protocols. Future studies should investigate different observation models (i.e., group/individual, online/in-person training) over subsequent seasons with multiple expert comparisons to further substantiate the ability of these monitoring programs to supply accurate broadscale datasets capable of answering pressing ecological questions about global change.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Bas Y, Devictor V, Moussus JP, Jiguet F (2008) Accounting for weather and time-of-day parameters when analyzing count data from monitoring programs. Biodivers Conserv 17:3403–3416

    Article  Google Scholar 

  • Beaubien EG, Hamann A (2011) Plant phenology networks of citizen scientists: recommendations from two decades of experience in Canada. Int J Biometeorol 55:833–841

    Article  Google Scholar 

  • Bell JJ (2007) The use of volunteers for conducting sponge biodiversity assessments and monitoring using a morphological approach on Indo-Pacific coral reefs. Aquat Conserv 17:133–145

    Article  Google Scholar 

  • Both C, van Asch M, Bijlsma RG, van den Burg AB, Visser ME (2009) Climate change and unequal phenological changes across four trophic levels: constraints or adaptations? J Anim Ecol 78:73–83

    Article  Google Scholar 

  • Boudreau SA, Yan ND (2004) Auditing the accuracy of a volunteer-based surveillance program for an aquatic invader Bythotrephes. Environ Monit Assess 91:17–26

    Article  Google Scholar 

  • Brossard D, Lewenstein B, Bonney R (2005) Scientific knowledge and attitude change: the impact of a citizen science project. Int J Sci Educ 27:1099–1121

    Article  Google Scholar 

  • Cleland EE, Chuine I, Menzel A, Mooney HA, Schwartz MD (2007) Shifting plant phenology in response to global change. Trends Ecol Evol 22:357–365

    Article  Google Scholar 

  • Cooper CB (2013) Is there a weekend bias in clutch-initiation dates from citizen science? Implications for studies of avian breeding phenology. Int J Biometeorol 58:1415–1419

  • Cooper CB, Dickinson J, Phillips TB, Bonney R (2007) Citizen science as a tool for conservation in residential ecosystems. Ecol Soc 12:11

    Google Scholar 

  • Couter JR, Johnson RJ, Stuyck CM, Lang BA, Kaiser EW (2012) Weekend bias in citizen science data reporting: implications for phenology studies. Int J Biometeorol 57:715–720

    Article  Google Scholar 

  • Crall, AW, Newman GJ, Stohlgren TJ, Holfelder KA, Graham J, DM Waller (2011) Assessing citizen science data quality: an invasive species case study. Conservation Letters 0:1–10

  • Delaney DG, Sperling CD, Adams CS, Leung B (2008) Marine invasive species: validation of citizen science and implications for national monitoring networks. Biol Invasions 10:117–128

    Article  Google Scholar 

  • Dickinson JL, Zuckerberg B, Bonter DN (2010) Citizen science as an ecological research tool: challenges and benefits. Annu Rev Ecol Evol Syst 4:149–172

    Article  Google Scholar 

  • Edwards M, Richardson AJ (2004) Impact of climate change on marine pelagic phenology and trophic mismatch. Nature 430:881–884

    Article  CAS  Google Scholar 

  • Fore LS, Paulsen K, O’Laughlin K (2001) Assessing the performance of volunteers in monitoring streams. Freshwater Biol 46(1):109–123

    Article  Google Scholar 

  • Foster-Smith J, Evans SM (2003) The value of marine ecological data collected by volunteers. Biol Conserv 113:199–213

    Article  Google Scholar 

  • Galloway AWE, Tudor MT, Vander Haegen WM (2006) The reliability of citizen science: a case study of Oregon white oak stand surveys. Wildlife Soc B 34:1425–1429

    Article  Google Scholar 

  • Gardiner MM, Allee LL, Brown PMJ, Losey JE, Roy HE, Smyth RR (2012) Lessons from lady beetles: accuracy of monitoring data from US and UK citizen-science programs. Front Ecol Environ 10:471–476

    Article  Google Scholar 

  • Greenwood J (2007) Citizens, science and bird conservation. J Ornithol 148:77–124

    Article  Google Scholar 

  • Haklay M (2010) How good is volunteered geographical information? A comparative study of OpenStreetMap and ordnance survey datasets. Environ Plann B 37:682–703

    Article  Google Scholar 

  • Ingwell LI, Preisser EL (2010) Using citizen science programs to identify host resistance in pest-invaded forests. Conserv Biol 25:182–188

    Article  Google Scholar 

  • Jiguet F (2009) Method learning caused by a first-time observer effect in a newly started breeding bird survey. Bird Study 56:253–258

    Article  Google Scholar 

  • Koch E (2010) Global framework for data collection—data bases, data availability, future networks, online databases. In: Hudson IL, Keatley MR (eds) Phenological research: methods for environmental and climate change analysis. Springer, Dordrecht, pp 23–61

    Chapter  Google Scholar 

  • Kremen C, Ullmann KS, Thorp RW (2010) Evaluating the quality of citizen-scientist data on pollinator communities. Conserv Biol 25:607–617

    Article  Google Scholar 

  • Lovell S, Hamer M, Slotow R, Herbert D (2009) An assessment of the use of volunteers for terrestrial invertebrate biodiversity surveys. Biodivers Conserv 18:3295–3307

    Article  Google Scholar 

  • Memmott J, Craze PG, Waser NM, Price MV (2007) Global warming and the disruption of plant-pollinator interactions. Ecol Lett 10:710–717

    Article  Google Scholar 

  • Menzel A, Sparks TH, Estrella N, Koch E, Aasa A, Ahas R, Alm-Kubler K, Bissolli P, Braslavska O, Briede A et al (2006) European phenological response to climate change matches the warming pattern. Glob Chang Biol 12:1969–1976

  • Nerbonne JF, Vondracek B (2003) Volunteer macroinvertebrate monitoring: assessing training needs through examining error and bias in untrained volunteers. J N Am Benthol Soc 22:152–163

    Article  Google Scholar 

  • Parmesan C, Yohe G (2003) A globally coherent fingerprint of climate change impacts across natural systems. Nature 421:37–42

    Article  CAS  Google Scholar 

  • Pearce-Higgins JW, Yalden DW, Whittingham MJ (2005) Warmer springs advance the breeding phenology of golden plovers Pluvialis apricaria and their prey (Tipulidae). Oecologia 143:470–476

    Article  CAS  Google Scholar 

  • Root TL, Price JT, Hall KR, Schneider SH, Rosenzweig C, Pounds JA (2003) Fingerprints of global warming on wild animals and plants. Nature 421:57–60

    Article  CAS  Google Scholar 

  • Schmeller DS, Henry P-Y, Julliard R, Gruber B, Clobert J, Dziock F, Lengyel S, Nowicki P, Déri E, Budrys E, Kull T, Tali K, Bauch B, Settele J, Van Swaay C, Kobler A, Babij V, Papastergiadou E, Henle K (2009) Advantages of volunteer-based biodiversity monitoring in Europe. Conserv Biol 23:307–316

    Article  Google Scholar 

  • Siebeck UE, Marshall NJ, Klüter A, Hoegh-Guldberg O (2006) Monitoring coral bleaching using a colour reference card. Coral Reefs 25:453–460

    Article  Google Scholar 

  • Sparks TH, Huber K, Tryianowski P (2008) Something for the weekend? Examining the bias in avian phenological recording. Int J Biometeorol 52:505–510

    Article  Google Scholar 

  • Thackeray SJ, Sparks TH, Frederiksen M, Burthe S, Bacon PJ, Bell JR, Botham MS, Brereton TM, Bright PW, Carvalho L, Clutton-Brock T, Dawson A, Edwards M, Elliott JM, Harrington R, Johns D, Jones ID, Jones JT, Leech DI, Roy DB, Scott WA, Smith M, Smithers RJ, Winfield IJ, Wanless S (2010) Trophic level asynchrony in rates of phenological change for marine, freshwater and terrestrial environments. Glob Change Biol 16:3304–3313

    Article  Google Scholar 

  • Tregidgo, DJ, SE West, and MR Ashmore (2014) Can citizen science produce good science? Testing the OPAL Air Survey methodology, using lichens as indicators of nitrogenous pollution. Environ Pollut 182:448–451

  • Trumbull D, Bonney R, Bascom D, Cabral A (2000) Thinking scientifically during participation in a citizen-science project. Sci Educ 84:265–275

    Article  Google Scholar 

  • Walther G, Post E, Convey P, Menzel A, Parmesan C, Beebee TJC, Fromentin J-M, Hoegh-Guldberg O, Bairlein F (2002) Ecological responses to recent climate changes. Nature 416:389–395

    Article  CAS  Google Scholar 

  • Winder M, Schindler D (2004) Climate change uncouples trophic interactions in an aquatic system. Ecology 85:2100–2106

    Article  Google Scholar 

  • Wolkovich EM, Cook BI, Allen JM, Crimmins TM, Travers S, Pau S, Regetz J, Davies TJ, Betancourt JL, Kraft NJB, Ault TR, Bolmgren K, Mazer SJ, McCabe GJ, McGill BJ, Parmesan C, Salamin N, Schwartz MD, Cleland EE (2012) Warming experiments underpredict plant phenological responses to climate change. Nature 485:494–497

    CAS  Google Scholar 

Download references

Acknowledgments

Sincere thanks to all contributors to this study including Lori Anne Barnett, Ellen Denny, Alyssa Rosemartin, Jake Weltzin, and USA-NPN National Coordinating Office staff, Dr. Marion Dresner, Dr. Sybil Kelly, Kelly Fisher, Jill Van Winkle, and Rhiannon LeFay from Portland State University, all volunteers from Portland Budwatch (with a special thanks to Harriet Denison and Mark Feldman for their dedication to the project), and Kendra Peterson-Morgan and Rachel Feliz of Portland Parks and Recreation. The USA-NPN National Coordinating Office is grateful for the critical role this study played in refining plant phenology protocols. This project was funded in part by NSF award #0941763, Interactive Teaching Materials for Understanding Ecological Response from Climate Change in Urban Forests.

Author information

Authors and Affiliations

  1. Department of Environmental Science and Management, Portland State University, Portland, OR, 97207-0751, USA

    Kerissa K. Fuccillo, Catherine E. de Rivera & Timothy S. Elder

  2. USA National Phenology Network, National Coordinating Office, 1955 E. Sixth St., Tucson, AZ, 85721, USA

    Theresa M. Crimmins

Authors
  1. Kerissa K. Fuccillo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Theresa M. Crimmins
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Catherine E. de Rivera
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Timothy S. Elder
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kerissa K. Fuccillo.

Electronic supplementary material

Below is the link to the electronic supplementary material.

ESM 1

(PDF 234 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fuccillo, K.K., Crimmins, T.M., de Rivera, C.E. et al. Assessing accuracy in citizen science-based plant phenology monitoring. Int J Biometeorol 59, 917–926 (2015). https://doi.org/10.1007/s00484-014-0892-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00484-014-0892-7

Keywords

Search

Navigation