In the USA, thousands of volunteers are engaged in tracking plant and animal phenology through a variety of citizen science programs for the purpose of amassing spatially and temporally comprehensive datasets useful to scientists and resource managers. The quality of these observations and their suitability for scientific analysis, however, remains largely unevaluated. We aimed to evaluate the accuracy of plant phenology observations collected by citizen scientist volunteers following protocols designed by the USA National Phenology Network (USA-NPN). Phenology observations made by volunteers receiving several hours of formal training were compared to those collected independently by a professional ecologist. Approximately 11,000 observations were recorded by 28 volunteers over the course of one field season. Volunteers consistently identified phenophases correctly (91 % overall) for the 19 species observed. Volunteers demonstrated greatest overall accuracy identifying unfolded leaves, ripe fruits, and open flowers. Transitional accuracy decreased for some species/phenophase combinations (70 % average), and accuracy varied significantly by phenophase and species (p < 0.0001). Volunteers who submitted fewer observations over the period of study did not exhibit a higher error rate than those who submitted more total observations. Overall, these results suggest that volunteers with limited training can provide reliable observations when following explicit, standardized protocols. Future studies should investigate different observation models (i.e., group/individual, online/in-person training) over subsequent seasons with multiple expert comparisons to further substantiate the ability of these monitoring programs to supply accurate broadscale datasets capable of answering pressing ecological questions about global change.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Bas Y, Devictor V, Moussus JP, Jiguet F (2008) Accounting for weather and time-of-day parameters when analyzing count data from monitoring programs. Biodivers Conserv 17:3403–3416
Beaubien EG, Hamann A (2011) Plant phenology networks of citizen scientists: recommendations from two decades of experience in Canada. Int J Biometeorol 55:833–841
Bell JJ (2007) The use of volunteers for conducting sponge biodiversity assessments and monitoring using a morphological approach on Indo-Pacific coral reefs. Aquat Conserv 17:133–145
Both C, van Asch M, Bijlsma RG, van den Burg AB, Visser ME (2009) Climate change and unequal phenological changes across four trophic levels: constraints or adaptations? J Anim Ecol 78:73–83
Boudreau SA, Yan ND (2004) Auditing the accuracy of a volunteer-based surveillance program for an aquatic invader Bythotrephes. Environ Monit Assess 91:17–26
Brossard D, Lewenstein B, Bonney R (2005) Scientific knowledge and attitude change: the impact of a citizen science project. Int J Sci Educ 27:1099–1121
Cleland EE, Chuine I, Menzel A, Mooney HA, Schwartz MD (2007) Shifting plant phenology in response to global change. Trends Ecol Evol 22:357–365
Cooper CB (2013) Is there a weekend bias in clutch-initiation dates from citizen science? Implications for studies of avian breeding phenology. Int J Biometeorol 58:1415–1419
Cooper CB, Dickinson J, Phillips TB, Bonney R (2007) Citizen science as a tool for conservation in residential ecosystems. Ecol Soc 12:11
Couter JR, Johnson RJ, Stuyck CM, Lang BA, Kaiser EW (2012) Weekend bias in citizen science data reporting: implications for phenology studies. Int J Biometeorol 57:715–720
Crall, AW, Newman GJ, Stohlgren TJ, Holfelder KA, Graham J, DM Waller (2011) Assessing citizen science data quality: an invasive species case study. Conservation Letters 0:1–10
Delaney DG, Sperling CD, Adams CS, Leung B (2008) Marine invasive species: validation of citizen science and implications for national monitoring networks. Biol Invasions 10:117–128
Dickinson JL, Zuckerberg B, Bonter DN (2010) Citizen science as an ecological research tool: challenges and benefits. Annu Rev Ecol Evol Syst 4:149–172
Edwards M, Richardson AJ (2004) Impact of climate change on marine pelagic phenology and trophic mismatch. Nature 430:881–884
Fore LS, Paulsen K, O’Laughlin K (2001) Assessing the performance of volunteers in monitoring streams. Freshwater Biol 46(1):109–123
Foster-Smith J, Evans SM (2003) The value of marine ecological data collected by volunteers. Biol Conserv 113:199–213
Galloway AWE, Tudor MT, Vander Haegen WM (2006) The reliability of citizen science: a case study of Oregon white oak stand surveys. Wildlife Soc B 34:1425–1429
Gardiner MM, Allee LL, Brown PMJ, Losey JE, Roy HE, Smyth RR (2012) Lessons from lady beetles: accuracy of monitoring data from US and UK citizen-science programs. Front Ecol Environ 10:471–476
Greenwood J (2007) Citizens, science and bird conservation. J Ornithol 148:77–124
Haklay M (2010) How good is volunteered geographical information? A comparative study of OpenStreetMap and ordnance survey datasets. Environ Plann B 37:682–703
Ingwell LI, Preisser EL (2010) Using citizen science programs to identify host resistance in pest-invaded forests. Conserv Biol 25:182–188
Jiguet F (2009) Method learning caused by a first-time observer effect in a newly started breeding bird survey. Bird Study 56:253–258
Koch E (2010) Global framework for data collection—data bases, data availability, future networks, online databases. In: Hudson IL, Keatley MR (eds) Phenological research: methods for environmental and climate change analysis. Springer, Dordrecht, pp 23–61
Kremen C, Ullmann KS, Thorp RW (2010) Evaluating the quality of citizen-scientist data on pollinator communities. Conserv Biol 25:607–617
Lovell S, Hamer M, Slotow R, Herbert D (2009) An assessment of the use of volunteers for terrestrial invertebrate biodiversity surveys. Biodivers Conserv 18:3295–3307
Memmott J, Craze PG, Waser NM, Price MV (2007) Global warming and the disruption of plant-pollinator interactions. Ecol Lett 10:710–717
Menzel A, Sparks TH, Estrella N, Koch E, Aasa A, Ahas R, Alm-Kubler K, Bissolli P, Braslavska O, Briede A et al (2006) European phenological response to climate change matches the warming pattern. Glob Chang Biol 12:1969–1976
Nerbonne JF, Vondracek B (2003) Volunteer macroinvertebrate monitoring: assessing training needs through examining error and bias in untrained volunteers. J N Am Benthol Soc 22:152–163
Parmesan C, Yohe G (2003) A globally coherent fingerprint of climate change impacts across natural systems. Nature 421:37–42
Pearce-Higgins JW, Yalden DW, Whittingham MJ (2005) Warmer springs advance the breeding phenology of golden plovers Pluvialis apricaria and their prey (Tipulidae). Oecologia 143:470–476
Root TL, Price JT, Hall KR, Schneider SH, Rosenzweig C, Pounds JA (2003) Fingerprints of global warming on wild animals and plants. Nature 421:57–60
Schmeller DS, Henry P-Y, Julliard R, Gruber B, Clobert J, Dziock F, Lengyel S, Nowicki P, Déri E, Budrys E, Kull T, Tali K, Bauch B, Settele J, Van Swaay C, Kobler A, Babij V, Papastergiadou E, Henle K (2009) Advantages of volunteer-based biodiversity monitoring in Europe. Conserv Biol 23:307–316
Siebeck UE, Marshall NJ, Klüter A, Hoegh-Guldberg O (2006) Monitoring coral bleaching using a colour reference card. Coral Reefs 25:453–460
Sparks TH, Huber K, Tryianowski P (2008) Something for the weekend? Examining the bias in avian phenological recording. Int J Biometeorol 52:505–510
Thackeray SJ, Sparks TH, Frederiksen M, Burthe S, Bacon PJ, Bell JR, Botham MS, Brereton TM, Bright PW, Carvalho L, Clutton-Brock T, Dawson A, Edwards M, Elliott JM, Harrington R, Johns D, Jones ID, Jones JT, Leech DI, Roy DB, Scott WA, Smith M, Smithers RJ, Winfield IJ, Wanless S (2010) Trophic level asynchrony in rates of phenological change for marine, freshwater and terrestrial environments. Glob Change Biol 16:3304–3313
Tregidgo, DJ, SE West, and MR Ashmore (2014) Can citizen science produce good science? Testing the OPAL Air Survey methodology, using lichens as indicators of nitrogenous pollution. Environ Pollut 182:448–451
Trumbull D, Bonney R, Bascom D, Cabral A (2000) Thinking scientifically during participation in a citizen-science project. Sci Educ 84:265–275
Walther G, Post E, Convey P, Menzel A, Parmesan C, Beebee TJC, Fromentin J-M, Hoegh-Guldberg O, Bairlein F (2002) Ecological responses to recent climate changes. Nature 416:389–395
Winder M, Schindler D (2004) Climate change uncouples trophic interactions in an aquatic system. Ecology 85:2100–2106
Wolkovich EM, Cook BI, Allen JM, Crimmins TM, Travers S, Pau S, Regetz J, Davies TJ, Betancourt JL, Kraft NJB, Ault TR, Bolmgren K, Mazer SJ, McCabe GJ, McGill BJ, Parmesan C, Salamin N, Schwartz MD, Cleland EE (2012) Warming experiments underpredict plant phenological responses to climate change. Nature 485:494–497
Sincere thanks to all contributors to this study including Lori Anne Barnett, Ellen Denny, Alyssa Rosemartin, Jake Weltzin, and USA-NPN National Coordinating Office staff, Dr. Marion Dresner, Dr. Sybil Kelly, Kelly Fisher, Jill Van Winkle, and Rhiannon LeFay from Portland State University, all volunteers from Portland Budwatch (with a special thanks to Harriet Denison and Mark Feldman for their dedication to the project), and Kendra Peterson-Morgan and Rachel Feliz of Portland Parks and Recreation. The USA-NPN National Coordinating Office is grateful for the critical role this study played in refining plant phenology protocols. This project was funded in part by NSF award #0941763, Interactive Teaching Materials for Understanding Ecological Response from Climate Change in Urban Forests.
Electronic supplementary material
Below is the link to the electronic supplementary material.
(PDF 234 kb)
About this article
Cite this article
Fuccillo, K.K., Crimmins, T.M., de Rivera, C.E. et al. Assessing accuracy in citizen science-based plant phenology monitoring. Int J Biometeorol 59, 917–926 (2015). https://doi.org/10.1007/s00484-014-0892-7
- Citizen science
- Data accuracy
- Nature’s Notebook
- USA National Phenology Network