Skip to main content
SpringerLink
Log in

Analysis of quality control outcomes of grass pollen identification and enumeration: experience matters

  • Original Paper
  • Published:
Aerobiologia Aims and scope Submit manuscript

Abstract

Pollen identification and enumeration is subject to human errors, and hence, it is crucial to evaluate the proficiency of pollen counters. Many networks still depend on manual pollen monitoring, and those adopting automation use manual counting data as a reference. A quality control exercise was undertaken across the AusPollen Aerobiology Collaboration Network to compare data analysis methods, gauge factors associated with accuracy, and improvements in counting proficiency. Counters were instructed to count grass and other pollen of the same two slides. Reported pollen concentrations were compared to an approximation of the true concentration values applying the published benchmark approach and alternative approach using bootstrapping technique. Participants were asked about their experience, training and usual practice via an online questionnaire. The majority (92% of 72) of reported values fell within acceptable ranges of variation from approximated true values. Outcomes were similar regardless of analysis approach, but bootstrapping did not require detection of outliers, and worked well with a small sample size with non-normal distribution. Counter reported pollen data were significantly shifted towards better outcomes compared to an initial exercise, and five of eight counters who were tested two times improved. Counting performance seemed not to be associated with amount of training received but was significantly related to counter experience. For future quality control exercises, particularly for small and skewed datasets, confidence limits of true pollen concentrations may be analysed by bootstrapping. Implementation of quality control exercises with harmonised analysis would enhance delivery of reliable pollen information to community, clinicians and government agencies for forecasting and environmental health management.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Data availability

The underlying research data can be accessed upon request to the corresponding author (Janet M. Davies; j36.davies@qut.edu.au).

References

  • Addison-Smith, B., Wraith, D., & Davies, J. M. (2020). Standardising pollen monitoring: Quantifying confidence intervals for measurements of airborne pollen concentration. Aerobiologia. https://doi.org/10.1007/s10453-020-09656-6

    Article  Google Scholar 

  • Adèr, H. J., Mellenbergh, G.J., Hand, D.J. (2008). Advising on research methods: A consultant's companion: Johannes van Kessel Publishing.

  • Alcázar, P., & Comtois, P. (2000). The influence of sampler height and orientation on airborne Ambrosia pollen counts in Montreal. Grana, 39(6), 303–307. https://doi.org/10.1080/00173130052504342

    Article  Google Scholar 

  • AusPollen Aerobiology Standard Working Party (2019). https://allergy.org.au/about-ascia/ascia-initiatives/auspollen. Accessed 21st of August 2020.

  • Australian Institute of Health and Welfare (2019). Allergic rhinitis (‘hay fever'). Canberra: AIHW. Cat. no: PHE 257. https://www.aihw.gov.au/reports/chronic-respiratory-conditions/allergic-rhinitis-hay-fever/contents/allergic-rhinitis. Accessed 4 August 2020.

  • Bannister, T., Ebert, E. E., Silver, J. D., Newbigin, E., Lampugnani, E. R., Hughes, N., et al. (2020). A pilot forecasting system for epidemic thunderstorm asthma in south-eastern Australia. Bulletin of the American Meteorological Society, 102, E399–E420. https://doi.org/10.1175/BAMS-D-19-0140.1.

    Article  Google Scholar 

  • Beggs, P. J., Davies, J. M., Milic, A., Haberle, S. G., Johnston, F. H., Jones, P. J., et al. (2018). Australian Airborne Pollen and Spore Monitoring Network Interim Standard and Protocols. https://www.allergy.org.au/images/stories/pospapers/Australian_Pollen_and_Spore_Monitoring_Interim_Standard_and_Protocols_v2_14092018.pdf. Accessed 03/01/2019.

  • Beggs, P. J., Katelaris, C. H., Medek, D., Johnston, F. H., Burton, P. K., Campbell, B., et al. (2015). Differences in grass pollen allergen exposure across Australia. Australian and New Zealand Journal of Public Health, 39(1), 51–55. https://doi.org/10.1111/1753-6405.12325

    Article  Google Scholar 

  • Berti, G., Isocrono, D., Ropolo, L., Caranci, N., Cesare, M. R., Fossa, V., et al. (2009). An experience of data quality evaluation in pollen monitoring activities. Journal of Environmental Monitoring, 11(4), 788–792.

    Article  CAS  Google Scholar 

  • Bousquet, J., Khaltaev, N., Cruz, A. A., Denburg, J., Fokkens, W., Togias, A., et al. (2008). Allergic rhinitis and its impact on asthma (ARIA) 2008. Allergy, 63(s86), 8–160.

    Article  Google Scholar 

  • Canonica, G., Bousquet, J., Mullol, J., Scadding, G., & Virchow, J. (2007). A survey of the burden of allergic rhinitis in Europe. Allergy, 62, 17–25.

    Article  Google Scholar 

  • Chappuis, C., Tummon, F., Clot, B., Konzelmann, T., Calpini, B., & Crouzy, B. (2019). Automatic pollen monitoring: first insights from hourly data. Aerobiologia, 1–12.

  • Comtois, P., Alcazar, P., & Néron, D. (1999). Pollen counts statistics and its relevance to precision. Aerobiologia, 15(1), 19–28.

    Article  Google Scholar 

  • Crouzy, B., Stella, M., Konzelmann, T., Calpini, B., & Clot, B. (2016). All-optical automatic pollen identification: Towards an operational system. Atmospheric Environment, 140, 202–212. https://doi.org/10.1016/j.atmosenv.2016.05.062

    Article  CAS  Google Scholar 

  • Damialis, A., Kaimakamis, E., Konoglou, M., Akritidis, I., Traidl-Hoffmann, C., & Gioulekas, D. (2017). Estimating the abundance of airborne pollen and fungal spores at variable elevations using an aircraft: How high can they fly? Scientific Reports, 7(1), 1–11.

    Article  Google Scholar 

  • Davies, J. M., Beggs, P. J., Medek, D. E., Newnham, R. M., Erbas, B., Thibaudon, M., et al. (2015). Trans-disciplinary research in synthesis of grass pollen aerobiology and its importance for respiratory health in Australasia. Science of the Total Environment, 534, 85–96. https://doi.org/10.1016/j.scitotenv.2015.04.001

    Article  CAS  Google Scholar 

  • Davies, J. M., Berman, D., Beggs, P. J., Ramón, G. D., Peter, J., Katelaris, C. H., et al. (2021). Global climate change and pollen aeroallergens: A southern hemisphere perspective. Immunology and Allergy Clinics, 41(1), 1–16.

    Article  Google Scholar 

  • Davies, J. M., Erbas, B., Katelaris, C., Newbigin, E., Huete, A., Ebert, E., et al. (2016). The AusPollen partnership: implementing a standardized national pollen alert system for better management of allergic respiratory health. Intern Med J, 46(S4), 13–14. https://doi.org/10.1111/imj.30_13197.

    Article  Google Scholar 

  • Davies, J., Erbas, B., Simunovic, M., Al Kouba, J., Milic, A., & Fagan, D. (2017). Literature review on thunderstorm asthma and its implications for public health advice. Victorian State Government Department of Health and Human Services. https://www2.health.vic.gov.au/about/publications/researchandreports/thunderstorm-asthma-literature-review-may-2107. Accessed 21 July 2020.

  • de Morton, J., Bye, J., Pezza, A., & Newbigin, E. (2011). On the causes of variability in amounts of airborne grass pollen in Melbourne, Australia [Article]. International Journal of Biometeorology, 55(4), 613–622.

    Article  Google Scholar 

  • Department of Health and Human Services Victoria (2017). Epidemic Thunderstorm Asthma Program. https://www2.health.vic.gov.au/public-health/environmental-health/climate-weather-and-public-health/thunderstorm-asthma/program. Accessed 21 July 2020.

  • DiCiccio, T. J., & Efron, B. (1996). Bootstrap confidence intervals. Statistical Science, 11, 189–228. https://doi.org/10.1214/ss/1032280214.

    Article  Google Scholar 

  • Efron, B., & Tibshirani, R. (1991). Statistical data analysis in the computer age. Science, 253(5018), 390–395.

    Article  CAS  Google Scholar 

  • Efron, B., & Tibshirani, R. J. (1994). An introduction to the bootstrap: CRC press.

  • Erbas, B., Jazayeri, M., Lambert, K. A., Katelaris, C. H., Prendergast, L. A., Tham, R., et al. (2018). Outdoor pollen is a trigger of child and adolescent asthma emergency department presentations: A systematic review and meta-analysis. Allergy, 73(8), 1632–1641.

    Article  CAS  Google Scholar 

  • Galán, C., Smith, M., Thibaudon, M., Frenguelli, G., Oteros, J., Gehrig, R., et al. (2014). Pollen monitoring: Minimum requirements and reproducibility of analysis. Aerobiologia, 30(4), 385–395. https://doi.org/10.1007/s10453-014-9335-5

    Article  Google Scholar 

  • Haberle, S. G., Bowman, D. M. J. S., Newnham, R. M., Johnston, F. H., Beggs, P. J., Buters, J., et al. (2014). The macroecology of airborne pollen in Australian and New Zealand urban areas. PLoS ONE. https://doi.org/10.1371/journal.pone.0097925

    Article  Google Scholar 

  • Henderson, A. R. (2005). The bootstrap: a technique for data-driven statistics. Using computer-intensive analyses to explore experimental data. Clinica chimica acta, 359(1), 1–26.

    Article  CAS  Google Scholar 

  • Hirst, J. M. (1952). An automatic volumetric spore trap. The Annals of Applied Biology, 39(2), 257–265. https://doi.org/10.1111/j.1744-7348.1952.tb00904.x

    Article  Google Scholar 

  • Jones, P. J., Koolhof, I. S., Wheeler, A. J., Williamson, G. J., Lucani, C., Campbell, S. L., et al. (2020). Can smartphone data identify the local environmental drivers of respiratory disease? Environmental Research, 182, 109118.

    Article  CAS  Google Scholar 

  • Kmenta, M., Bastl, K., Berger, U., Kramer, M. F., Heath, M. D., Pätsi, S., et al. (2017). The grass pollen season 2015: A proof of concept multi-approach study in three different European cities. World Allergy Organization Journal, 10(1), 31.

    Article  Google Scholar 

  • Medek, D. E., Beggs, P. J., Erbas, B., Jaggard, A. K., Campbell, B. C., Vicendese, D., et al. (2016). Regional and seasonal variation in airborne grass pollen levels between cities of Australia and New Zealand. Aerobiologia, 32(2), 289–302. https://doi.org/10.1007/s10453-015-9399-x

    Article  Google Scholar 

  • Milic, A., Addison-Smith, B., Jones, P. J., Beggs, P. J., Erbas, B., & Davies, J. M. (2019). Quality control of pollen identification and quantification exercise for the AusPollen Aerobiology Collaboration Network: a pilot study. Aerobiologia, 1–5.

  • Mooney, C. Z., & Duvall, R. (1993). Bootstrapping: A nonparametric approach to statistical inference (Vol. 95): Sage University papers series. Quantitative applications in the social sciences.

  • National Environmental Monitoring Site Register AusPollen Aerobiology Collaboration Network. http://www.neii.gov.au/nemsr (add data: auspollen). Accessed July 21 2020.

  • Osborne, J. W., & Overbay, A. (2004). The power of outliers (and why researchers should always check for them). Practical Assessment, Research, and Evaluation, 9(1), 6.

    Google Scholar 

  • Oteros, J., Galán, C., Alcázar, P., & Domínguez-Vilches, E. (2013). Quality control in bio-monitoring networks, Spanish Aerobiology Network. Science of the Total Environment, 443, 559–565. https://doi.org/10.1016/j.scitotenv.2012.11.040

    Article  CAS  Google Scholar 

  • Razali, N. M., & Wah, Y. B. (2011). Power comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson-Darling tests. Journal of Statistical Modeling and Analytics, 2(1), 21–33.

    Google Scholar 

  • Rojo, J., Oteros, J., Pérez-Badia, R., Cervigón, P., Ferencova, Z., Gutiérrez-Bustillo, A. M., et al. (2019). Near-ground effect of height on pollen exposure. Environmental Research, 174, 160–169.

    Article  CAS  Google Scholar 

  • Sauvageat, E., Zeder, Y., Auderset, K., Calpini, B., Clot, B., Crouzy, B., et al. (2019). Real-time pollen monitoring using digital holography. Atmos. Meas. Tech. Discuss., 13, 1539–1550. https://doi.org/10.5194/amt-13-1539-2020.

    Article  CAS  Google Scholar 

  • Sikoparija, B., Galán, C., Smith, M., & Group E. Q. W. (2017). Pollen-monitoring: Between analyst proficiency testing. Aerobiologia, 33(2), 191–199.

    Article  Google Scholar 

  • Silver, J. D., Spriggs, K., Haberle, S. G., Katelaris, C. H., Newbigin, E. J., & Lampugnani, E. R. (2020). Using crowd-sourced allergic rhinitis symptom data to improve grass pollen forecasts and predict individual symptoms. Science of the Total Environment, 720, 137351.

    Article  CAS  Google Scholar 

  • Simunovic, M., Dwarakanath, D., Addison-Smith, B., Susanto, N. H., Erbas, B., Baker, P., et al. (2020). Grass pollen as a trigger of emergency department presentations and hospital admissions for respiratory conditions in the subtropics: A systematic review. Environmental Research, 182, 109125.

    Article  CAS  Google Scholar 

  • Smith, M., Oteros, J., Schmidt-Weber, C., & Buters, J. T. (2019). An abbreviated method for the quality control of pollen counters. Grana, 58(3), 185–190.

    Article  Google Scholar 

  • Thien, F., Beggs, P. J., Csutoros, D., Darvall, J., Hew, M., Davies, J. M., et al. (2018). The Melbourne epidemic thunderstorm asthma event 2016: An investigation of environmental triggers, effect on health services, and patient risk factors. The Lancet Planetary Health, 2(6), e255–e263. https://doi.org/10.1016/S2542-5196(18)30120-7

    Article  Google Scholar 

Download references

Acknowledgements

For their participation in the 2019 QC exercise, we thank participating pollen counters of the AusPollen Aerobiology Collaboration Network including members of the National Health and Medical Research Council AusPollen Partnership (GNT1116107), ARC Discovery (DP170101630), Victorian Thunderstorm Asthma Pollen Surveillance and AirRater projects, as well as coordinators of these pollen and spore monitoring sites. Special thanks go to Dr Penelope J. Jones (University of Tasmania), Ms Victoria Timbrell (Queensland University of Technology) and Dr Jane Al Kouba (Macquarie University) for undertaking the reference counting. We thank Professor Ed Newbigin and Ms Kerryn Popa (The University of Melbourne) for providing the reference slides. For their consultation on data analysis approaches, we thank Professor Bircan Erbas and Dr Don Vicendese (La Trobe University), as well as Dr Zoe Dettrick and Dr Darren Wraith (Queensland University of Technology). This study was funded by the Victorian State Government Department of Health and Human Services through the Epidemic Thunderstorm Asthma Program. The project was also supported by the National Health and Medical Research Council AusPollen Partnership (GNT1116107) and programme management of the Bureau of Meteorology.

Funding

This study was funded by the Victorian State Government Department of Health and Human Services through the Epidemic Thunderstorm Asthma Program. This study was supported by the programme management from the Bureau of Meteorology. The study was also supported by the National Health and Medical Research Council AusPollen Partnership (GNT1116107).

Author information

Authors and Affiliations

  1. Centre for Environment, Centre for Immunology and Infection Control, School of Biomedical Sciences, Queensland University of Technology, Brisbane, QLD, Australia

    Andelija Milic, Beth Addison-Smith, Shanice Van Haeften & Janet M. Davies

  2. Office of Research, Metro North Hospital and Health Service, Herston, QLD, Australia

    Janet M. Davies

Authors
  1. Andelija Milic
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Beth Addison-Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shanice Van Haeften
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Janet M. Davies
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Janet M. Davies.

Ethics declarations

Conflict of interest

Professor Janet M. Davies leads the NHMRC AusPollen Partnership Project (GNT 1116107) with matching cash and in kind co-sponsorship from The Australasian Society for Clinical Immunology and Allergy, Asthma Australia, Bureau of Meteorology, Commonwealth Scientific and Industrial Research Organisation, Stallergenes Australia, Federal Office of Meteorology and Climatology MeteoSwiss, Switzerland. Outside the scope of this study, she is an investigator of current grants from The National Foundation for Medical Research Innovation with co-sponsorship from Abionic Switzerland, The Australian Research Council (LP190100216, DP190100376 and DP170101630) and The Emergency Medicine Foundation. She is a named inventor on patents assigned to QUT. Her institute has received Honorarium payments and travel expenses for education sessions and conference presentations in the last five years from Stallergenes Australia, and WyMedical. Other authors declare no conflicts of interest that are relevant to the content of this article.

Ethical approval

All procedures performed in this study involving human participants were in accordance with the ethical standards of QUT and the Australian Guidelines for Responsible Conduct of Research including the 1964 Helsinki declaration. The study was approved by the QUT Human Research Ethics Committee (1800000161).

Consent to participate

Written informed consent was obtained from all individual participants included in the study.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 28 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Milic, A., Addison-Smith, B., Van Haeften, S. et al. Analysis of quality control outcomes of grass pollen identification and enumeration: experience matters. Aerobiologia 37, 797–808 (2021). https://doi.org/10.1007/s10453-021-09723-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10453-021-09723-6

Keywords

Search

Navigation