Skip to main content
SpringerLink
Log in

Data Mining Approaches to the Real-Time Monitoring and Early Warning of Convective Weather Using Lightning Data

  • Chapter
  • First Online:
Towards Mathematics, Computers and Environment: A Disasters Perspective

Abstract

Tracking and monitoring of convective events may require the analysis of a huge amount of data from sensors on the ground, such as weather radars, or on board of satellites, in addition to the forecasts of numerical models. Thunderstorms associated with severe convective events have a potential to cause strong winds, floods, and landslides, with serious environmental and socio-economic impacts. New approaches based on data mining have been proposed for countries like Brazil that lack a complete weather radar coverage, but have some ground-based lightning detector networks. Lightning data may help to visualize the current state of convective systems in near real-time or to estimate the amount of convective precipitation in a given area and period of time. Data mining algorithms can be trained using numerical model data and lightning data yielding specific data mining models, which can be used to predict the occurrence of convective activity from numerical model forecasts. These data mining models may help meteorologists to improve the accuracy of early warnings and forecastings, mainly in countries that lack a complete weather radar coverage.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adler, R.F., Fenn, D.D.: Thunderstorm intensity as determined from satellite data. J. Appl. Meteorol. 18(4), 502–517 (1979)

    Article  Google Scholar 

  2. Betz, H.D., Schmidt, K., Oettinger, W.P., Montag, B.: Cell-tracking with lightning data from LINET. Adv. Geosci. 17, 55–61 (2008)

    Article  Google Scholar 

  3. Browning, K.A.: Review lecture: local weather forecasting. In: Proceedings of the Royal Society of London. Series A: Mathematical, Physical, and Engineering Sciences, vol. 371, pp. 179–211. The Royal Society, London (1980)

    Google Scholar 

  4. Calheiros, A.J.P., Machado, L.A.T.: The HydroTrack: a nowcasting application using GOES data. In: Current Problems in Atmospheric Radiation (IRS 2008): Proceedings of the International Radiation Symposium (IRC/IAMAS), vol. 1100, pp. 361–364. AIP Conference Proceedings (2009)

    Google Scholar 

  5. Chronis, T., Carey, L.D., Schultz, C.J., Schultz, E.V., Calhoun, K.M., Goodman, S.J.: Exploring lightning jump characteristics. Weather Forecast. 30(1), 23–37 (2015)

    Article  Google Scholar 

  6. Chronis, T., Lang, T., Koshak, W., Blakeslee, R., Christian, H., McCaul, E., Bailey, J.: Diurnal characteristics of lightning flashes detected over the São Paulo lightning mapping array. J. Geophys. Res. Atmos. 120(23) (2015)

    Google Scholar 

  7. Freitas, S., et al.: The Brazilian developments on the regional atmospheric modeling system (BRAMS 5.2): an integrated environmental model tuned for tropical areas. Geosci. Model Dev. 10(5), 189–222 (2017)

    Google Scholar 

  8. Garcia, J.V.C.: Monitoring and prediction of convective events using data mining approaches. Ph.D. thesis, Applied Computing Post-graduate Program, INPE, Brazil (2014)

    Google Scholar 

  9. Garcia, J.V.C., Stephany, S., D’Oliveira, A.B.: Estimation of convective precipitation mass from lightning data using a temporal sliding-window for a series of thunderstorms in Southeastern Brazil. Atmos. Sci. Lett. 14, 281–286 (2013)

    Article  Google Scholar 

  10. Goodman, S.J., Blakeslee, R.J., Koshak, W.J., Mach, D., Bailey, J., Buechler, D., Carey, L., Schultz, C., Bateman, M., McCaul, E., et al.: The GOES-R geostationary lightning mapper (GLM). Atmos. Res. 125, 34–49 (2013)

    Article  Google Scholar 

  11. Han, J., Kambler, M.: Data Mining – Concepts and Techniques, 3 edn. Elsevier, New York (2011)

    Google Scholar 

  12. Harris, R.J., Mecikalski, J.R., MacKenzie Jr, W.M., Durkee, P.A., Nielsen, K.E.: The definition of GOES infrared lightning initiation interest fields. J. Appl. Meteorol. Climatol. 49(12), 2527–2543 (2010)

    Article  Google Scholar 

  13. Haykin, S.O.: Neural Networks and Learning Machines, 3 edn. Pearson/Prentice-Hall, Inc., Upper Saddle River (2008)

    Google Scholar 

  14. Hinneburg, A., Gabriel, H.H.: DENCLUE 2.0: fast clustering based on kernel density estimation. In: Berthold, M.R., Shawe-Taylor, J., Lavraĉ, N. (eds.) Advances in Intelligent Data Analysis VII. Lecture Notes in Computer Science, vol. 4723, pp. 70–80. Springer, Berlin (2007)

    Google Scholar 

  15. Karagiannidis, A., Lagouvardos, K., Kotroni, V.: The use of lightning data and Meteosat infrared imagery for the nowcasting of lightning activity. Atmos. Res. 168, 57–69 (2016)

    Article  Google Scholar 

  16. Lang, T.J., Rutledge, S.A.: Relationships between convective storm kinematics, precipitation, and lightning. Mon. Weather Rev. 130, 2492–2506 (2002)

    Article  Google Scholar 

  17. Lang, T.J., Rutledge, S.A.: A framework for the statistical analysis of large radar and lightning datasets: results from STEPS 2000. Mon. Weather Rev. 139(8), 2536–2551 (2011)

    Article  Google Scholar 

  18. Lima, G.R.T., Stephany, S.: A new classification approach for detecting severe weather patterns. Comput. Geosci. 57, 158–165 (2013)

    Article  Google Scholar 

  19. Lima, G.R.T., Stephany, S.: Training a neural network to detect patterns associated with severe weather. Learn Nonlinear Models 11, 123–152 (2013)

    Article  Google Scholar 

  20. López, R.E., Aubagnac, J.P.: The lightning activity of a hailstorm as a function of changes in its microphysical characteristics inferred from polarimetric radar observations. J. Geophys. Res. Atmos. 102(D14), 16,799–16,813 (1997)

    Article  Google Scholar 

  21. Lund, N.R., MacGorman, D.R., Schuur, T.J., Biggerstaff, M.I., Rust, W.D.: Relationships between lightning location and polarimetric radar signatures in a small mesoscale convective system. Mon. Weather Rev. 137(12), 4151–4170 (2009)

    Article  Google Scholar 

  22. Machado, L.A.T., Silva Dias, M.A.F., Morales, C., Fisch, G., Vila, D., Albrecht, R., Goodman, S.J., Calheiros, A.J.P., Biscaro, T., Kummerow, C., et al.: The CHUVA project: How does convection vary across Brazil? Bull. Am. Meteorol. Soc. 95(9), 1365–1380 (2014)

    Article  Google Scholar 

  23. Matthee, R., Mecikalski, J.R., Carey, L.D., Bitzer, P.M.: Quantitative differences between lightning and nonlightning convective rainfall events as observed with polarimetric radar and MSG satellite data. Mon. Weather Rev. 142(10), 3651–3665 (2014)

    Article  Google Scholar 

  24. Mecikalski, J.R., Bedka, K.M., Paech, S.J., Litten, L.A.: A statistical evaluation of GOES cloud-top properties for nowcasting convective initiation. Mon. Weather Rev. 136(12), 4899–4914 (2008)

    Article  Google Scholar 

  25. Mecikalski, J.R., Li, X., Carey, L.D., McCaul Jr, E.W., Coleman, T.A.: Regional comparison of GOES cloud-top properties and radar characteristics in advance of first-flash lightning initiation. Mon. Weather Rev. 141(1), 55–74 (2013)

    Article  Google Scholar 

  26. Mesinger, F., et al.: The step-mountain coordinate: model description and performance for cases of Alpine Lee cyclogenesis and for a case of an Appalachian redevelopment. Mon. Weather Rev. 116(7), 1493–1518 (1988)

    Article  Google Scholar 

  27. Meyer, V.K., Höller, H., Betz, H.D.: Automated thunderstorm tracking: utilization of three-dimensional lightning and radar data. Atmos. Chem. Phys. 13(10), 5137–5150 (2013)

    Article  Google Scholar 

  28. Moller, A.R.: Severe local storms forecasting. Meteorol. Monogr. 28(50), 433–480 (2001)

    Article  Google Scholar 

  29. Naccarato, K., Pinto Jr, O.: Improvements in the detection efficiency model for the Brazilian lightning detection network (BrasilDAT). Atmos. Res. 91(2), 546–563 (2009)

    Article  Google Scholar 

  30. Pawlak, Z.: Rough sets. Int. J. Comput. Inf. Sci. 11(5), 341–356 (1982)

    Article  Google Scholar 

  31. Pawlak, Z.: Rough Sets – Theoretical Aspects of Reasoning About Data. Kluwer Academic Publishers, Dordrecht (1991)

    MATH  Google Scholar 

  32. Pessoa, A.S.A.: Prediction of severe events from meteorological model outputs employing the Rough Sets Theory and metaheuristics for attribute reduction. Ph.D. thesis, Applied Computing Post-graduate Program, INPE, Brazil (2014)

    Google Scholar 

  33. Pessoa, A.S.A., Stephany, S.: An innovative approach for attribute reduction in Rough Set Theory. Intell. Inf. Manag. 06, 223–239 (2014)

    Google Scholar 

  34. Pierce, C., Seed, A., Ballard, S., Simonin, D., Li, Z.: Nowcasting. In: Bech, J. (ed.) Doppler Radar Observations – Weather Radar, Wind Profiler, Ionospheric Radar, and Other Advanced Applications. InTech, London (2012). https://doi.org/10.5772/39054. https://www.intechopen.com/books/doppler-radar-observations-weather-radar-wind-profiler-ionospheric-radar-and-other-advanced-applications/nowcasting

  35. Reynolds, D.W.: Observations of damaging hailstorms from geosynchronous satellite digital data. Mon. Weather Rev. 108(3), 337–348 (1980)

    Article  Google Scholar 

  36. Roberts, R.D., Burgess, D., Meister, M.: Developing tools for nowcasting storm severity. Weather Forecast. 21(4), 540–558 (2006)

    Article  Google Scholar 

  37. Samarasinghe, S.: Neural Networks for Applied Sciences and Engineering: from Fundamentals to Complex Pattern Recognition. Auerbach Publications, New York (2006)

    Book  Google Scholar 

  38. Schmetz, J., Tjemkes, S.A., Gube, M., Van de Berg, L.: Monitoring deep convection and convective overshooting with METEOSAT. Adv. Space Res. 19(3), 433–441 (1997)

    Article  Google Scholar 

  39. Schultz, C.J., Petersen, W.A., Carey, L.D.: Preliminary development and evaluation of lightning jump algorithms for the real-time detection of severe weather. J. Appl. Meteorol. Climatol. 48(12), 2543–2563 (2009)

    Article  Google Scholar 

  40. Schultz, E., Schultz, C.J., Carey, L.D., Cecil, D.J., Bateman, M.: Automated storm tracking and the lightning jump algorithm using GOES-R Geostationary Lightning Mapper (GLM) proxy data. J. Oper. Meteorol. 3(1), 1–7 (2016)

    Google Scholar 

  41. Scott, D.W.: Multivariate Density Estimation – Theory, Practice and Visualization. John Wiley & Sons, Inc., New York (1992)

    Book  Google Scholar 

  42. Silverman, B.W.: Density Estimation for Statistics and Data Analysis. Chapman and Hall, London (1986)

    Book  Google Scholar 

  43. Sist, M., Zauli, F., Biron, D., Melfi, D.: A study about the correlation link between lightning data and meteorological data. In: 2010 EUMETSAT Meteorological Satellite Conference, Córdoba, vol. 1 (2010)

    Google Scholar 

  44. Skamarock, W.C., Klemp, J.B., Dudhia, J., Gill, D.O., Barker, D.M., Duda, M.G., Huang, X.Y., Wang, W., Powers, J.G.: A description of the advanced research WRF version 3. Tech. Rep. NCAR/TN-4751STR, NCAR (2008)

    Google Scholar 

  45. Steinacker, R., Dorninger, M., Wölfelmaier, F., Krennert, T.: Automatic tracking of convective cells and cell complexes from lightning and radar data. Meteorol. Atmos. Phys. 72(2), 101–110 (2000)

    Article  Google Scholar 

  46. Steiner, M., Houze Jr., R.A., Yuter, S.E.: Climatological characterization of three-dimensional storm structure from operational radar and rain gauge data. J. Appl. Meteorol. 34(9), 1978–2007 (1995)

    Article  Google Scholar 

  47. Strauss, C.: Monitoring and prediction of convective events using data mining approaches. Ph.D. thesis, Applied Computing Post-graduate Program, INPE, Brazil (2013)

    Google Scholar 

  48. Strauss, C., Stephany, S., Caetano, M.: A ferramenta EDDA de geração de campos de densidade de descargas atmosféricas para mineração de dados meteorológicos. In: Anais…, vol. 3, pp. 269–275. Congr. Nac. de Mat. Apl. e Comput., SBMAC, São Carlos (2010)

    Google Scholar 

  49. Strauss, C., Rosa, M.B., Stephany, S.: Spatio-temporal clustering and density estimation of lightning data for the tracking of convective events. Atmos. Res. 134, 87–99 (2013)

    Article  Google Scholar 

  50. Sun, J.: Convective-scale assimilation of radar data: progress and challenges. Q. J. Royal Meteorol. Soc. 131(613), 3439–3463 (2005)

    Article  Google Scholar 

  51. Tapia, A., Smith, J.A., Dixon, M.: Estimation of convective rainfall from lightning observations. J. Appl. Meteorol. 37, 1497–1509 (1998)

    Article  Google Scholar 

  52. Tukey, J.W.: Exploratory data analysis. Addison-Wesley, Boston (1977)

    MATH  Google Scholar 

  53. Tuomi, T.J., Larjavaara, M.: Identification and analysis of flash cells in thunderstorms. Q. J. Royal Meteorol. Soc. 131(607), 1191–1214 (2005)

    Article  Google Scholar 

  54. Vendrasco, E.P., Sun, J., Herdies, D.L., de Angelis, C.F.: Constraining a 3DVAR radar data assimilation system with large-scale analysis to improve short-range precipitation forecasts. J. Appl. Meteorol. Climatol. 55(3), 673–690 (2016)

    Article  Google Scholar 

  55. Vila, D.A., Machado, L.A.T., Laurent, H., Velasco, I.: Forecast and tracking the evolution of cloud clusters (ForTraCC) using satellite infrared imagery: methodology and validation. Weather Forecast. 23(2), 233–245 (2008)

    Article  Google Scholar 

  56. Wang, Y., Yang, Y., Wang, C.: Improving forecasting of strong convection by assimilating cloud-to-ground lightning data using the physical initialization method. Atmos. Res. 150, 31–41 (2014)

    Article  Google Scholar 

  57. Wang, C., Zheng, D., Zhang, Y., Liu, L.: Relationship between lightning activity and vertical airflow characteristics in thunderstorms. Atmos. Res. 191, 12–19 (2017)

    Article  Google Scholar 

Download references

Acknowledgements

Author Stephan Stephany thanks CNPq (Conselho Nacional de Desenvolvimento Científico e Tecnológico) for grant 307460/2015-0.

Author information

Authors and Affiliations

  1. National Institute for Space Research (INPE), São José dos Campos, Brazil

    Stephan Stephany, Cesar Strauss & Alan James Peixoto Calheiros

  2. National Center for Monitoring and Early Warning of Natural Disasters (CEMADEN), São José dos Campos, Brazil

    Glauston Roberto Teixeira de Lima & João Victor Cal Garcia

  3. Climatempo Meteorologia, São José dos Campos, Brazil

    Alex Sandro Aguiar Pessoa

Authors
  1. Stephan Stephany
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cesar Strauss
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alan James Peixoto Calheiros
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Glauston Roberto Teixeira de Lima
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. João Victor Cal Garcia
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alex Sandro Aguiar Pessoa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stephan Stephany .

Editor information

Editors and Affiliations

  1. National Center for Monitoring and Early Warning of Natural Disasters (CEMADEN), São José dos Campos, São Paulo, Brazil

    Leonardo Bacelar Lima Santos

  2. Institute of Science and Technology, São Paulo State University (UNESP), São José dos Campos, São Paulo, Brazil

    Rogério Galante Negri

  3. Department of Informatics, Federal Institute of São Paulo (IFSP), Campinas, São Paulo, Brazil

    Tiago José de Carvalho

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Stephany, S., Strauss, C., Calheiros, A.J.P., de Lima, G.R.T., Garcia, J.V.C., Pessoa, A.S.A. (2019). Data Mining Approaches to the Real-Time Monitoring and Early Warning of Convective Weather Using Lightning Data. In: Bacelar Lima Santos, L., Galante Negri, R., de Carvalho, T. (eds) Towards Mathematics, Computers and Environment: A Disasters Perspective. Springer, Cham. https://doi.org/10.1007/978-3-030-21205-6_5

Download citation

Publish with us

Policies and ethics

Search

Navigation