Abstract
Experiments in high-energy astroparticle physics produce large amounts of data as continuous high-volume streams. Gaining insights from the observed data poses a number of challenges to data analysis at various steps in the analysis chain of the experiments. Machine learning methods have already cleaved their way selectively at some particular stages of the overall data mangling process.
In this paper we investigate the deployment of machine learning methods at various stages of the data analysis chain in a gamma-ray astronomy experiment. Aiming at online and real-time performance, we build up on prominent software libraries and discuss the complete cycle of data processing from raw-data capturing to high-level classification using a data-flow based rapid-prototyping environment. In the context of a gamma-ray experiment, we review user requirements in this interdisciplinary setting and demonstrate the applicability of our approach in a real-world setting to provide results from high-volume data streams in real-time performance.
Chapter PDF
Similar content being viewed by others
References
Abeysekara, A.U., et al.: On the sensitivity of the HAWC observatory to gamma-ray bursts. Astroparticle Physics 35, 641–650 (2012)
Anderhub, H., et al.: Fact - the first cherenkov telescope using a g-apd camera for tev gamma-ray astronomy. Nuclear Instruments and Methods in Physics Research A 639, 58–61 (2011)
Artikis, A., Weidlich, M., Schnitzler, F., et al.: Heterogeneous stream processing and crowdsourcing for urban traffic management. In: Proceedings of the 17th International Conference on Extending Database Technology (2014)
Atkins, R., et al.: Milagrito, a tev air-shower array. Nuclear Instruments and Methods in Physics Research 449, 478–499 (2000)
Badran, H.M., Weekes, T.C.: Improvement of gamma-hadron discrimination at tev energies using a new parameter, image surface brightness. Astroparticle Physics 7(4), 307–314 (1997)
Berger, K., Bretz, T., Dorner, D., Hoehne, D., Riegel, B.: A robust way of estimating the energy of a gamma ray shower detected by the magic telescope. In: Proceedings of the 29th International Cosmic Ray Conference, pp. 100–104 (2005)
Bifet, A., Holmes, G., Kirkby, R., Pfahringer, B.: Moa: Massive online analysis. J. Mach. Learn. Res. 11, 1601–1604 (2010)
Bock, R.K., Chilingarian, A., et al.: Methods for multidimensional event classification: a case study using images from a cherenkov gamma-ray telescope. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 516(2–3), 511–528 (2004)
Bockermann, C., Blom, H.: The streams framework. Technical Report 5, TU Dortmund University, vol. 12 (2012)
Breiman, L.: Random forests. Machine Learning 45(1), 5–32 (2001)
Bussino, S., Mari, S.M.: Gamma-hadron discrimination in extensive air showers using a neural network. Astroparticle Physics 15(1), 65–77 (2001)
Carroll, B.W., Ostlie, D.A.: An Introduction to Modern Astrophysics, 2nd edn. Pearson Addison-Wesley, San Francisco (2007)
De Naurois, M.: Analysis methods for atmospheric cerenkov telescopes (2006). arXiv preprint astro-ph/0607247
Faleiro, E., Muñoz, L., Relaño, A., Retamosa, J.: Discriminant analysis based on spectral statistics applied to TeV cosmic \(\gamma \)/proton separation. Astroparticle Physics 35, 785–791 (2012)
Gillessen, S., Harney, H.L.: Significance in gamma-ray astronomy - the li and ma problem in bayesian statistics. Astronomy and Astrophysics 430(1), 355–362 (2004)
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The weka data mining software: An update. SIGKDD Explor. Newsl. 11(1), 10–18 (2009)
Heck, D., Knapp, J., Capdevielle, J.N., Schatz, G., Thouw, T.: CORSIKA: A Monte Carlo Code to Simulate Extensive Air Showers. Forschungszentrum Karlsruhe GmbH, Karlsruhe (1998)
Kieda, D.B.: VERITAS Collab. Status of the VERITAS ground based GeV/TeV Gamma-Ray Observatory. In: High Energy Astrophysics Division, vol. 36, p. 910. Bulletin of the American Astronomical Society, August 2004
Hillas, A.M.: Cerenkov light images of EAS produced by primary gamma rays and by nuclei. In: Jones, F.C. (ed.) Proceedings of the 19th International Cosmic Ray Conference, vol. 3, pp. 445–448, La Jolla, August 1985
Mierswa, I., Wurst, M., Klinkenberg, R., Scholz, M., Euler, T.: Yale: Rapid prototyping for complex data mining tasks. In: Ungar, L., Craven, M., Gunopulos, D., Eliassi-Rad, T. (eds.) Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2006, pp. 935–940. ACM, New York, August 2006
Petry, D., et al.: The MAGIC Telescope - prospects for GRB research. Astronomy & Astrophysics Supplement Series 138, 601–602 (1999)
Pivato, G., et al.: Fermi LAT and WMAP Observations of the Supernova Remnant HB 21. The Astrophysical Journal 779, 179 (2013)
Schäfer, B.M., Hofmann, W., Lampeitl, H., Hemberger, M.: Particle identification by multifractal parameters in \(\gamma \)-astronomy with the hegra-cherenkov-telescopes. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 465(2–3), 394–403 (2001)
Schlüter, J., Odenthal, H.-J., Uebber, N., Blom, H., Beckers, T., Morik, K.: Reliable bof endpoint prediction by novel data-driven modeling. In: AISTech Conference Proceedings. AISTech (2014)
Schnitzler, F., Artikis, A., Weidlich, M., Boutsis, I., Liebig, T., Piatkowski, N., Bockermann, C., Morik, K., Kalogeraki, V., Marecek, J., Gal, A., Mannor, S., Kinane, D., Gunopulos, D.: Heterogeneous stream processing and crowdsourcing for traffic monitoring: highlights. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds.) ECML PKDD 2014, Part III. LNCS, vol. 8726, pp. 520–523. Springer, Heidelberg (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Bockermann, C. et al. (2015). Online Analysis of High-Volume Data Streams in Astroparticle Physics. In: Bifet, A., et al. Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2015. Lecture Notes in Computer Science(), vol 9286. Springer, Cham. https://doi.org/10.1007/978-3-319-23461-8_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-23461-8_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-23460-1
Online ISBN: 978-3-319-23461-8
eBook Packages: Computer ScienceComputer Science (R0)