Deep Learning in the Natural Sciences: Applications to Physics

  • Peter Sadowski
  • Pierre BaldiEmail author
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11100)


Machine learning is increasingly being used not only in engineering applications such as computer vision and speech recognition, but in data analysis for the natural sciences. Here we describe applications of deep learning to four areas of experimental sub-atomic physics — high-energy physics, antimatter physics, neutrino physics, and dark matter physics.


  1. 1.
    Abdesselam, A.: Boosted objects: a probe of beyond the standard model physics. Eur. Phys. J. C 71, 1661 (2011)CrossRefGoogle Scholar
  2. 2.
    Adams, D., Arce, A., Asquith, L., Backovic, M., Barillari, T., et al.: Towards an understanding of the correlations in jet substructure. Eur. Phys. J. C 75, 409 (2015)Google Scholar
  3. 3.
    Aghion, S.: A moiré deflectometer for antimatter. Nat. Commun. 5, 4538 (2014)CrossRefGoogle Scholar
  4. 4.
    Altheimer, A.: Jet substructure at the Tevatron and LHC: new results, new tools, new benchmarks. J. Phys. G39, 063001 (2012)CrossRefGoogle Scholar
  5. 5.
    Altheimer, A.: Boosted objects and jet substructure at the LHC. Report of BOOST2012, held at IFIC Valencia, 23rd–27th of July 2012. Eur. Phys. J. C74(3), 2792 (2014)CrossRefGoogle Scholar
  6. 6.
    Alwall, J.: MadGraph 5: going beyond. JHEP 1106, 128 (2011)CrossRefGoogle Scholar
  7. 7.
    Amole, C.: Description and first application of a new technique to measure the gravitational mass of antihydrogen. Nat. Commun. 4, 1785 (2013)CrossRefGoogle Scholar
  8. 8.
    Amole, C., et al.: The alpha antihydrogen trapping apparatus. Nucl. Instr. Meth. A 735, 319–340 (2014)CrossRefGoogle Scholar
  9. 9.
    Amoretti, M., et al.: The athena antihydrogen apparatus. Nucl. Instr. Meth. A 518, 679–711 (2004)CrossRefGoogle Scholar
  10. 10.
    Andresen, G.B., et al.: Confinement of antihydrogen for 1,000 seconds. Nat. Phys. 7, 558–564 (2011)CrossRefGoogle Scholar
  11. 11.
    Andresen, G., et al.: Trapped antihydrogen. Nature 468(7324), 673–676 (2010)CrossRefGoogle Scholar
  12. 12.
    ATLAS Collaboration: ATLAS experiment at the CERN Large Hadron Collider. JINST 3, S08003 (2008)Google Scholar
  13. 13.
    ATLAS Collaboration: Luminosity determination in PP collisions at \(\sqrt{s}=7\) TeV using the ATLAS detector at the LHC. Eur. Phys.J C73, 2518 (2013)Google Scholar
  14. 14.
    Aurisano, A., et al.: A convolutional neural network neutrino event classifier. J. Instrum. 11(09), P09001 (2016). Scholar
  15. 15.
    Bahr, M.: Herwig++ physics and manual. Eur. Phys. J. C 58, 639–707 (2008)CrossRefGoogle Scholar
  16. 16.
    Baldi, P.: Deep learning in biomedical data science. Ann. Rev. Biomed. Data Sci. 1, 181 (2018)CrossRefGoogle Scholar
  17. 17.
    Baldi, P.: The inner and outer approaches for the design of recursive neural networks architectures. Data Mining Knowl. Disc. 32, 218–230 (2017). Scholar
  18. 18.
    Baldi, P., Brunak, S.: Bioinformatics: The Machine Learning Approach, second edition edn. MIT Press, Cambridge (2001)zbMATHGoogle Scholar
  19. 19.
    Baldi, P., Brunak, S., Frasconi, P., Pollastri, G., Soda, G.: Exploiting the past and the future in protein secondary structure prediction. Bioinformatics 15, 937–946 (1999)CrossRefGoogle Scholar
  20. 20.
    Baldi, P., Chauvin, Y.: Neural networks for fingerprint recognition. Neural Comput. 5(3), 402–418 (1993)CrossRefGoogle Scholar
  21. 21.
    Baldi, P., Pollastri, G.: The principled design of large-scale recursive neural network architectures-DAG-RNNs and the protein structure prediction problem. J. Mach. Learn. Res. 4, 575–602 (2003)zbMATHGoogle Scholar
  22. 22.
    Baldi, P., Sadowski, P.: The dropout learning algorithm. Artif. Intell. 210C, 78–122 (2014)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Baldi, P., Sadowski, P., Whiteson, D.: Searching for exotic particles in high-energy physics with deep learning. Nat. Commun. 5, Article no. 4308 (2014)Google Scholar
  24. 24.
    Baldi, P., Sadowski, P., Whiteson, D.: Enhanced higgs boson to \(\tau \)\(\tau \) search with deep learning. Phys. Rev. Lett. 114, 111801 (2015)CrossRefGoogle Scholar
  25. 25.
    Baldi, P., Bauer, K., Eng, C., Sadowski, P., Whiteson, D.: Jet substructure classification in high-energy physics with deep neural networks. Phys. Rev. D 93, 094034 (2016). Scholar
  26. 26.
    Baldi, P., Cranmer, K., Faucett, T., Sadowski, P., Whiteson, D.: Parameterized neural networks for high-energy physics. Eur. Phys. J. C 76(5), 235 (2016). Scholar
  27. 27.
    Beringer, J.: Review of particle physics. Phys. Rev. D 86, 010001 (2012)CrossRefGoogle Scholar
  28. 28.
    Cazaux, S., Lerch, T., Aune, S.: Detecteur courbe de particules gazeux, patent App. EP20,130,188,550, April.
  29. 29.
    Cheng, J., Randall, A.Z., Sweredoski, M., Baldi, P.: Scratch: a protein structure and structural feature prediction server. Nucleic Acids Res. 33, W72–W76 (2005)CrossRefGoogle Scholar
  30. 30.
    Chollet, F.: Keras. GitHub (2015).
  31. 31.
    CMS Collaboration: Search for light vector resonances decaying to quarks at 13 TeV. CMS-PAS-EXO-16-030 (2016)Google Scholar
  32. 32.
    Corradini, M., et al.: Experimental apparatus for annihilation cross-section measurements of low energy antiprotons. Nucl. Instr. Meth. A 711, 12–20 (2013)CrossRefGoogle Scholar
  33. 33.
    Corradini, M.: Scintillating bar detector for antiproton annihilations measurements. Hyperfine Interact. 233, 53–58 (2015)CrossRefGoogle Scholar
  34. 34.
    Cranmer, K., Pavez, J., Louppe, G.: Approximating likelihood ratios with calibrated discriminative classifiers (2015)Google Scholar
  35. 35.
    Krohn, D., Thaler, J., Wang, L.T.: Jet trimming. JHEP 1002, 084 (2010)Google Scholar
  36. 36.
    Dasgupta, M., Fregoso, A., Marzani, S., Powling, A.: Jet substructure with analytical methods. Eur. Phys. J. C 73(11), 2623 (2013)CrossRefGoogle Scholar
  37. 37.
    Dasgupta, M., Powling, A., Siodmok, A.: On jet substructure methods for signal jets. JHEP 08, 079 (2015)CrossRefGoogle Scholar
  38. 38.
    Dolen, J., Harris, P., Marzani, S., Rappoccio, S., Tran, N.: Thinking outside the ROCs: designing decorrelated taggers (DDT) for jet substructure. JHEP 05, 156 (2016)CrossRefGoogle Scholar
  39. 39.
    Duvenaud, D., et al.: Convolutional networks on graphs for learning molecular fingerprints. In: Neural Information Processing Systems (2015)Google Scholar
  40. 40.
    Edwards, H., Storkey, A.J.: Censoring representations with an adversary (2016).
  41. 41.
    Feng, J.L.: Dark matter candidates from particle physics and methods of detection. Ann. Rev. Astron. Astrophys. 48, 495–545 (2010)CrossRefGoogle Scholar
  42. 42.
    Gabrielse, G., et al.: Trapped antihydrogen in its ground state. Phys. Rev. Lett. 108, 113002 (2012). Scholar
  43. 43.
    Ganin, Y., et al.: Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17(1), 2096–2030 (2016).
  44. 44.
    Giomataris, Y., Rebourgeard, P., Robert, J.P., Charpak, G.: Micromegas: a high-granularity position-sensitive gaseous detector for high particle-flux environments. Nucl. Instr. Meth. A 376, 29 (1996)CrossRefGoogle Scholar
  45. 45.
    Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016).
  46. 46.
    Guest, D., Collado, J., Baldi, P., Hsu, S.C., Urban, G., Whiteson, D.: Jet flavor classification in high-energy physics with deep neural networks. Phys. Rev. D 94, 112002 (2016). Scholar
  47. 47.
    He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: The IEEE International Conference on Computer Vision (ICCV), December 2015Google Scholar
  48. 48.
    Hertel, L., Li, L., Baldi, P., Bian, J.: Convolutional neural networks for electron neutrino and electron shower energy reconstruction in the nova detectors. In: Deep Learning for Physical Sciences Workshop at Neural Information Processing Systems (2017)Google Scholar
  49. 49.
    Hori, M., Yamashita, K., Hayano, R., Yamazaki, T.: Analog cherenkov detectors used in laser spectroscopy experiments on antiprotonic helium. Nucl. Instr. Meth. A 496, 102–122 (2003)CrossRefGoogle Scholar
  50. 50.
    Kaplan, D.E., Rehermann, K., Schwartz, M.D., Tweedie, B.: Top tagging: a method for identifying boosted hadronically decaying top quarks. Phys. Rev. Lett. 101, 142001 (2008)CrossRefGoogle Scholar
  51. 51.
    Kayala, M., Azencott, C., Chen, J., Baldi, P.: Learning to predict chemical reactions. J. Chem. Inf. Model. 51(9), 2209–2222 (2011)CrossRefGoogle Scholar
  52. 52.
    Kayala, M., Baldi, P.: Reactionpredictor: prediction of complex chemical reactions at the mechanistic level using machine learning. J. Chem. Inf. Model. 52(10), 2526–2540 (2012)CrossRefGoogle Scholar
  53. 53.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations (ICLR) (2014)Google Scholar
  54. 54.
    Kuroda, N.: A source of antihydrogen for in-flight hyperfine spectroscopy. Nat. Commun. 5, 3089 (2014)CrossRefGoogle Scholar
  55. 55.
    Larkoski, A.J., Marzani, S., Soyez, G., Thaler, J.: Soft Drop. JHEP 1405, 146 (2014)CrossRefGoogle Scholar
  56. 56.
    Larkoski, A.J., Moult, I., Neill, D.: Power counting to better jet observables. JHEP 12, 009 (2014)CrossRefGoogle Scholar
  57. 57.
    Larkoski, A.J., Salam, G.P., Thaler, J.: Energy correlation functions for jet substructure. JHEP 1306, 108 (2013)MathSciNetCrossRefGoogle Scholar
  58. 58.
    Louppe, G., Cho, K., Becot, C., Cranmer, K.: QCD-aware recursive neural networks for jet physics (2017)Google Scholar
  59. 59.
    Louppe, G., Kagan, M., Cranmer, K.: Learning to pivot with adversarial networks (2016)Google Scholar
  60. 60.
    Lusci, A., Pollastri, G., Baldi, P.: Deep architectures and deep learning in chemoinformatics: the prediction of aqueous solubility for drug-like molecules. J. Chem. Inf. Model. 53(7), 1563–1575 (2013)CrossRefGoogle Scholar
  61. 61.
    Dasgupta, M., Fregoso, A., Marzani, S., Salam, G.P.: Towards an understanding of jet substructure. JHEP 9, 029 (2013)Google Scholar
  62. 62.
    Magnan, C.N., Baldi, P.: SSpro/ACCpro 5: almost perfect prediction of protein secondary structure and relative solvent accessibility using profiles, machine learning, and structural similarity. Bioinformatics 30(18), 2592–2597 (2014)CrossRefGoogle Scholar
  63. 63.
    Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Furnkranz, J., Joachims, T. (eds.) Proceedings of the 27th International Conference on Machine Learning (ICML 2010), pp. 807–814. Omnipress (2010).
  64. 64.
    Nix, D.A., Weigend, A.S.: Estimating the mean and variance of the target probability distribution. In: 1994 IEEE International Conference on Neural Networks, IEEE World Congress on Computational Intelligence, vol. 1, pp. 55–60, June 1994Google Scholar
  65. 65.
    de Oliveira, L., Kagan, M., Mackey, L., Nachman, B., Schwartzman, A.: Jet-images – deep learning edition. J. High Energy Phys. 2016(7), 69 (2016). Scholar
  66. 66.
    Ovyn, S., Rouby, X., Lemaitre, V.: DELPHES, a framework for fast simulation of a generic collider experiment (2009)Google Scholar
  67. 67.
    Planck Collaboration: Planck 2013 results. XVI, Cosmological parameters (2013)Google Scholar
  68. 68.
    Plehn, T., Spannowsky, M., Takeuchi, M., Zerwas, D.: Stop reconstruction with tagged tops. JHEP 1010, 078 (2010)CrossRefGoogle Scholar
  69. 69.
    Pollastri, G., Przybylski, D., Rost, B., Baldi, P.: Improving the prediction of protein secondary strucure in three and eight classes using recurrent neural networks and profiles. Proteins 47, 228–235 (2001)CrossRefGoogle Scholar
  70. 70.
    Pérez, P.: The GBAR antimatter gravity experiment. Hyperfine Interact. 233, 21–27 (2015)CrossRefGoogle Scholar
  71. 71.
    Racah, E., et al.: Revealing fundamental physics from the daya bay neutrino experiment using deep neural networks. In: 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 892–897, December 2016Google Scholar
  72. 72.
    Radics, B., Murtagh, D.J., Yamazaki, Y., Robicheaux, F.: Scaling behavior of the ground-state antihydrogen yield as a function of positron density and temperature from classical-trajectory Monte Carlo simulations. Phys. Rev. A 90(3), 032704 (2014). Scholar
  73. 73.
    Radics, B., et al.: The ASACUSA micromegas tracker: a cylindrical, bulk micromegas detector for antimatter research. Rev. Sci. Instrum. 86, 083304 (2015)CrossRefGoogle Scholar
  74. 74.
    Ellis, S.D., Vermilion, C.K., Walsh, J.R.: Recombination algorithms and jet substructure: pruning as a tool for heavy particle searches. Phys. Rev. D81, 094023 (2010)Google Scholar
  75. 75.
    Sadowski, P., Collado, J., Whiteson, D., Baldi, P.: Deep learning, dark knowledge, and dark matter. J. Mach. Learn. Res. 42, 81–97 (2015). Workshop and Conference ProceedingsGoogle Scholar
  76. 76.
    Sadowski, P., Radics, B., Ananya, Yamazaki, Y., Baldi, P.: Efficient antihydrogen detection in antimatter physics by deep learning. J. Phys. Commun. 1(2), 025001 (2017). Scholar
  77. 77.
    Schmidhuber, J.: Learning factorial codes by predictability minimization. Neural Comput. 4, 863–879 (1991)CrossRefGoogle Scholar
  78. 78.
    Shimmin, C., et al.: Decorrelated jet substructure tagging using adversarial neural networks. Phys. Rev. D 96, 074034 (2017). arXiv: 1703.03507
  79. 79.
    Shimmin, C., Whiteson, D.: Boosting low-mass hadronic resonances. Phys. Rev. D 94, 055001 (2016). Scholar
  80. 80.
    Sjostrand, T., et al.: PYTHIA 6.4 physics and manual. JHEP 05, 026 (2006)CrossRefGoogle Scholar
  81. 81.
    Snoek, J., Larochelle, H., Adams, R.P.: Practical Bayesian optimization of machine learning algorithms. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 25, pp. 2951–2959. Curran Associates, Inc. (2012)Google Scholar
  82. 82.
    Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014).
  83. 83.
    Storey, J.: Particle tracking at 4k: the fast annihilation cryogenic tracking (fact) detector for the aegis antimatter gravity experiment. Nucl. Instr. Meth. A 732, 437–441 (2013)CrossRefGoogle Scholar
  84. 84.
    Strandlie, A., Frühwirth, R.: Track and vertex reconstruction: from classical to adaptive methods. Rev. Mod. Phys. 82, 1419 (2010)CrossRefGoogle Scholar
  85. 85.
    Szegedy, C., et al.: Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9, June 2015Google Scholar
  86. 86.
    Thaler, J., Van Tilburg, K.: Identifying boosted objects with N-subjettiness. JHEP 1103, 015 (2011)CrossRefGoogle Scholar
  87. 87.
    Thaler, J., Van Tilburg, K.: Maximizing boosted top identification by minimizing n-subjettiness. JHEP 02, 093 (2012)CrossRefGoogle Scholar
  88. 88.
    Theano Development Team: Theano: a Python framework for fast computation of mathematical expressions. arXiv e-prints, May 2016.
  89. 89.
    Tishby, N., Zaslavsky, N.: Deep learning and the information bottleneck principle. In: 2015 IEEE Information Theory Workshop (ITW), pp. 1–5, April 2015Google Scholar
  90. 90.
    XENON Collaboration: First dark matter search results from the XENON1T experiment. Phys. Rev. Lett. 119(18), 181301 (2017)Google Scholar
  91. 91.
    XENON Collaboration: The XENON1T dark matter experiment. Eur. Phys. J. C 77(12), 881, December 2017.
  92. 92.
    Zhang, Z., Oelert, W., Grzonka, D., Sefzick, T.: The antiproton annihilation detector system of the atrap experiment. Chin. Sci. Bull. 54, 189–195 (2009)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of California, IrvineIrvineUSA

Personalised recommendations