Adversarially-trained autoencoders for robust unsupervised new physics searches

  • Andrew Blance
  • Michael Spannowsky
  • Philip WaiteEmail author
Open Access
Regular Article - Experimental Physics


Machine learning techniques in particle physics are most powerful when they are trained directly on data, to avoid sensitivity to theoretical uncertainties or an underlying bias on the expected signal. To be able to train on data in searches for new physics, anomaly detection methods are imperative, which can be realised by an autoencoder acting as an unsupervised classifier. The last source of uncertainties affecting the classifier are then experimental uncertainties in the reconstruction of the final-state objects. To mitigate their effect on the classifier and to allow for a realistic assessment of the method, we propose to combine the autoencoder with an adversarial neural network to remove its sensitivity to the smearing of the final-state objects. We quantify its effect and show that one can achieve a robust anomaly detection in resonance-induced \( t\overline{t} \) final states.


Beyond Standard Model Particle correlations and fluctuations Jet physics Top physics Hadron-Hadron scattering (experiments) 


Open Access

This article is distributed under the terms of the Creative Commons Attribution License (CC-BY 4.0), which permits any use, distribution and reproduction in any medium, provided the original author(s) and source are credited


  1. [1]
    B. Nachman et al., Jets from jets: re-clustering as a tool for large radius jet reconstruction and grooming at the LHC, JHEP02 (2015) 075 [arXiv:1407.2922] [INSPIRE].ADSCrossRefGoogle Scholar
  2. [2]
    P.T. Komiske, E.M. Metodiev and M.D. Schwartz, Deep learning in color: towards automated quark/gluon jet discrimination, JHEP01 (2017) 110 [arXiv:1612.01551] [INSPIRE].ADSzbMATHCrossRefGoogle Scholar
  3. [3]
    J. Barnard, E.N. Dawe, M.J. Dolan and N. Rajcic, Parton shower uncertainties in jet substructure analyses with deep neural networks, Phys. Rev.D 95 (2017) 014018 [arXiv:1609.00607] [INSPIRE].ADSGoogle Scholar
  4. [4]
    L.M. Dery, B. Nachman, F. Rubbo and A. Schwartzman, Weakly supervised classification in high energy physics, JHEP05 (2017) 145 [arXiv:1702.00414] [INSPIRE].ADSzbMATHCrossRefGoogle Scholar
  5. [5]
    A. Butter, G. Kasieczka, T. Plehn and M. Russell, Deep-learned top tagging with a Lorentz layer, SciPost Phys.5 (2018) 028 [arXiv:1707.08966] [INSPIRE].ADSCrossRefGoogle Scholar
  6. [6]
    T. Cohen, M. Freytsis and B. Ostdiek, (Machine) learning to do more with less, JHEP02 (2018) 034 [arXiv:1706.09451] [INSPIRE].ADSCrossRefGoogle Scholar
  7. [7]
    S. Chang, T. Cohen and B. Ostdiek, What is the machine learning?, Phys. Rev.D 97 (2018) 056009 [arXiv:1709.10106] [INSPIRE].ADSGoogle Scholar
  8. [8]
    J. Pearkes, W. Fedorko, A. Lister and C. Gay, Jet constituents for deep neural network based top quark tagging, arXiv:1704.02124 [INSPIRE].
  9. [9]
    G. Louppe, K. Cho, C. Becot and K. Cranmer, QCD-aware recursive neural networks for jet physics, JHEP01 (2019) 057 [arXiv:1702.00748] [INSPIRE].ADSCrossRefGoogle Scholar
  10. [10]
    G. Kasieczka, T. Plehn, M. Russell and T. Schell, Deep-learning top taggers or the end of QCD?, JHEP05 (2017) 006 [arXiv:1701.08784] [INSPIRE].ADSCrossRefGoogle Scholar
  11. [11]
    L. de Oliveira, M. Paganini and B. Nachman, Learning particle physics by example: location-aware generative adversarial networks for physics synthesis, Comput. Softw. Big Sci.1 (2017) 4 [arXiv:1701.05927] [INSPIRE].CrossRefGoogle Scholar
  12. [12]
    H. Lüo, M.-x. Luo, K. Wang, T. Xu and G. Zhu, Quark jet versus gluon jet: fully-connected neural networks with high-level features, Sci. China Phys. Mech. Astron.62 (2019) 991011 [arXiv:1712.03634] [INSPIRE].CrossRefGoogle Scholar
  13. [13]
    K. Datta and A.J. Larkoski, Novel jet observables from machine learning, JHEP03 (2018) 086 [arXiv:1710.01305] [INSPIRE].ADSCrossRefGoogle Scholar
  14. [14]
    A.J. Larkoski, I. Moult and B. Nachman, Jet substructure at the large hadron collider: a review of recent advances in theory and machine learning, arXiv:1709.04464 [INSPIRE].
  15. [15]
    C. Shimmin et al., Decorrelated jet substructure tagging using adversarial neural networks, Phys. Rev.D 96 (2017) 074034 [arXiv:1703.03507] [INSPIRE].ADSGoogle Scholar
  16. [16]
    E.M. Metodiev, B. Nachman and J. Thaler, Classification without labels: Learning from mixed samples in high energy physics, JHEP10 (2017) 174 [arXiv:1708.02949] [INSPIRE].ADSCrossRefGoogle Scholar
  17. [17]
    T. Roxlo and M. Reece, Opening the black box of neural nets: case studies in stop/top discrimination, arXiv:1804.09278 [INSPIRE].
  18. [18]
    J. Brehmer, K. Cranmer, G. Louppe and J. Pavez, Constraining effective field theories with machine learning, Phys. Rev. Lett.121 (2018) 111801 [arXiv:1805.00013] [INSPIRE].ADSCrossRefGoogle Scholar
  19. [19]
    J. Brehmer, K. Cranmer, G. Louppe and J. Pavez, A guide to constraining effective field theories with machine learning, Phys. Rev.D 98 (2018) 052004 [arXiv:1805.00020] [INSPIRE].ADSGoogle Scholar
  20. [20]
    J.H. Collins, K. Howe and B. Nachman, Anomaly detection for resonant new physics with machine learning, Phys. Rev. Lett.121 (2018) 241803 [arXiv:1805.02664] [INSPIRE].ADSCrossRefGoogle Scholar
  21. [21]
    J. Duarte et al., Fast inference of deep neural networks in FPGAs for particle physics, 2018 JINST13 P07027 [arXiv:1804.06913] [INSPIRE].CrossRefGoogle Scholar
  22. [22]
    K. Fraser and M.D. Schwartz, Jet charge and machine learning, JHEP10 (2018) 093 [arXiv:1803.08066] [INSPIRE].ADSCrossRefGoogle Scholar
  23. [23]
    P.T. Komiske, E.M. Metodiev, B. Nachman and M.D. Schwartz, Learning to classify from impure samples with high-dimensional data, Phys. Rev.D 98 (2018) 011502 [arXiv:1801.10158] [INSPIRE].ADSGoogle Scholar
  24. [24]
    S. Macaluso and D. Shih, Pulling out all the tops with computer vision and deep learning, JHEP10 (2018) 121 [arXiv:1803.00107] [INSPIRE].ADSCrossRefGoogle Scholar
  25. [25]
    A. Andreassen, I. Feige, C. Frye and M.D. Schwartz, JUNIPR: a framework for unsupervised machine learning in particle physics, Eur. Phys. J.C 79 (2019) 102 [arXiv:1804.09720] [INSPIRE].ADSCrossRefGoogle Scholar
  26. [26]
    P. De Castro and T. Dorigo, INFERNO: inference-aware neural optimisation, Comput. Phys. Commun.244 (2019) 170 [arXiv:1806.04743] [INSPIRE].ADSCrossRefGoogle Scholar
  27. [27]
    R.T. D’Agnolo and A. Wulzer, Learning new physics from a machine, Phys. Rev.D 99 (2019) 015014 [arXiv:1806.02350] [INSPIRE].ADSGoogle Scholar
  28. [28]
    J. Brehmer, G. Louppe, J. Pavez and K. Cranmer, Mining gold from implicit models to improve likelihood-free inference, arXiv:1805.12244 [INSPIRE].
  29. [29]
    J.W. Monk, Deep learning as a parton shower, JHEP12 (2018) 021 [arXiv:1807.03685] [INSPIRE].ADSCrossRefGoogle Scholar
  30. [30]
    L. Moore, K. Nordström, S. Varma and M. Fairbairn, Reports of my demise are greatly exaggerated: N -subjettiness taggers take on jet images, arXiv:1807.04769 [INSPIRE].
  31. [31]
    A. De Simone and T. Jacques, Guiding new physics searches with unsupervised learning, Eur. Phys. J.C 79 (2019) 289 [arXiv:1807.06038] [INSPIRE].ADSCrossRefGoogle Scholar
  32. [32]
    S. Bollweg et al., Deep-learning jets with uncertainties and more, arXiv:1904.10004 [INSPIRE].
  33. [33]
    O. Cerri et al., Variational autoencoders for new physics mining at the Large Hadron Collider, JHEP05 (2019) 036 [arXiv:1811.10276] [INSPIRE].ADSCrossRefGoogle Scholar
  34. [34]
    ATLAS collaboration, Generalized numerical inversion: a neural network approach to jet calibration, ATL-PHYS-PUB-2018-013 (2018).Google Scholar
  35. [35]
    ATLAS collaboration, Performance of the ATLAS track reconstruction algorithms in dense environments in LHC Run 2, Eur. Phys. J.C 77 (2017) 673 [arXiv:1704.07983] [INSPIRE].
  36. [36]
    CMS collaboration, Performance of the CMS missing transverse momentum reconstruction in pp data at \( \sqrt{s} \) = 8 TeV, 2015 JINST10 P02006 [arXiv:1411.0511] [INSPIRE].
  37. [37]
    CMS collaboration, Performance of electron reconstruction and selection with the CMS detector in proton-proton collisions at \( \sqrt{s} \) = 8 TeV, 2015 JINST10 P06005 [arXiv:1502.02701] [INSPIRE].
  38. [38]
    CMS collaboration, Performance of photon reconstruction and identification with the cms detector in proton-proton collisions at \( \sqrt{s} \) = 8 TeV, 2015 JINST10 P08010 [arXiv:1502.02702] [INSPIRE].
  39. [39]
    T. Gleisberg et al., Event generation with SHERPA 1.1, JHEP02 (2009) 007 [arXiv:0811.4622] [INSPIRE].ADSCrossRefGoogle Scholar
  40. [40]
    J. Bellm et al., HERWIG 7.0/HERWIG++ 3.0 release note, Eur. Phys. J.C 76 (2016) 196 [arXiv:1512.01178] [INSPIRE].ADSCrossRefGoogle Scholar
  41. [41]
    T. Sjöstrand et al., An introduction to PYTHIA 8.2, Comput. Phys. Commun.191 (2015) 159 [arXiv:1410.3012] [INSPIRE].ADSzbMATHCrossRefGoogle Scholar
  42. [42]
    C. Englert, R. Kogler, H. Schulz and M. Spannowsky, Higgs characterisation in the presence of theoretical uncertainties and invisible decays, Eur. Phys. J.C 77 (2017) 789 [arXiv:1708.06355] [INSPIRE].ADSCrossRefGoogle Scholar
  43. [43]
    C. Englert, P. Galler, A. Pilkington and M. Spannowsky, Approaching robust EFT limits for CP-violation in the Higgs sector, Phys. Rev.D 99 (2019) 095007 [arXiv:1901.05982] [INSPIRE].ADSGoogle Scholar
  44. [44]
    S. Schaetzel and M. Spannowsky, Tagging highly boosted top quarks, Phys. Rev.D 89 (2014) 014007 [arXiv:1308.0540] [INSPIRE].ADSGoogle Scholar
  45. [45]
    ATLAS collaboration, Light-quark and gluon jet discrimination in pp collisions at \( \sqrt{s} \) = 7 TeV with the ATLAS detector, Eur. Phys. J.C 74 (2014) 3023 [arXiv:1405.6583] [INSPIRE].
  46. [46]
    C. Englert, P. Galler, P. Harris and M. Spannowsky, Machine learning uncertainties with adversarial neural networks, Eur. Phys. J.C 79 (2019) 4 [arXiv:1807.08763] [INSPIRE].ADSCrossRefGoogle Scholar
  47. [47]
    G. Louppe, M. Kagan and K. Cranmer, Learning to pivot with adversarial networks, arXiv:1611.01046 [INSPIRE].
  48. [48]
    T. Heimel, G. Kasieczka, T. Plehn and J.M. Thompson, QCD or what?, SciPost Phys.6 (2019) 030 [arXiv:1808.08979] [INSPIRE].ADSCrossRefGoogle Scholar
  49. [49]
    K. Kondo, Dynamical likelihood method for reconstruction of events with missing momentum. 1: method and toy models, J. Phys. Soc. Jap.57 (1988) 4126 [INSPIRE].ADSCrossRefGoogle Scholar
  50. [50]
    D0 collaboration, A precision measurement of the mass of the top quark, Nature429 (2004) 638 [hep-ex/0406031] [INSPIRE].
  51. [51]
    CDF collaboration, Measurement of the top quark mass with the dynamical likelihood method using lepton plus jets events with b-tags in pp̄ collisions at \( \sqrt{s} \) = 1.96 TeV, Phys. Rev.D 73 (2006) 092002 [hep-ex/0512009] [INSPIRE].
  52. [52]
    P. Artoisenet, V. Lemaitre, F. Maltoni and O. Mattelaer, Automation of the matrix element reweighting method, JHEP12 (2010) 068 [arXiv:1007.3300] [INSPIRE].ADSzbMATHCrossRefGoogle Scholar
  53. [53]
    T. Martini and P. Uwer, Extending the matrix element method beyond the Born approximation: calculating event weights at next-to-leading order accuracy, JHEP09 (2015) 083 [arXiv:1506.08798] [INSPIRE].ADSCrossRefGoogle Scholar
  54. [54]
    D.E. Soper and M. Spannowsky, Finding physics signals with shower deconstruction, Phys. Rev.D 84 (2011) 074002 [arXiv:1102.3480] [INSPIRE].ADSGoogle Scholar
  55. [55]
    D.E. Soper and M. Spannowsky, Finding top quarks with shower deconstruction, Phys. Rev.D 87 (2013) 054012 [arXiv:1211.3140] [INSPIRE].ADSGoogle Scholar
  56. [56]
    D.E. Soper and M. Spannowsky, Finding physics signals with event deconstruction, Phys. Rev.D 89 (2014) 094005 [arXiv:1402.1189] [INSPIRE].ADSGoogle Scholar
  57. [57]
    C. Englert, O. Mattelaer and M. Spannowsky, Measuring the Higgs-bottom coupling in weak boson fusion, Phys. Lett.B 756 (2016) 103 [arXiv:1512.03429] [INSPIRE].ADSCrossRefGoogle Scholar
  58. [58]
    D.E. Ferreira de Lima, O. Mattelaer and M. Spannowsky, Searching for processes with invisible particles using a matrix element-based method, Phys. Lett.B 787 (2018) 100 [arXiv:1712.03266] [INSPIRE].ADSCrossRefGoogle Scholar
  59. [59]
    S. Prestel and M. Spannowsky, HYTREES: combining matrix elements and parton shower for hypothesis testing, Eur. Phys. J.C 79 (2019) 546 [arXiv:1901.11035] [INSPIRE].ADSCrossRefGoogle Scholar
  60. [60]
    B. Kiran, D. Mathew Thomas and R. Parakkal, An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos, J. Imaging 4 (2018) [arXiv:1801.03149].
  61. [61]
    D.P. Kingma and M. Welling, Auto-encoding variational Bayes, arXiv:1312.6114 [INSPIRE].
  62. [62]
    P. Vincent, H. Larochelle, Y. Bengio and P.A. Manzagol, Extracting and composing robust features with denoising autoencoders, in the proceedings of the 25thInternational Conference on Machine Learning (ICML’08), July 5–9, New York, U.S.A. (2008).Google Scholar
  63. [63]
    S. Otten et al., Event generation and statistical sampling for physics with deep generative models and a density information buffer, arXiv:1901.00875 [INSPIRE].
  64. [64]
    M. Farina, Y. Nakai and D. Shih, Searching for new physics with deep autoencoders, arXiv:1808.08992 [INSPIRE].
  65. [65]
    J. Hajer, Y.-Y. Li, T. Liu and H. Wang, Novelty detection meets collider physics, arXiv:1807.10261 [INSPIRE].
  66. [66]
    T.S. Roy and A.H. Vijay, A robust anomaly finder based on autoencoder, arXiv:1903.02032 [INSPIRE].
  67. [67]
    K. Joshi, A.D. Pilkington and M. Spannowsky, The dependency of boosted tagging algorithms on the event colour structure, Phys. Rev.D 86 (2012) 114016 [arXiv:1207.6066] [INSPIRE].ADSGoogle Scholar
  68. [68]
    CMS collaboration, Search for anomalous tt ̄production in the highly-boosted all-hadronic final state, JHEP09 (2012) 029 [Erratum ibid.03 (2014) 132] [arXiv:1204.2488] [INSPIRE].
  69. [69]
    ATLAS collaboration, Search for heavy particles decaying into top-quark pairs using lepton-plus-jets events in proton-proton collisions at \( \sqrt{s} \) = 13 TeV with the ATLAS detector, Eur. Phys. J.C 78 (2018) 565 [arXiv:1804.10823] [INSPIRE].
  70. [70]
    ATLAS collaboration, Search for heavy particles decaying into a top-quark pair in the fully hadronic final state in pp collisions at \( \sqrt{s} \) = 13 TeV with the ATLAS detector, Phys. Rev.D 99 (2019) 092004 [arXiv:1902.10077] [INSPIRE].
  71. [71]
    ATLAS collaboration, Search for heavy higgs bosons A/H decaying to a top quark pair in pp collisions at \( \sqrt{s} \) = 8 TeV with the ATLAS detector, Phys. Rev. Lett.119 (2017) 191803 [arXiv:1707.06025] [INSPIRE].
  72. [72]
    J. Alwall et al., The automated computation of tree-level and next-to-leading order differential cross sections and their matching to parton shower simulations, JHEP07 (2014) 079 [arXiv:1405.0301] [INSPIRE].ADSCrossRefGoogle Scholar
  73. [73]
    G. Altarelli, B. Mele and M. Ruiz-Altaba, Searching for new heavy vector bosons in \( p\overline{p} \)colliders, Z. Phys.C 45 (1989) 109 [Erratum ibid.C 47 (1990) 676] [INSPIRE].
  74. [74]
    T. Plehn and M. Spannowsky, Top tagging, J. Phys.G 39 (2012) 083001 [arXiv:1112.4441] [INSPIRE].ADSCrossRefGoogle Scholar
  75. [75]
    T. Plehn, M. Spannowsky and M. Takeuchi, How to improve top tagging, Phys. Rev.D 85 (2012) 034029 [arXiv:1111.5034] [INSPIRE].ADSGoogle Scholar
  76. [76]
    Y.L. Dokshitzer, G.D. Leder, S. Moretti and B.R. Webber, Better jet clustering algorithms, JHEP08 (1997) 001 [hep-ph/9707323] [INSPIRE].ADSCrossRefGoogle Scholar
  77. [77]
    M. Cacciari, G.P. Salam and G. Soyez, FastJet user manual, Eur. Phys. J.C 72 (2012) 1896 [arXiv:1111.6097] [INSPIRE].ADSzbMATHCrossRefGoogle Scholar
  78. [78]
    A. Buckley et al., Rivet user manual, Comput. Phys. Commun.184 (2013) 2803 [arXiv:1003.0694] [INSPIRE].ADSCrossRefGoogle Scholar
  79. [79]
    ATLAS collaboration, Data-driven determination of the energy scale and resolution of jets reconstructed in the ATLAS calorimeters using dijet and multijet events at \( \sqrt{s} \) = 8 TeV, ATLAS-CONF-2015-017 (2015).Google Scholar
  80. [80]
    ATLAS collaboration, Performance of missing transverse momentum reconstruction in proton-proton collisions at 7 TeV with ATLAS, Eur. Phys. J.C 72 (2012) 1844 [arXiv:1108.5602] [INSPIRE].
  81. [81]
    D.P. Kingma and J. Ba, Adam: a method for stochastic optimization, arXiv:1412.6980 [INSPIRE].
  82. [82]
    F. Chollet et al., Keras, (2015).
  83. [83]
    M. Abadi et al., TensorFlow: large-scale machine learning on heterogeneous distributed systems, arXiv:1603.04467 [INSPIRE].
  84. [84]
    R. Frederix and F. Maltoni, Top pair invariant mass distribution: a window on new physics, JHEP01 (2009) 047 [arXiv:0712.2355] [INSPIRE].ADSCrossRefGoogle Scholar

Copyright information

© The Author(s) 2019

Authors and Affiliations

  1. 1.Institute for Particle Physics Phenomenology, Department of PhysicsDurham UniversityDurhamU.K.
  2. 2.Institute for Data ScienceDurham UniversityDurhamU.K.

Personalised recommendations