Recurrent Neural Networks with Grid Data Quantization for Modeling LHC Superconducting Magnets Behavior

  • Maciej WielgoszEmail author
  • Andrzej Skoczeń
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 945)


This paper presents a model based on Recurrent Neural Network architecture, in particular LSTM, for modeling the behavior of Large Hadron Collider superconducting magnets. High resolution data available in Post Mortem database was used to train a set of models and compare their performance with respect to various hyper-parameters such as input data quantization and number of cells. A novel approach to signal level quantization allowed to reduce a size of the model, simplify tuning of the magnet monitoring system and make the process scalable. The paper shows that RNNs such as LSTM or GRU may be used for modeling high resolution signals with an accuracy over 0.95 and as small number of the parameters ranging from 800 to 1200. This makes the solution suitable for hardware implementation essential in the case of monitoring performance critical and high speed signal of Large Hadron Collider superconducting magnets.


LHC RNN GRU LSTM Signals modeling Anomaly detection 


  1. 1.
    Wielgosz, M., Skoczeń, A.: Recurrent Neural Networks with grid data quantization for modeling LHC superconducting magnets behavior. In: Kulczycki, P., Kowalski, P., Łukasik, S. (eds.) Contemporary Computational Science, p. 240. AGH-UST Press, Cracow (2018). Scholar
  2. 2.
    Brüning, O., Collier, P.: Building a behemoth. Nature 448, 285–289 (2007). Scholar
  3. 3.
    Evans, L., Bryant, P.: LHC machine. J. Instrum. 3(08), S08,001 (2008). Scholar
  4. 4.
    Wenninger, J.: Machine protection and operation for LHC. CERN Yellow Report CERN-2016-002 (2016)Google Scholar
  5. 5.
    Bordry, F., Denz, R., Mess, K.H., Puccio, B., Rodriguez-Mateos, F., Schmidt, R.: Machine protection for the LHC: architecture of the beam and powering interlock system. LHC Project Report 521, CERN (2001).
  6. 6.
    Schmidt, R.: Machine protection and interlock systems for circular machines – example for LHC. CERN Yellow Report CERN-2016-002 (2016)Google Scholar
  7. 7.
    Ciapala, E., Rodríguez-Mateos, F., Schmidt, R., Wenninger, J.: The LHC post-mortem system. Technical report LHC-PROJECT-NOTE-303, CERN, Geneva (2002).
  8. 8.
    Lauckner, R.J.: What data is needed to understand failures during LHC operation. In: 11th Workshop of the LHC, Chamonix XI, pp. 278–283 (2001). CERN-SL-2001-003.
  9. 9.
    Borland, M.: A brief introduction to the SDDS Toolkit. Technical report, Argonne National Laboratory, USA (1998).
  10. 10.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25, pp. 1097–1105. Curran Associates, Inc. (2012).
  11. 11.
    LeCun, Y.: Deep learning of convolutional networks. In: 2015 IEEE Hot Chips 27 Symposium (HCS), pp. 1–95 (2015).
  12. 12.
    Graves, A.: Neural Networks. Springer, Heidelberg (2012). Scholar
  13. 13.
    Morton, J., Wheeler, T.A., Kochenderfer, M.J.: Analysis of recurrent neural networks for probabilistic modelling of driver behaviour. IEEE Trans. Intell. Transp. Syst. PP(99), 1–10 (2016). Scholar
  14. 14.
    Pouladi, F., Salehinejad, H., Gilani, A.M.: Recurrent neural networks for sequential phenotype prediction in genomics. In: 2015 International Conference on Developments of E-Systems Engineering (DeSE), pp. 225–230 (2015).
  15. 15.
    Chen, X., Liu, X., Wang, Y., Gales, M.J.F., Woodland, P.C.: Efficient training and evaluation of recurrent neural network language models for automatic speech recognition. IEEE/ACM Trans. Audio Speech Lang. Process. 24(11), 2146–2157 (2016). Scholar
  16. 16.
    Greff, K., Srivastava, R.K., Koutník, J., Steunebrink, B.R., Schmidhuber, J.: LSTM: a search space odyssey. IEEE Trans. Neural Netw. Learn. Syst. 28(10), 2222–2232 (2017). Scholar
  17. 17.
    Wielgosz, M., Skoczeń, A., Mertik, M.: Using LSTM recurrent neural networks for detecting anomalous behavior of LHC superconducting magnets. Nucl. Instrum. Methods Phys. Res. A 867, 40–50 (2017). Scholar
  18. 18.
    Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Gated feedback recurrent neural networks. In: Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37, ICML 2015, pp. 2067–2075. (2015).
  19. 19.
    Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. In: NIPS 2014 Workshop on Deep Learning, December 2014Google Scholar
  20. 20.
    Malhotra, P., Vig, L., Shroff, G., Agarwal, P.: Long short term memory networks for anomaly detection in time series. In: Proceedings of the 23rd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, ESANN 2015, Bruges, Belgium, pp. 89–94. Presses universitaires de Louvain (2015).
  21. 21.
    Marchi, E., Vesperini, F., Eyben, F., Squartini, S., Schuller, B.: A novel approach for automatic acoustic novelty detection using a denoising autoencoder with bidirectional LSTM neural networks. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1996–2000 (2015).
  22. 22.
    Marchi, E., Vesperini, F., Weninger, F., Eyben, F., Squartini, S., Schuller, B.: Non-linear prediction with LSTM recurrent neural networks for acoustic novelty detection. In: 2015 International Joint Conference on Neural Networks (IJCNN), pp. 1–7 (2015).
  23. 23.
    Chong, Y.S., Tay, Y.H.: Abnormal event detection in videos using spatiotemporal autoencoder. In: Cong, F., Leung, A., Wei, Q. (eds.) Advances in Neural Networks - ISNN 2017, pp. 189–196. Springer, Cham (2017)CrossRefGoogle Scholar
  24. 24.
    Chang, A.X.M., Martini, B., Culurciello, E.: Recurrent neural networks hardware implementation on FPGA. CoRR abs/1511.05552 (2015).
  25. 25.
    Han, S., Kang, J., Mao, H., Hu, Y., Li, X., Li, Y., Xie, D., Luo, H., Yao, S., Wang, Y., Yang, H., Dally, W.B.J.: ESE: efficient speech recognition engine with sparse LSTM on FPGA. In: Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA 2017), pp. 75–84 (2017).
  26. 26.
    Lee, M., Hwang, K., Park, J., Choi, S., Shin, S., Sung, W.: FPGA-based low-power speech recognition with recurrent neural networks. In: 2016 IEEE International Workshop on Signal Processing Systems (SiPS), pp. 230–235 (2016)Google Scholar
  27. 27.
    Strecht, P., Cruz, L., Soares, C., Mendes-Moreira, J., Abreu, R.: A comparative study of regression and classification algorithms for modelling students’ academic performance. In: Proceedings of the 8th International Conference on Educational Data Mining, EDM 2015, Madrid, Spain, 26–29 June 2015, pp. 392–395 (2015).
  28. 28.
    Chollet, F., et al.: Keras. GitHub (2015). GitHub repository.

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Faculty of Computer Science, Electronics and TelecommunicationsAGH University of Science and TechnologyKrakówPoland
  2. 2.Faculty of Physics and Applied Computer ScienceAGH University of Science and TechnologyKrakówPoland

Personalised recommendations