Information Augmentation, Reduction and Compression for Interpreting Multi-layered Neural Networks

  • Ryotaro KamimuraEmail author
Conference paper
Part of the Lecture Notes in Networks and Systems book series (LNNS, volume 70)


The present paper aims to propose a new type of learning method for interpreting relations between inputs and outputs in multi-layered neural networks. The method is composed of information augmentation, reduction and compression component. In the information augmentation component, information in inputs is forced to increase for the subsequent learning to choose appropriate information among many options. In the information reduction component, information is reduced by selectively choosing strong and active connection weights. Finally, in the information compression component, information contained in multi-layered neural networks is compressed by multiplying all connection weights in all layers for summarizing the main characteristics of connection weights. The method was applied to the improvement of an EC (electric commerce) web site for better profitability. The method could clarify relations between inputs and outputs and its interpretation was more natural than that by the conventional logistic regression analysis. The results suggest that multi-layered neural networks can be used to improve generalization and in addition to interpret final results, which is more important in many applications fields.


Information augmentation Reduction Compression Generalization Interpretation Multi-layered neural networks 


  1. 1.
    Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)zbMATHGoogle Scholar
  2. 2.
    Rakotomamonjy, A.: Variable selection using SVM-based criteria. J. Mach. Learn. Res. 3, 1357–1370 (2003)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Perkins, S., Lacker, K., Theiler, J.: Grafting: fast, incremental feature selection by gradient descent in function space. J. Mach. Learn. Res. 3, 1333–1356 (2003)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Wang, J., Perez, L.: The Effectiveness of Data Augmentation in Image Classification Using Deep Learning. Technical report (2017)Google Scholar
  5. 5.
    Asperti, A., Mastronardo, C.: The effectiveness of data augmentation for detection of gastrointestinal diseases from endoscopical images. arXiv:1712.03689 (2017)
  6. 6.
    Xu, Y., Jia, R.., Mou, L., Li, G., Chen, Y., Lu, Y., Jin, Z.: Improved relation classification by deep recurrent neural networks with data augmentation. arXiv:1601.03651 (2016)
  7. 7.
    Marchesi, M.: Megapixel Size Image Creation Using Generative Adversarial Networks (2017)Google Scholar
  8. 8.
    Kingma, D.P., Welling, M.: Auto-Encoding Variational Bayes (2013)Google Scholar
  9. 9.
    Allen, D.M.: The relationship between variable selection and data agumentation and a method for prediction. Technometrics 16(1), 125–127 (1974)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Moody, J., Hanson, S., Krogh, A., Hertz, J.A.: A simple weight decay can improve generalization. Adv. Neural Inf. Process. Syst. 4, 950–957 (1995)Google Scholar
  11. 11.
    Hinton, G.E.: A practical guide to training restricted boltzmann machines. In: Neural Networks: Tricks of the Trade. Springer, Berlin, pp. 599–619 (2012)Google Scholar
  12. 12.
    Nishiuchi, H.: Statistical Analysis for Billion People (in Japanese). Nikkei BP marketing, Tokyo, Japan (2014)Google Scholar
  13. 13.
    Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)zbMATHGoogle Scholar
  14. 14.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Tokai UniversityTokyoJapan

Personalised recommendations