Abstract
We introduce an algorithm that improves Neural Network classification/registration of corrupted data belonging to low-dimensional manifolds. The algorithm combines ideas of the Orthogonal Greedy Algorithm with the standard gradient back-propagation engine incorporated in Neural Networks. Therefore, we call it the Greedient algorithm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Becker, S., Le Cun, Y.: Improving the convergence of backpropagation learning with second order methods. In: Proceedings of the 1988 Connectionist Models Summer School, pp. 29–37. Morgan Kaufmann, San Matteo (1988)
Candès, E.J., Tao, T.: Decoding by linear programming. IEEE Trans. Inf. Theory 51, 4203–4215 (2005)
Donoho, D.: Compressed sensing. IEEE Trans. Inf. Theory 52, 1289–1306 (2006)
Martens, J., Sutskever, I., Swersky, K.: Estimating the Hessian by back-propagating curvature (2012). arXiv:1206.6464
Martens, J.: Second-order optimization for neural networks. Ph.D. thesis, University of Toronto (2016)
Mousavi, A., Baraniuk, R. G.: Learning to invert: signal recovery via deep convolutional networks (2017). arXiv:1701.03891v1 [stat.ML]
Petukhov, A.: Fast implementation of orthogonal greedy algorithm for tight wavelet frames. Eur. J. Signal Process. 86, 471–479 (2006)
Rudelson, M., Vershynin, R.: Geometric approach to error correcting codes and reconstruction of signals. Int. Math. Res. Not. 64, 4019–4041 (2005)
Simpson, A.: Deep Transform: Error Correction via Probabilistic Re-Synthesis (2015). arXiv:1502.04617 [cs.LG]
Simpson, A.: Deep Transform: Time-Domain Audio Error Correction via Probabilistic Re-Synthesis (2015). arXiv:1503.05849 [cs.SD]
Zhao, J., Mathieu, M., LeCun, Y.: Energy-Based Generative Adversarial Network (2017). arXiv:1609.03126 [cs.LG]
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer International Publishing AG, part of Springer Nature
About this chapter
Cite this chapter
Petukhov, A., Kozlov, I. (2019). Error Resilient Neural Networks on Low-Dimensional Manifolds. In: Esposito, A., Faundez-Zanuy, M., Morabito, F., Pasero, E. (eds) Neural Advances in Processing Nonlinear Dynamic Signals. WIRN 2017 2017. Smart Innovation, Systems and Technologies, vol 102. Springer, Cham. https://doi.org/10.1007/978-3-319-95098-3_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-95098-3_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-95097-6
Online ISBN: 978-3-319-95098-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)