Abstract
Deep neural networks have shown remarkable performance across a wide range of vision-based tasks, particularly due to the availability of large-scale datasets for training and better architectures. However, data seen in the real world are often affected by distortions that not accounted for by the training datasets. In this paper, we address the challenge of robustness and stability of neural networks and propose a general training method that can be used to make the existing neural network architectures more robust and stable to input visual perturbations while using only available datasets for training. Proposed training method is convenient to use as it does not require data augmentation or changes in the network architecture. We provide theoretical proof as well as empirical evidence for the efficiency of the proposed training method by performing experiments with existing neural network architectures and demonstrate that same architecture when trained with the proposed training method perform better than when trained with conventional training approach in the presence of noisy datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Szegedy, C., et al.: Going deeper with convolutions. In: CVPR, June 2015
Wang, F., et al.: Residual attention network for image classification. CoRR, abs/1704.06904 (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, June 2016
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) NIPS 2012, pp. 1097–1105. Curran Associates Inc. (2012)
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: CVPR, June 2016
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) NIPS 2015, pp. 91–99. Curran Associates Inc. (2015)
Vinyals, O., Toshev, A., Bengio, S., Erhan, D.: Show and tell: a neural image caption generator. CoRR, abs/1411.4555 (2014)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Silver, D., et al.: Mastering the game of go without human knowledge. Nature 550, 354 (2017)
Caterini, A.L., Chang, D.E.: Deep Neural Networks in a Mathematical Framework. SCS. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75304-1
Deng, J., Dong, W., Socher, R., Li, L., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: CVPR, pp. 248–255, June 2009
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. CoRR, abs/1603.05027 (2016)
Paszke, A., et al.: Automatic differentiation in pytorch, Alban Desmaison (2017)
Abadi, M., et al.: TensorFlow: Large-scale machine learning on heterogeneous systems (2015). Software tensorflow.org
Dodge, S.F., Karam, L.J.: Understanding how image quality affects deep neural networks. CoRR, abs/1604.04004 (2016)
Vasiljevic, I., Chakrabarti, A., Shakhnarovich, G.: Examining the impact of blur on recognition by convolutional networks. CoRR, abs/1611.05760 (2016)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556 (2014)
Rozsa, A., Günther, M., Boult, T.E.: Towards robust deep neural networks with BANG. CoRR, abs/1612.00138 (2016)
Zheng, S., Song, Y., Leung, T., Goodfellow, I.J.: Improving the robustness of deep neural networks via stability training. CoRR, abs/1604.04326 (2016)
Zhou, Y., Song, S., Cheung, N.-M.: On classification of distorted images with deep convolutional neural networks. CoRR, abs/1701.01924 (2017)
Dodge, S.F., Karam, L.J.: Quality resilient deep neural networks. CoRR, abs/1703.08119 (2017)
Szegedy, C., et al.: Intriguing properties of neural networks. CoRR, abs/1312.6199 (2013)
Goodfellow, I., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: International Conference on Learning Representations (2015)
LeCun, Y., Cortes, C.: MNIST handwritten digit database (2010)
Krizhevsky, A.: Learning multiple layers of features from tiny images. Technical report (2009)
Coates, A., Ng, A., Lee. H.: An analysis of single-layer networks in unsupervised feature learning. In: Gordon, G., Dunson, D., Dudk, M. (eds.) Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, vol. 15 of Proceedings of Machine Learning Research, pp. 215–223, Fort Lauderdale, FL, USA, 11–13 Apr 2011. PMLR (2011)
Acknowledgement
This research has been in part supported by the ICT R&D program of MSIP/IITP [2016-0-00563, Research on Adaptive Machine Learning Technology Development for Intelligent Autonomous Digital Companion].
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Usama, M., Chang, D.E. (2019). Towards Robust Neural Networks with Lipschitz Continuity. In: Yoo, C., Shi, YQ., Kim, H., Piva, A., Kim, G. (eds) Digital Forensics and Watermarking. IWDW 2018. Lecture Notes in Computer Science(), vol 11378. Springer, Cham. https://doi.org/10.1007/978-3-030-11389-6_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-11389-6_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-11388-9
Online ISBN: 978-3-030-11389-6
eBook Packages: Computer ScienceComputer Science (R0)