Safety and Robustness of Deep Neural Networks Object Recognition Under Generic Attacks

Abstract

Embedding machine or deep learning software into safety-critical systems such as autonomous vehicles requires software verification and validation. Such software adds non traceable hazards to traditional hardware and sensors failures, not to mention attacks that fool the prediction of a DNN and hampers its robustness. Formal methods from computer science are now applied to deep neural networks to assess the local and global robustness of a given DNN. Typically static analysis with Abstract Interpretation or SAT solvers approaches are applied to neural networks and leverages the important progress of formal methods over the last decades. Such approaches estimate bounds on the perturbation of the inputs and formally guarantee the same DNN prediction within these bounds. However formal methods over DNN for image perception system have only been applied to simple image attacks (2D rotation, brightness). In this work, we extend the definition of Lower and Upper Bounds to assess the robustness of a DNN perception system against more generic attacks. We propose a general method to verify object recognition systems using Abstract Interpretation theory. Another major contribution is the adaptation of Upper and Lower Bounds with the abstract intervals to support more complex attacks. We consider the three following classes: convolutional attacks, occlusion attacks and geometrical transformations. For the last one, we generalize the geometrical transformations with displacements in the three-dimensional space.