Advertisement

International Journal of Computer Vision

, Volume 127, Issue 6–7, pp 719–742 | Cite as

Detecting and Mitigating Adversarial Perturbations for Robust Face Recognition

  • Gaurav Goswami
  • Akshay Agarwal
  • Nalini Ratha
  • Richa Singh
  • Mayank VatsaEmail author
Article

Abstract

Deep neural network (DNN) architecture based models have high expressive power and learning capacity. However, they are essentially a black box method since it is not easy to mathematically formulate the functions that are learned within its many layers of representation. Realizing this, many researchers have started to design methods to exploit the drawbacks of deep learning based algorithms questioning their robustness and exposing their singularities. In this paper, we attempt to unravel three aspects related to the robustness of DNNs for face recognition: (i) assessing the impact of deep architectures for face recognition in terms of vulnerabilities to attacks, (ii) detecting the singularities by characterizing abnormal filter response behavior in the hidden layers of deep networks; and (iii) making corrections to the processing pipeline to alleviate the problem. Our experimental evaluation using multiple open-source DNN-based face recognition networks, and three publicly available face databases demonstrates that the performance of deep learning based face recognition algorithms can suffer greatly in the presence of such distortions. We also evaluate the proposed approaches on four existing quasi-imperceptible distortions: DeepFool, Universal adversarial perturbations, \(l_2\), and Elastic-Net (EAD). The proposed method is able to detect both types of attacks with very high accuracy by suitably designing a classifier using the response of the hidden layers in the network. Finally, we present effective countermeasures to mitigate the impact of adversarial attacks and improve the overall robustness of DNN-based face recognition.

Keywords

Face recognition Deep learning Adversarial Dropout Adversarial learning Attack detection Attack mitigation 

Notes

Acknowledgements

G. Goswami was partly supported through IBM PhD Fellowship, A. Agarwal is partly supported by Visvesvaraya PhD Fellowship, and M. Vatsa and R. Singh are partly supported through CAI@IIIT-Delhi. M. Vatsa is also partially supported through Department of Science and Technology, Government of India through Swarnajayanti Fellowship.

References

  1. Addad, B., Kodjabashian, J., & Meyer, C. (2018). Clipping free attacks against artificial neural networks. arXiv preprint arXiv:1803.09468.
  2. Agarwal, A., Singh, R., & Vatsa, M. (2016). Face anti-spoofing using haralick features. In 2016 IEEE 8th international conference on biometrics theory, applications and systems (pp. 1–6).Google Scholar
  3. Agarwal, A., Singh, R., Vatsa, M., & Noore, A. (2017a). SWAPPED! Digital face presentation attack detection via weighted local magnitude pattern. In 2017 IEEE International Joint Conference on Biometrics (IJCB) (pp. 659–665).  https://doi.org/10.1109/BTAS.2017.8272754.
  4. Agarwal, A., Singh, R., Vatsa, M., & Ratha, N. (2018). Are image-agnostic universal adversarial perturbations for face recognition difficult to detect? In IEEE international conference on biometrics: Theory, applications, and systems.Google Scholar
  5. Agarwal, A., Yadav, D., Kohli, N., Singh, R., Vatsa, M., & Noore, A. (2017b). Face presentation attack with latex masks in multispectral videos. In IEEE conference on computer vision and pattern recognition workshops (pp. 275–283).Google Scholar
  6. Akbulut, Y., Şengür, A., Budak, Ü., & Ekici, S. (2017). Deep learning based face liveness detection in videos. In 2017 international artificial intelligence and data processing symposium (IDAP) (pp. 1–4). Malatya.  https://doi.org/10.1109/IDAP.2017.8090202.
  7. Akhtar, N., Liu, J., & Mian, A. (2017). Defense against universal adversarial perturbations. arXiv preprint arXiv:1711.05929.
  8. Akhtar, N., & Mian, A. (2018). Threat of adversarial attacks on deep learning in computer vision: A survey. IEEE Access, 6, 14410–14430.CrossRefGoogle Scholar
  9. Alaifari, R., Alberti, G. S., & Gauksson, T. (2018). Adef: An iterative algorithm to construct adversarial deformations. arXiv preprint arXiv:1804.07729.
  10. Amos, B., Ludwiczuk, B., Harkes, J., Pillai, P., Elgazzar, K., & Satyanarayanan, M. (2016). OpenFace: Face recognition with deep neural networks. http://github.com/cmusatyalab/openface. Accessed 10 Apr 2016.
  11. Athalye, A., & Sutskever, I. (2018). Synthesizing robust adversarial examples. In International conference on machine learning.Google Scholar
  12. Bay, H., Tuytelaars, T., & Van Gool, L. (2006). Surf: Speeded up robust features. In European conference on computer vision (pp. 404–417).Google Scholar
  13. Beveridge, J., Phillips, P., Bolme, D., Draper, B., Given, G., Lui, Y. M., Teli, M., Zhang, H., Scruggs, W., Bowyer, K., Flynn, P., & Cheng, S. (2013). The challenge of face recognition from digital point-and-shoot cameras. In IEEE conference on biometrics: Theory, applications and systems Google Scholar
  14. Bhagoji, A. N., Cullina, D., & Mittal, P. (2017). Dimensionality reduction as a defense against evasion attacks on machine learning classifiers. arXiv preprint arXiv:1704.02654.
  15. Bharati, A., Singh, R., Vatsa, M., & Bowyer, K. W. (2016). Detecting facial retouching using supervised deep learning. IEEE Transactions on Information Forensics and Security, 11(9), 1903–1913.CrossRefGoogle Scholar
  16. Biggio, B., Fumera, G., Marcialis, G. L., & Roli, F. (2017). Statistical meta-analysis of presentation attacks for secure multibiometric systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(3), 561–575.CrossRefGoogle Scholar
  17. Boulkenafet, Z., Komulainen, J., & Hadid, A. (2016). Face spoofing detection using colour texture analysis. IEEE Transactions on Information Forensics and Security, 11(8), 1818–1830.CrossRefGoogle Scholar
  18. Boulkenafet, Z., Komulainen, J., & Hadid, A. (2017). Face antispoofing using speeded-up robust features and fisher vector encoding. IEEE Signal Processing Letters, 24(2), 141–145.Google Scholar
  19. Bousmalis, K., Trigeorgis, G., Silberman, N., Krishnan, D., & Erhan, D. (2016). Domain separation networks. Advances in Neural Information Processing Systems, 29, 343–351.Google Scholar
  20. Carlini, N., & Wagner, D. (2017). Towards evaluating the robustness of neural networks. In IEEE symposium on security and privacy (pp. 39–57).Google Scholar
  21. Chen, J., Deng, Y., Bai, G., & Su, G. (2015). Face image quality assessment based on learning to rank. IEEE Signal Processing Letters, 22(1), 90–94.CrossRefGoogle Scholar
  22. Chen, P. Y., Sharma, Y., Zhang, H., Yi, J., & Hsieh, C. J. (2018). EAD: elastic-net attacks to deep neural networks via adversarial examples. In Thirty-second AAAI conference on artificial intelligence.Google Scholar
  23. Chhabra, S., Singh, R., Vatsa, M., & Gupta, G. (2018). Anonymizing k-facial attributes via adversarial perturbations. In International joint conferences on artificial intelligence (pp. 656–662).Google Scholar
  24. Chingovska, I., Erdogmus, N., Anjos, A., & Marcel, S. (2016). Face recognition systems under spoofing attacks. In T. Bourlai (Ed.), Face recognition across the imaging spectrum. Cham: Springer.  https://doi.org/10.1007/978-3-319-28501-6_8.
  25. Cisse, M. M., Adi, Y., Neverova, N., & Keshet, J. (2017). Houdini: Fooling deep structured visual and speech recognition models with adversarial examples. In Advances in neural information processing systems (pp. 6977–6987).Google Scholar
  26. Das, N., Shanbhogue, M., Chen, S. T., Hohman, F., Chen, L., Kounavis, M. E., & Chau, D. H. (2017). Keeping the bad guys out: Protecting and vaccinating deep learning with jpeg compression. arXiv preprint arXiv:1705.02900.
  27. de Souza, G. B., da Silva Santos, D. F., Pires, R. G., Marana, A. N., & Papa, J. P. (2017). Deep texture features for robust face spoofing detection. IEEE Transactions on Circuits and Systems II: Express Briefs, 64(12), 1397–1401.CrossRefGoogle Scholar
  28. Deng, J., Dong, W., Socher, R., Li, L., Li, K., & Li, F.-F. (2009). ImageNet: A large-scale hierarchical image database. In IEEE conference on computer vision and pattern recognition (pp. 248–255).Google Scholar
  29. Dziugaite, G. K., Ghahramani, Z., & Roy, D. M. (2016). A study of the effect of jpg compression on adversarial images. arXiv preprint arXiv:1608.00853.
  30. Feinman, R., Curtin, R. R., Shintre, S., & Gardner, A. B. (2017). Detecting adversarial samples from artifacts. arXiv preprint arXiv:1703.00410.
  31. Gan, J., Li, S., Zhai, Y., & Liu, C. (2017). 3d convolutional neural network based on face anti-spoofing. In 2017 2nd international conference on multimedia and image processing (ICMIP) (pp. 1–5).Google Scholar
  32. Goel, A., Singh, A., Agarwal, A., Vatsa, M., & Singh, R. (2018). Smartbox: Benchmarking adversarial detection and mitigation algorithms for face recognition. In IEEE International conference on biometrics: Theory, applications, and systems Google Scholar
  33. Gong, Z., Wang, W., & Ku, W. S. (2017). Adversarial and clean data are not twins. arXiv preprint arXiv:1704.04960.
  34. Goodfellow, I. J., Shlens, J., & Szegedy, C. (2015). Explaining and harnessing adversarial examples. In International conference on learning representations.Google Scholar
  35. Goswami, G., Ratha, N., Agarwal, A., Singh, R., & Vatsa, M. (2018). Unravelling robustness of deep learning based face recognition against adversarial attacks. In Association for the advancement of artificial intelligence.Google Scholar
  36. Gross, R., Matthews, I., Cohn, J., Kanade, T., & Baker, S. (2010). Multi-PIE. Image and Vision Computing, 28(5), 807–813.CrossRefGoogle Scholar
  37. Grosse, K., Manoharan, P., Papernot, N., Backes, M., & McDaniel, P. (2017). On the (statistical) detection of adversarial examples. arXiv preprint arXiv:1702.06280.
  38. Gu, S., & Rigazio, L. (2014). Towards deep neural network architectures robust to adversarial examples. arXiv preprint arXiv:1412.5068.
  39. Guo, C., Rana, M., Cissé, M., & van der Maaten, L. (2018). Countering adversarial images using input transformations. In International conference on learning representations.Google Scholar
  40. Hinton, G., Vinyals, O., & Dean, J. (2015). Distilling the knowledge in a neural network. Stat, 1050, 9.Google Scholar
  41. Huang, G. B., Ramesh, M., Berg, T., & Learned-Miller, E. (2007). Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Tech. Rep. 07–49, University of Massachusetts, Amherst.Google Scholar
  42. King, D. E. (2009). Dlib-ml: A machine learning toolkit. Journal of Machine Learning Research, 10, 1755–1758.Google Scholar
  43. Kurakin, A., Goodfellow, I., & Bengio, S. (2016). Adversarial examples in the physical world. arXiv preprint arXiv:1607.02533.
  44. Laskov, P., & Lippmann, R. (2010). Machine learning in adversarial environments. Machine Learning, 81(2), 115–119.CrossRefGoogle Scholar
  45. Lee, H., Han, S., & Lee, J. (2017). Generative adversarial trainer: Defense to adversarial perturbations with gan. arXiv preprint arXiv:1705.03387.
  46. Li, X., & Li, F. (2017). Adversarial examples detection in deep networks with convolutional filter statistics. In International conference on computer vision.Google Scholar
  47. Liang, B., Li, H., Su, M., Li, X., Shi, W., & Wang, X. (2017). Detecting adversarial examples in deep networks with adaptive noise reduction. URL arXiv:1705.08378
  48. Liu, J., Deng, Y., Bai, T., & Huang, C. (2015). Targeting ultimate accuracy: Face recognition via deep embedding. URL arXiv:1506.07310.
  49. Liu, L., Liu, B., Huang, H., & Bovik, A. C. (2014). No-reference image quality assessment based on spatial and spectral entropies. Signal Processing: Image Communication, 29(8), 856–863.Google Scholar
  50. Liu, M. Y., & Tuzel, O. (2016). Coupled generative adversarial networks. Advances in Neural Information Processing Systems, 29, 469–477.Google Scholar
  51. Lu, J., Issaranon, T., & Forsyth, D. (2017). Safetynet: Detecting and rejecting adversarial examples robustly. In IEEE international conference on computer vision (pp. 446–454).Google Scholar
  52. Luo, Y., Boix, X., Roig, G., Poggio, T., & Zhao, Q. (2015). Foveation-based mechanisms alleviate adversarial examples. arXiv preprint arXiv:1511.06292.
  53. Majumdar, A., Singh, R., & Vatsa, M. (2017). Face verification via class sparsity based supervised encoding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1273–1280.CrossRefGoogle Scholar
  54. Manjani, I., Tariyal, S., Vatsa, M., Singh, R., & Majumdar, A. (2017). Detecting silicone mask-based presentation attack via deep dictionary learning. IEEE Transactions on Information Forensics and Security, 12(7), 1713–1723.CrossRefGoogle Scholar
  55. Meng, D., & Chen, H. (2017). Magnet: a two-pronged defense against adversarial examples. In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security (pp. 135–147).Google Scholar
  56. Metzen, J. H., Genewein, T., Fischer, V., & Bischoff, B. (2017). On detecting adversarial perturbations. In International conference on learning representations.Google Scholar
  57. Miyato, T., Dai, A. M., & Goodfellow, I. (2017). Adversarial training methods for semi-supervised text classification. In International conference on learning representations.Google Scholar
  58. Moorthy, A. K., & Bovik, A. C. (2010). A two-step framework for constructing blind image quality indices. IEEE Signal Processing Letters, 17(5), 513–516.CrossRefGoogle Scholar
  59. Moosavi-Dezfooli, S.M., Fawzi, A., Fawzi, O., & Frossard, P. (2017). Universal adversarial perturbations. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1765–1773).Google Scholar
  60. Moosavi-Dezfooli, S. M., Fawzi, A., & Frossard, P. (2016). Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2574–2582).Google Scholar
  61. Multiple encounters dataset (MEDS). (2011). Retrieved October 6, 2017 from http://www.nist.gov/itl/iad/ig/sd32.cfm.
  62. Nayebi, A., & Ganguli, S. (2017). Biologically inspired protection of deep networks from adversarial attacks. arXiv preprint arXiv:1703.09202.
  63. Nguyen, A., Yosinski, J., & Clune, J. (2015). Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In IEEE conference on computer vision and pattern recognition (pp. 427–436).Google Scholar
  64. NIST face recognition vendor test ongoing. (2018). Retrieved December 10, 2017 from https://www.nist.gov/programs-projects/face-recognition-vendor-test-frvt-ongoing.
  65. Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z. B., & Swami, A. (2017). Practical black-box attacks against machine learning. In Proceedings of the ACM on Asia conference on computer and communications security (pp. 506–519). ACM.Google Scholar
  66. Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z. B., & Swami, A. (2016a). The limitations of deep learning in adversarial settings. In IEEE European symposium on security and privacy (pp. 372–387).Google Scholar
  67. Papernot, N., McDaniel, P., Wu, X., Jha, S., & Swami, A. (2016b). Distillation as a defense to adversarial perturbations against deep neural networks. In 2016 IEEE symposium on security and privacy (pp. 582–597).Google Scholar
  68. Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015). Deep face recognition. In British machine vision conference (vol. 1, p. 6).Google Scholar
  69. Patel, K., Han, H., Jain, A. K., & Ott, G. (2015). Live face video vs. spoof face video: Use of moire patterns to detect replay video attacks. In 2015 international conference on biometrics (pp. 98–105).Google Scholar
  70. Phillips, P. J., Flynn, P. J., Beveridge, J. R., Scruggs, W., O’Toole, A. J., Bolme, D., Bowyer, K. W., Draper, B. A., Givens G. H., Lui, Y. M., Sahibzada, H., Scallan, J. A., & Weimer, S. (2009). Overview of the multiple biometrics grand challenge. In Advances in biometrics, (pp. 705–714).Google Scholar
  71. Prakash, A., Moran, N., Garber, S., DiLillo, A., & Storer, J. (2018). Deflecting adversarial attacks with pixel deflection. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 8571–8580).Google Scholar
  72. Radford, A., Metz, L., & Chintala, S. (2015). Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434.
  73. Raghavendra, R., Venkatesh, S., Raja, K., Cheikh, F., & Busch, C. (2017). On the vulnerability of extended multispectral face recognition systems towards presentation attacks. In IEEE international conference on identity, security and behavior analysis.Google Scholar
  74. Rakin, A. S., Yi, J., Gong, B., & Fan, D. (2018). Defend deep neural networks against adversarial examples via fixed anddynamic quantized activation functions. arXiv preprint arXiv:1807.06714.
  75. Ramachandra, R., & Busch, C. (2017). Presentation attack detection methods for face recognition systems: A comprehensive survey. ACM Computing Survey, 50(1), 8:1–8:37.Google Scholar
  76. Ranjan, R., Sankaranarayanan, S., Castillo, C. D., & Chellappa, R. (2017). Improving network robustness against adversarial attacks with compact convolution. arXiv preprint arXiv:1712.00699.
  77. Ratha, N. K., Connell, J. H., & Bolle, R. M. (2001). An analysis of minutiae matching strength. In Audio- and video-based biometric person authentication: Third international conference, proceedings (pp. 223–228).Google Scholar
  78. Rauber, J., Brendel, W., & Bethge, M. (2017). Foolbox v0.8.0: A python toolbox to benchmark the robustness of machine learning models. URL arXiv:1707.04131.
  79. Ross, A. S., & Doshi-Velez, F. (2018). Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients. In Thirty-second AAAI conference on artificial intelligence.Google Scholar
  80. Rozsa, A., Günther, M., & Boult, T. E. (2017a). LOTS about attacking deep features. In 2017 IEEE International Joint Conference on Biometrics (IJCB) (pp. 168–176). Denver, CO.  https://doi.org/10.1109/BTAS.2017.8272695.
  81. Rozsa, A., Günther, M., Rudd, E. M., & Boult, T. E. (2016). Are facial attributes adversarially robust? In International conference on pattern recognition (pp. 3121–3127).Google Scholar
  82. Rozsa, A., Günther, M., Rudd, E. M., & Boult, T. E. (2017b). Facial attributes: Accuracy and adversarial robustness. Pattern Recognition Letters.  https://doi.org/10.1016/j.patrec.2017.10.024.
  83. Rudd, E. M., Gunther, M., & Boult, T. E. (2016). Paraph: Presentation attack rejection by analyzing polarization hypotheses. In The IEEE conference on computer vision and pattern recognition workshops.Google Scholar
  84. Sabour, S., Cao, Y., Faghri, F., & Fleet, D. J. (2016). Adversarial manipulation of deep representations. In International conference on learning representations.Google Scholar
  85. Samangouei, P., Kabkab, M., & Chellappa, R. (2018). Defense-gan: Protecting classifiers against adversarial attacks using generative models. In International conference on learning representations.Google Scholar
  86. Schroff, F., Kalenichenko, D., & Philbin, J. (2015). Facenet: A unified embedding for face recognition and clustering. In IEEE conference on computer vision and pattern recognition (pp. 815–823).Google Scholar
  87. Sharif, M., Bhagavatula, S., Bauer, L., & Reiter, M. K. (2016). Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. In ACM SIGSAC conference on computer and communications security (pp. 1528–1540).Google Scholar
  88. Siddiqui, T. A., Bharadwaj, S., Dhamecha, T. I., Agarwal, A., Vatsa, M., Singh, R., & Ratha, N. (2016). Face anti-spoofing with multifeature videolet aggregation. In IEEE international conference on pattern recognition (pp. 1035–1040).Google Scholar
  89. Singh, M., Singh, R., Vatsa, M., Ratha, N., & Chellappa, R. (2019). Recognizing disguised faces in the wild. IEEE Transactions on Biometrics, Behavior, and Identity Science.  https://doi.org/10.1109/TBIOM.2019.2903860.
  90. Smith, D. F., Wiliem, A., & Lovell, B. C. (2015). Face recognition on consumer devices: Reflections on replay attacks. IEEE Transactions on Information Forensics and Security, 10(4), 736–745.CrossRefGoogle Scholar
  91. Song, Y., Kim, T., Nowozin, S., Ermon, S., & Kushman, N. (2018). Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. In International conference on learning representations.Google Scholar
  92. Sun, Y., Wang, X., & Tang, X. (2015). Deeply learned face representations are sparse, selective, and robust. In The IEEE conference on computer vision and pattern recognition.Google Scholar
  93. Suykens, J. A., & Vandewalle, J. (1999). Least squares support vector machine classifiers. Neural Processing Letters, 9(3), 293–300.CrossRefGoogle Scholar
  94. Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2014). Intriguing properties of neural networks. In International conference on learning representations. URL arXiv:1312.6199.
  95. Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014). DeepFace: Closing the Gap to Human-Level Performance in Face Verification. In IEEE conference on computer vision and pattern recognition (pp. 1701 – 1708).Google Scholar
  96. Tramèr, F., Kurakin, A., Papernot, N., Goodfellow, I., Boneh, D., & McDaniel, P. (2018). Ensemble adversarial training: Attacks and defenses. In International conference on learning representations.Google Scholar
  97. Viola, P., & Jones, M. J. (2004). Robust real-time face detection. International Journal of Computer Vision, 57(2), 137–154.CrossRefGoogle Scholar
  98. Wu, X., He, R., Sun, Z., & Tan, T. (2018). A light cnn for deep face representation with noisy labels. IEEE Transactions on Information Forensics and Security, 13(11), 2884–2896.CrossRefGoogle Scholar
  99. Xie, C., Wang, J., Zhang, Z., Ren, Z., & Yuille, A. (2018). Mitigating adversarial effects through randomization. In International conference on learning representations.Google Scholar
  100. Xie, C., Wang, J., Zhang, Z., Zhou, Y., Xie, L., & Yuille, A. (2017). Adversarial examples for semantic segmentation and object detection. In IEEE international conference on computer vision.Google Scholar
  101. Xu, W., Evans, D., & Qi, Y. (2018). Feature squeezing: Detecting adversarial examples in deep neural networks. In Network and distributed system security symposium.Google Scholar
  102. Ye, S., Wang, S., Wang, X., Yuan, B., Wen, W., & Lin, X. (2018). Defending DNN adversarial attacks with pruning and logits augmentation. In International conference on learning representations workshop.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.IIIT-DelhiNew DelhiIndia
  2. 2.IBM, TJ Watson Research CenterYorktown HeightsUSA

Personalised recommendations