Advertisement

Simulating Homomorphic Evaluation of Deep Learning Predictions

  • Christina Boura
  • Nicolas Gama
  • Mariya GeorgievaEmail author
  • Dimitar Jetchev
Conference paper
  • 613 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11527)

Abstract

Convolutional neural networks (CNNs) is a category of deep neural networks that are primarily used for classifying image data. Yet, their continuous gain in popularity poses important privacy concerns for the potentially sensitive data that they process. A solution to this problem is to combine CNNs with Fully Homomorphic Encryption (FHE) techniques. In this work, we study this approach by focusing on two popular FHE schemes, \(\mathsf {TFHE}\) and \(\mathsf {HEAAN}\), that can work in the approximated computational model. We start by providing an analysis of the noise after each principal homomorphic operation, i.e. multiplication, linear combination, rotation and bootstrapping. Then, we provide a theoretical study on how the most important non-linear operations of a CNN (i.e. \(\max , \mathtt {Abs}, \mathtt {ReLU} \)), can be evaluated in each scheme. Finally, we measure via practical experiments on the plaintext the robustness of different neural networks against perturbations of their internal weights that could potentially result from the propagation of large homomorphic noise. This allows us to simulate homomorphic evaluations with large amounts of noise and to predict the effect on the classification accuracy without a real evaluation of heavy and time-consuming homomorphic operations. In addition, this approach enables us to correctly choose smaller and more efficient parameter sets for both schemes.

Keywords

Neural networks Homomorphic encryption \(\mathsf {TFHE}\) \(\mathsf {HEAAN}\) 

References

  1. 1.
  2. 2.
    Track 2: Secure parallel genome wide association studies using homomorphic encryption (2018). www.humangenomeprivacy.org/2018/competition-tasks.html
  3. 3.
    Albrecht, M., et al.: Homomorphic encryption security standard. Technical report, HomomorphicEncryption.org, Toronto, Canada, November 2018
  4. 4.
    Badawi, A.A., et al.: The AlexNet moment for homomorphic encryption: HCNN, the first homomorphic CNN on encrypted data with GPUs. Cryptology ePrint Archive, Report 2018/1056 (2018). https://eprint.iacr.org/2018/1056
  5. 5.
    Boura, C., Chillotti, I., Gama, N., Jetchev, D., Peceny, S., Petric, A.: High-precision privacy-preserving real-valued function evaluation. IACR Cryptology ePrint Archive 2017, 1234 (2017)Google Scholar
  6. 6.
    Boura, C., Gama, N., Georgieva, M.: Chimera: a unified framework for B/FV, TFHE and HEAAN fully homomorphic encryption and predictions for deep learning. Cryptology ePrint Archive, Report 2018/758 (2018)Google Scholar
  7. 7.
    Bourse, F., Minelli, M., Minihold, M., Paillier, P.: Fast homomorphic evaluation of deep discretized neural networks. In: Shacham, H., Boldyreva, A. (eds.) CRYPTO 2018. LNCS, vol. 10993, pp. 483–512. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-96878-0_17CrossRefGoogle Scholar
  8. 8.
    Brakerski, Z.: Fully homomorphic encryption without modulus switching from classical GapSVP. In: Safavi-Naini, R., Canetti, R. (eds.) CRYPTO 2012. LNCS, vol. 7417, pp. 868–886. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-32009-5_50CrossRefGoogle Scholar
  9. 9.
    Brakerski, Z., Gentry, C., Vaikuntanathan, V.: (Leveled) fully homomorphic encryption without bootstrapping. In: ITCS 2012, pp. 309–325. ACM (2012)Google Scholar
  10. 10.
    Carpov, S., Gama, N., Georgieva, M., Troncoso-Pastoriza, J.R.: Privacy-preserving semi-parallel logistic regression training with fully homomorphic encryption. Cryptology ePrint Archive, Report 2019/101 (2019). https://eprint.iacr.org/2019/101
  11. 11.
    Chabanne, H., de Wargny, A., Milgram, J., Morel, C., Prouff, E.: Privacy-preserving classification on deep neural network. Cryptology ePrint Archive, Report 2017/035 (2017). https://eprint.iacr.org/2017/035
  12. 12.
    Chen, H., Laine, K., Player, R.: Simple encrypted arithmetic library - SEAL v2.1. In: Brenner, M., et al. (eds.) FC 2017. LNCS, vol. 10323, pp. 3–18. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-70278-0_1CrossRefGoogle Scholar
  13. 13.
    Cheney, N., Schrimpf, M., Kreiman, G.: On the robustness of convolutional neural networks to internal architecture and weight perturbations. CoRR, abs/1703.08245 (2017)Google Scholar
  14. 14.
    Cheon, J.H., Han, K., Kim, A., Kim, M., Song, Y.: Bootstrapping for approximate homomorphic encryption. In: Nielsen, J.B., Rijmen, V. (eds.) EUROCRYPT 2018. LNCS, vol. 10820, pp. 360–384. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-78381-9_14CrossRefGoogle Scholar
  15. 15.
    Cheon, J.H., Kim, A., Kim, M., Song, Y.: Homomorphic encryption for arithmetic of approximate numbers. In: Takagi, T., Peyrin, T. (eds.) ASIACRYPT 2017. LNCS, vol. 10624, pp. 409–437. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-70694-8_15CrossRefGoogle Scholar
  16. 16.
    Chillotti, I., Gama, N., Georgieva, M., Izabachène, M.: Faster fully homomorphic encryption: bootstrapping in less than 0.1 seconds. In: Cheon, J.H., Takagi, T. (eds.) ASIACRYPT 2016. LNCS, vol. 10031, pp. 3–33. Springer, Heidelberg (2016).  https://doi.org/10.1007/978-3-662-53887-6_1CrossRefzbMATHGoogle Scholar
  17. 17.
    Chillotti, I., Gama, N., Georgieva, M., Izabachène, M.: TFHE: fast fully homomorphic encryption over the torus. Cryptology ePrint Archive, Report 2018/421 (2018). https://eprint.iacr.org/2018/421
  18. 18.
    Ducas, L., Micciancio, D.: FHEW: bootstrapping homomorphic encryption in less than a second. In: Oswald, E., Fischlin, M. (eds.) EUROCRYPT 2015. LNCS, vol. 9056, pp. 617–640. Springer, Heidelberg (2015).  https://doi.org/10.1007/978-3-662-46800-5_24CrossRefzbMATHGoogle Scholar
  19. 19.
    Elson, J., Douceur, J.R., Howell, J., Saul. J.: Asirra: a CAPTCHA that exploits interest-aligned manual image categorization. In: Proceedings of the 2007 ACM Security, CCS 2007, pp. 366–374. ACM (2007)Google Scholar
  20. 20.
    Fan, J., Vercauteren, F.: Somewhat practical fully homomorphic encryption. IACR Cryptology ePrint Archive 2012, 144 (2012)Google Scholar
  21. 21.
    Gentry, C., Sahai, A., Waters, B.: Homomorphic encryption from learning with errors: conceptually-simpler, asymptotically-faster, attribute-based. In: Canetti, R., Garay, J.A. (eds.) CRYPTO 2013. LNCS, vol. 8042, pp. 75–92. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-40041-4_5CrossRefGoogle Scholar
  22. 22.
    Gilad-Bachrach, R., Dowlin, N., Laine, K., Lauter, K.E., Naehrig, M., Wernsing, J.: CryptoNets: applying neural networks to encrypted data with high throughput and accuracy. In: Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, 19–24 June 2016, pp. 201–210 (2016)Google Scholar
  23. 23.
    Gilad-Bachrach, R., Dowlin, N., Laine, K., Lauter, K.E., Naehrig, M., Wernsing, J.: CryptoNets: applying neural networks to encrypted data with high throughput and accuracy. In: ICML 2016. JMLR Workshop and Conference Proceedings, vol. 48, pp. 201–210. JMLR.org (2016)Google Scholar
  24. 24.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR 2016, pp. 770–778. IEEE Computer Society (2016)Google Scholar
  25. 25.
    Jiang, X., Kim, M., Lauter, K.E., Song, Y.: Secure outsourced matrix computation and application to neural networks. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, CCS 2018, Toronto, ON, Canada, 15–19 October 2018, pp. 1209–1222. ACM (2018)Google Scholar
  26. 26.
    King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)Google Scholar
  27. 27.
    Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE, pp. 2278–2324 (1998)CrossRefGoogle Scholar
  28. 28.
    Lecun, Y., Cortes, C., Burges, C.J.: The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist/
  29. 29.
    Lyubashevsky, V., Peikert, C., Regev, O.: On ideal lattices and learning with errors over rings. In: Gilbert, H. (ed.) EUROCRYPT 2010. LNCS, vol. 6110, pp. 1–23. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-13190-5_1CrossRefGoogle Scholar
  30. 30.
    Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. IJCV 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)MathSciNetzbMATHGoogle Scholar
  32. 32.
    Wagh, S., Gupta, D., Chandran, N.: SecureNN: efficient and private neural network training. Cryptology ePrint Archive, Report 2018/442 (2018). https://eprint.iacr.org/2018/442

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Christina Boura
    • 1
    • 4
  • Nicolas Gama
    • 1
    • 2
  • Mariya Georgieva
    • 2
    • 3
    Email author
  • Dimitar Jetchev
    • 2
    • 3
  1. 1.Laboratoire de Mathématiques de Versailles, UVSQ, CNRS, Université Paris-SaclayVersaillesFrance
  2. 2.InpherLausanneSwitzerland
  3. 3.EPFLLausanneSwitzerland
  4. 4.InriaParisFrance

Personalised recommendations