Skip to main content
SpringerLink
Log in

Privacy-Preserving Machine Learning Using Cryptography

  • Chapter
  • First Online:
Security and Artificial Intelligence

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13049))

Abstract

Data scientists require an extensive training set to train an accurate and reliable machine learning model – the bigger and diverse the training set, the better. However, acquiring such a vast training set can be difficult, especially when sensitive user data is involved. The General Data Protection Regulation (GDPR) and similar regulations may prohibit the gathering and processing of this sensitive data. Privacy-preserving cryptographic protocols and primitives, like secure multi-party computation (MPC) and fully homomorphic encryption (FHE), may provide a solution to this problem. They allow us to perform calculations on private and unknown data and can, therefore, be used to classify and train on GDPR protected data sets. While still considered very inefficient, privacy-preserving machine learning using MPC and FHE has been heavily researched in recent years. In this chapter, we give an introduction to MPC and FHE, how they can be used, their limitations, and describe how state-of-the-art publications apply them to machine learning algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://eur-lex.europa.eu/legal-content/EN/ALL/?uri=celex:32016R0679.

References

  1. Abadi, M., et al.: Tensorflow: large-scale machine learning on heterogeneous distributed systems. CoRR abs/1603.04467 (2016)

    Google Scholar 

  2. Alexandra Institute: FRESCO - a FRamework for Efficient Secure COmputation (2020). https://github.com/aicis/fresco

  3. Aly, A., Keller, M., Rotaru, D., Scholl, P., Smart, N.P., Wood, T.: SCALE-MAMBA (2020). https://homes.esat.kuleuven.be/nsmart/SCALE/

  4. Beaver, D.: Efficient multiparty protocols using circuit randomization. In: Feigenbaum, J. (ed.) CRYPTO 1991. LNCS, vol. 576, pp. 420–432. Springer, Heidelberg (1992). https://doi.org/10.1007/3-540-46766-1_34

    Chapter  Google Scholar 

  5. Beaver, D., Micali, S., Rogaway, P.: The round complexity of secure protocols (extended abstract). In: STOC, pp. 503–513. ACM (1990)

    Google Scholar 

  6. Bergamaschi, F., Halevi, S., Halevi, T.T., Hunt, H.: Homomorphic training of 30,000 logistic regression models. In: Deng, R.H., Gauthier-Umaña, V., Ochoa, M., Yung, M. (eds.) ACNS 2019. LNCS, vol. 11464, pp. 592–611. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-21568-2_29

    Chapter  Google Scholar 

  7. Boemer, F., Cammarota, R., Demmler, D., Schneider, T., Yalame, H.: MP2ML: a mixed-protocol machine learning framework for private inference. In: ARES, pp. 14:1–14:10. ACM (2020)

    Google Scholar 

  8. Boemer, F., Costache, A., Cammarota, R., Wierzynski, C.: nGraph-HE2: a high-throughput framework for neural network inference on encrypted data. In: WAHC@CCS, pp. 45–56. ACM (2019)

    Google Scholar 

  9. Boemer, F., Lao, Y., Cammarota, R., Wierzynski, C.: nGraph-HE: a graph compiler for deep learning on homomorphically encrypted data. In: CF, pp. 3–13. ACM (2019)

    Google Scholar 

  10. Bourse, F., Minelli, M., Minihold, M., Paillier, P.: Fast homomorphic evaluation of deep discretized neural networks. In: Shacham, H., Boldyreva, A. (eds.) CRYPTO 2018. LNCS, vol. 10993, pp. 483–512. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-96878-0_17

    Chapter  Google Scholar 

  11. Bourse, F., Sanders, O., Traoré, J.: Improved secure integer comparison via homomorphic encryption. In: Jarecki, S. (ed.) CT-RSA 2020. LNCS, vol. 12006, pp. 391–416. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-40186-3_17

    Chapter  Google Scholar 

  12. Boyd, S.P., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  Google Scholar 

  13. Brakerski, Z.: Fully homomorphic encryption without modulus switching from classical GapSVP. In: Safavi-Naini, R., Canetti, R. (eds.) CRYPTO 2012. LNCS, vol. 7417, pp. 868–886. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32009-5_50

    Chapter  Google Scholar 

  14. Brakerski, Z., Gentry, C., Vaikuntanathan, V.: (Leveled) fully homomorphic encryption without bootstrapping. In: ITCS, pp. 309–325. ACM (2012)

    Google Scholar 

  15. Cheon, J.H., Kim, A., Kim, M., Song, Y.: Homomorphic encryption for arithmetic of approximate numbers. In: Takagi, T., Peyrin, T. (eds.) ASIACRYPT 2017. LNCS, vol. 10624, pp. 409–437. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-70694-8_15

    Chapter  Google Scholar 

  16. Chillotti, I., Gama, N., Georgieva, M., Izabachène, M.: Faster fully homomorphic encryption: bootstrapping in less than 0.1 seconds. In: Cheon, J.H., Takagi, T. (eds.) ASIACRYPT 2016. LNCS, vol. 10031, pp. 3–33. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-662-53887-6_1

    Chapter  MATH  Google Scholar 

  17. Chillotti, I., Gama, N., Georgieva, M., Izabachène, M.: TFHE: fast fully homomorphic encryption library (2016). https://tfhe.github.io/tfhe/

  18. Chollet, F., et al.: Keras (2015). https://keras.io

  19. Cyphers, S., et al.: Intel nGraph: an intermediate representation, compiler, and executor for deep learning. CoRR abs/1801.08058 (2018)

    Google Scholar 

  20. Damgård, I., Keller, M., Larraia, E., Pastro, V., Scholl, P., Smart, N.P.: Practical covertly secure MPC for dishonest majority – Or: breaking the SPDZ limits. In: Crampton, J., Jajodia, S., Mayes, K. (eds.) ESORICS 2013. LNCS, vol. 8134, pp. 1–18. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40203-6_1

    Chapter  Google Scholar 

  21. Damgård, I., Pastro, V., Smart, N., Zakarias, S.: Multiparty computation from somewhat homomorphic encryption. In: Safavi-Naini, R., Canetti, R. (eds.) CRYPTO 2012. LNCS, vol. 7417, pp. 643–662. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32009-5_38

    Chapter  Google Scholar 

  22. Dathathri, R., et al.: CHET: an optimizing compiler for fully-homomorphic neural-network inferencing. In: PLDI, pp. 142–156. ACM (2019)

    Google Scholar 

  23. Dwork, C.: Differential privacy. In: Bugliesi, M., Preneel, B., Sassone, V., Wegener, I. (eds.) ICALP 2006. LNCS, vol. 4052, pp. 1–12. Springer, Heidelberg (2006). https://doi.org/10.1007/11787006_1

    Chapter  Google Scholar 

  24. Escudero, D., Ghosh, S., Keller, M., Rachuri, R., Scholl, P.: Improved primitives for MPC over mixed arithmetic-binary circuits. In: Micciancio, D., Ristenpart, T. (eds.) CRYPTO 2020. LNCS, vol. 12171, pp. 823–852. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-56880-1_29

    Chapter  Google Scholar 

  25. Even, S., Goldreich, O., Lempel, A.: A randomized protocol for signing contracts. Commun. ACM 28(6), 637–647 (1985)

    Article  MathSciNet  Google Scholar 

  26. Fan, J., Vercauteren, F.: Somewhat practical fully homomorphic encryption. IACR Cryptol. ePrint Arch. 2012, 144 (2012)

    Google Scholar 

  27. Gentry, C.: Fully homomorphic encryption using ideal lattices. In: STOC, pp. 169–178. ACM (2009)

    Google Scholar 

  28. Gilad-Bachrach, R., Dowlin, N., Laine, K., Lauter, K.E., Naehrig, M., Wernsing, J.: CryptoNets: applying neural networks to encrypted data with high throughput and accuracy. In: ICML. JMLR Workshop and Conference Proceedings, vol. 48, pp. 201–210. JMLR.org (2016)

    Google Scholar 

  29. Halevi, S., Shoup, V.: Design and implementation of a homomorphicencryption library (2013). https://github.com/homenc/HElib

  30. Halevi, S., Shoup, V.: Algorithms in HElib. In: Garay, J.A., Gennaro, R. (eds.) CRYPTO 2014. LNCS, vol. 8616, pp. 554–571. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-44371-2_31

    Chapter  MATH  Google Scholar 

  31. Hesamifard, E., Takabi, H., Ghasemi, M.: Deep neural networks classification over encrypted data. In: CODASPY, pp. 97–108. ACM (2019)

    Google Scholar 

  32. Iandola, F.N., Moskewicz, M.W., Ashraf, K., Han, S., Dally, W.J., Keutzer, K.: SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and \(<\)1mb model size. CoRR abs/1602.07360 (2016)

    Google Scholar 

  33. Jean, J.: TikZ for Cryptographers. (2016) https://www.iacr.org/authors/tikz/

  34. Jiang, X., Kim, M., Lauter, K.E., Song, Y.: Secure outsourced matrix computation and application to neural networks. In: CCS, pp. 1209–1222. ACM (2018)

    Google Scholar 

  35. Juvekar, C., Vaikuntanathan, V., Chandrakasan, A.: GAZELLE: a low latency framework for secure neural network inference. In: USENIX, pp. 1651–1669. USENIX Association (2018)

    Google Scholar 

  36. Kales, D., Rechberger, C., Schneider, T., Senker, M., Weinert, C.: Mobile private contact discovery at scale. In: USENIX. pp. 1447–1464. USENIX Association (2019)

    Google Scholar 

  37. Keller, M.: MP-SPDZ: a versatile framework for multi-party computation. In: CCS, pp. 1575–1590. ACM (2020)

    Google Scholar 

  38. Kilian, J.: Founding cryptography on oblivious transfer. In: STOC, pp. 20–31. ACM (1988)

    Google Scholar 

  39. Kim, A., Song, Y., Kim, M., Lee, K., Cheon, J.H.: Logistic regression model training based on the approximate homomorphic encryption. IACR Cryptol. ePrint Arch. 2018, 254 (2018)

    Google Scholar 

  40. Kolesnikov, V., Schneider, T.: Improved garbled circuit: free XOR gates and applications. In: Aceto, L., Damgård, I., Goldberg, L.A., Halldórsson, M.M., Ingólfsdóttir, A., Walukiewicz, I. (eds.) ICALP 2008. LNCS, vol. 5126, pp. 486–498. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-70583-3_40

    Chapter  MATH  Google Scholar 

  41. Krizhevsky, A.: The CIFAR-10 Dataset (2009). http://www.cs.toronto.edu/kriz/cifar.html

  42. LeCun, Y., Cortes, C., Burges, C.: The MNIST Database of Handwritten Digits (2009). http://yann.lecun.com/exdb/mnist/

  43. Liu, B., Ding, M., Shaham, S., Rahayu, W., Farokhi, F., Lin, Z.: When machine learning meets privacy: a survey and outlook. CoRR abs/2011.11819 (2020)

    Google Scholar 

  44. Lyubashevsky, V., Peikert, C., Regev, O.: On ideal lattices and learning with errors over rings. In: Gilbert, H. (ed.) EUROCRYPT 2010. LNCS, vol. 6110, pp. 1–23. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13190-5_1

    Chapter  Google Scholar 

  45. Mohassel, P., Rindal, P.: Aby\(^{3}\): a mixed protocol framework for machine learning. In: CCS, pp. 35–52. ACM (2018)

    Google Scholar 

  46. Naor, M., Pinkas, B., Sumner, R.: Privacy preserving auctions and mechanism design. In: EC, pp. 129–139. ACM (1999)

    Google Scholar 

  47. Paillier, P.: Public-key cryptosystems based on composite degree residuosity classes. In: Stern, J. (ed.) EUROCRYPT 1999. LNCS, vol. 1592, pp. 223–238. Springer, Heidelberg (1999). https://doi.org/10.1007/3-540-48910-X_16

    Chapter  Google Scholar 

  48. Polyakov, Y., Rohloff, K., Ryan, G., Cousins, D.: Palisade lattice cryptography library (2020). https://palisade-crypto.org/software-library/

  49. Rabin, M.: How to exchange secrets by oblivious transfer. Technical report, TR-81, Aiken Computation Laboratory, Harvard University (1981)

    Google Scholar 

  50. Regev, O.: On lattices, learning with errors, random linear codes, and cryptography. In: STOC, pp. 84–93. ACM (2005)

    Google Scholar 

  51. Riazi, M.S., Samragh, M., Chen, H., Laine, K., Lauter, K.E., Koushanfar, F.: XONN: xnor-based oblivious deep neural network inference. In: USENIX, pp. 1501–1518. USENIX Association (2019)

    Google Scholar 

  52. Rindal, P.: libOTe: an efficient, portable, and easy to use Oblivious Transfer Library. https://github.com/osu-crypto/libOTe

  53. Rivest, R.L., Shamir, A., Adleman, L.M.: A method for obtaining digital signatures and public-key cryptosystems. Commun. ACM 21(2), 120–126 (1978)

    Article  MathSciNet  Google Scholar 

  54. Romero, A., Ballas, N., Kahou, S.E., Chassang, A., Gatta, C., Bengio, Y.: FitNets: hints for thin deep nets. In: ICLR (Poster) (2015)

    Google Scholar 

  55. Microsoft SEAL (release 3.6) (2020). https://github.com/Microsoft/SEAL. Microsoft Research, Redmond, WA

  56. Shamir, A.: How to share a secret. Commun. ACM 22(11), 612–613 (1979)

    Article  MathSciNet  Google Scholar 

  57. Songhori, E.M., Hussain, S.U., Sadeghi, A., Schneider, T., Koushanfar, F.: TinyGarble: highly compressed and scalable sequential garbled circuits. In: IEEE S&P, pp. 411–428. IEEE (2015)

    Google Scholar 

  58. University of Bristol: Multi-Protocol SPDZ (2020). https://github.com/data61/MP-SPDZ

  59. Wagh, S., Gupta, D., Chandran, N.: SecureNN: 3-party secure computation for neural network training. Proc. Priv. Enhancing Technol. 2019(3), 26–49 (2019)

    Article  Google Scholar 

  60. Wang, X., Malozemoff, A.J., Katz, J.: EMP-toolkit: efficient MultiParty computation toolkit (2016). https://github.com/emp-toolkit

  61. Yao, A.C.: How to generate and exchange secrets (extended abstract). In: FOCS, pp. 162–167. IEEE (1986)

    Google Scholar 

  62. Zahur, S., Rosulek, M., Evans, D.: Two halves make a whole. Reducing data transfer in garbled circuits using half gates. In: Oswald, E., Fischlin, M. (eds.) EUROCRYPT 2015. LNCS, vol. 9057, pp. 220–250. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-662-46803-6_8

    Chapter  MATH  Google Scholar 

  63. Zheng, W., Popa, R.A., Gonzalez, J.E., Stoica, I.: Helen: maliciously secure coopetitive learning for linear models. In: IEEE S&P, pp. 724–738. IEEE (2019)

    Google Scholar 

Download references

Acknowledgments

This work was supported by the “DDAI" COMET Module within the COMET – Competence Centers for Excellent Technologies Programme, funded by the Austrian Federal Ministry for Transport, Innovation and Technology (bmvit), the Austrian Federal Ministry for Digital and Economic Affairs (bmdw), the Austrian Research Promotion Agency (FFG), the province of Styria (SFG) and partners from industry and academia. The COMET Programme is managed by FFG.

Author information

Authors and Affiliations

  1. Graz University of Technology, Graz, Austria

    Christian Rechberger & Roman Walch

  2. Know-Center GmbH, Graz, Austria

    Roman Walch

Authors
  1. Christian Rechberger
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Roman Walch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roman Walch .

Editor information

Editors and Affiliations

  1. Radboud University Nijmegen, Nijmegen, The Netherlands

    Lejla Batina

  2. Leiden University, Leiden, The Netherlands

    Thomas Bäck

  3. Radboud University Nijmegen, Nijmegen, The Netherlands

    Ileana Buhan

  4. Radboud University Nijmegen, Nijmegen, The Netherlands

    Stjepan Picek

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Rechberger, C., Walch, R. (2022). Privacy-Preserving Machine Learning Using Cryptography. In: Batina, L., Bäck, T., Buhan, I., Picek, S. (eds) Security and Artificial Intelligence. Lecture Notes in Computer Science, vol 13049. Springer, Cham. https://doi.org/10.1007/978-3-030-98795-4_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-98795-4_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-98794-7

  • Online ISBN: 978-3-030-98795-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation