Advertisement

A Recommender System for Efficient Implementation of Privacy Preserving Machine Learning Primitives Based on FHE

  • Imtiyazuddin ShaikEmail author
  • Ajeet Kumar Singh
  • Harika Narumanchi
  • Nitesh Emmadi
  • Rajan Mindigal Alasingara Bhattachar
Conference paper
  • 56 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12161)

Abstract

With the increased dependence on cloud computing, there is growing concern for privacy of data that is stored and processed on third party cloud service providers. Of many solutions that achieve privacy preserving computations, fully homomorphic encryption (FHE) is a promising direction. FHE has several applications that can be used to perform computations on encrypted data without decrypting them. In this paper, we focus on realizing privacy preserving machine learning (PPML) using FHE. Our prime motivation behind choosing PPML is the increased use of machine learning algorithms on end-user’s data for predictions or classification, where privacy of end-user’s data is at stake. Given the importance of PPML and FHE, we formulate a recommender system that enables machine learning experts who are new to cryptography to efficiently realize a machine learning application in privacy preserving manner. We formulate the recommender system as a multi objective multi constraints optimization problem along with a simpler single objective multi constraint optimization problem. We solve this optimization using TOPSIS based on experimental analysis performed on three prominent FHE libraries HElib, SEAL and HEAAN from the PPML perspective. We present the observations on the performance parameters such as elapsed time and memory usage for the primitive machine learning algorithms such as linear regression and logistic regression. We also discuss the technical issues in making the FHE schemes practically deployable and give insights into selection of parameters to efficiently implement PPML algorithms. We observe that our estimates for matrix multiplication and linear regression correlate with the experimental analysis when assessed using an optimizer. The proposed recommendation system can be used in FHE compilers to facilitate optimal implementation of PPML applications.

Keywords

Homomorphic encryption Privacy preserving machine learning Linear regression Logistic regression Secure outsourcing 

Notes

Acknowledgement

We would like to thank the anonymous reviewers for their valuable feedback and comments on improving this paper.

References

  1. 1.
    Trusted execution environment specification (2015). https://globalplatform.org/specs-library/tee-initial-configuration-v1-1/
  2. 2.
    Goldreich, O.: Secure multi-party computation. Manuscript. Preliminary version, 78 (1998)Google Scholar
  3. 3.
    Huang, Y., Evans, D., Katz, J., Malka, L.: Faster secure two-party computation using garbled circuits. In: USENIX Security Symposium, vol. 201, pp. 331–335 (2011)Google Scholar
  4. 4.
    Gentry, C., Boneh, D.: A fully homomorphic encryption scheme. Stanford University (2009)Google Scholar
  5. 5.
    Rivest, R.L., Adleman, L., Dertouzos, M.L.: On data banks and privacy homomorphisms. Found. Secure Comput. 4, 169–180 (1978)MathSciNetGoogle Scholar
  6. 6.
    Technique for order of preference by similarity to ideal solution. https://decision-radar.com/Topsis.html
  7. 7.
    iDash competition (2019). http://www.humangenomeprivacy.org/2019/
  8. 8.
    Kim, M., Lauter, K.: Private genome analysis through homomorphic encryption. BMC Med. Inform. Decis. Mak. 15, S3 (2015)CrossRefGoogle Scholar
  9. 9.
    Brakerski, Z., Gentry, C., Vaikuntanathan, V.: (leveled) fully homomorphic encryption without bootstrapping. ACM Trans. Comput. Theory (TOCT) 6(3), 13 (2014)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Bos, J.W., Lauter, K., Loftus, J., Naehrig, M.: Improved security for a ring-based fully homomorphic encryption scheme. In: Stam, M. (ed.) IMACC 2013. LNCS, vol. 8308, pp. 45–64. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-45239-0_4CrossRefGoogle Scholar
  11. 11.
    Lu, W.-J., Yamada, Y., Sakuma, J.: Privacy-preserving genome-wide association studies on cloud environment using fully homomorphic encryption. BMC Med. Inform. Decis. Mak. 15, S1 (2015)CrossRefGoogle Scholar
  12. 12.
    Zhang, Y., Dai, W., Jiang, X., Xiong, H., Wang, S.: FORESEE: fully outsourced secure genome study based on homomorphic encryption. BMC Med. Inform. Decis. Mak. 15, S5 (2015)CrossRefGoogle Scholar
  13. 13.
    Çetin, G.S., Chen, H., Laine, K., Lauter, K., Rindal, P., Xia, Y.: Private queries on encrypted genomic data. BMC Med. Genom. 10(2), 45 (2017)CrossRefGoogle Scholar
  14. 14.
    Kim, A., Song, Y., Kim, M., Lee, K., Cheon, J.H.: Logistic regression model training based on the approximate homomorphic encryption. BMC Med. Genom. 11(4), 83 (2018)CrossRefGoogle Scholar
  15. 15.
    Chen, H., et al.: Logistic regression over encrypted data from fully homomorphic encryption. BMC Med. Genom. 11(4), 81 (2018)CrossRefGoogle Scholar
  16. 16.
  17. 17.
    Han, K., Hong, S., Cheon, J.H., Park, D.: Efficient logistic regression on large encrypted data. Technical report, Cryptology ePrint Archive, Report 2018/662. https://eprint.iacr.org/2018/662 (2018)
  18. 18.
    Bergamaschi, F., Halevi, S., Halevi, T.T., Hunt, H.: Homomorphic training of 30,000 logistic regression models. In: Deng, R.H., Gauthier-Umaña, V., Ochoa, M., Yung, M. (eds.) ACNS 2019. LNCS, vol. 11464, pp. 592–611. Springer, Cham (2019).  https://doi.org/10.1007/978-3-030-21568-2_29CrossRefGoogle Scholar
  19. 19.
    Kim, M., Song, Y., Li, B., Micciancio, D.: Semi-parallel logistic regression for GWAS on encrypted data (2019)Google Scholar
  20. 20.
    Crawford, J.L.H., Gentry, C., Halevi, S., Platt, D., Shoup, V.: Doing real work with FHE: the case of logistic regression. In: Proceedings of the 6th Workshop on Encrypted Computing & Applied Homomorphic Cryptography, pp. 1–12. ACM (2018)Google Scholar
  21. 21.
    Kim, M., Song, Y., Wang, S., Xia, Y., Jiang, X.: Secure logistic regression based on homomorphic encryption: design and evaluation. JMIR Med. Inform. 6(2), e19 (2018)CrossRefGoogle Scholar
  22. 22.
    Wang, S., et al.: HEALER: homomorphic computation of exact logistic regression for secure rare disease variants analysis in GWAS. Bioinformatics 32(2), 211–218 (2015)Google Scholar
  23. 23.
  24. 24.
    Hesamifard, E., Takabi, H., Ghasemi, M., Jones, C.: Privacy-preserving machine learning in cloud. In: Proceedings of the 2017 on Cloud Computing Security Workshop, pp. 39–43. ACM (2017)Google Scholar
  25. 25.
    Martins, P., Sousa, L., Mariano, A.: A survey on fully homomorphic encryption: an engineering perspective. ACM Comput. Surv. (CSUR) 50(6), 83 (2018)CrossRefGoogle Scholar
  26. 26.
    Hallman, R.A., Diallo, M.H., August, M.A., Graves, C.T.: Homomorphic encryption for secure computation on big data. In: IoTBDS, pp. 340–347 (2018)Google Scholar
  27. 27.
    Bost, R., Popa, R.A., Tu, S., Goldwasser, S.: Machine learning classification over encrypted data. In: NDSS, vol. 4324, p. 4325 (2015)Google Scholar
  28. 28.
    Du, Y., Gustafson, L., Huang, D., Peterson, K.: Implementing ML algorithms with HE. In: MIT Course 6. 857: Computer and Network Security (2017)Google Scholar
  29. 29.
  30. 30.
  31. 31.
    TFHE: fast fully homomorphic encryption over the torus. https://tfhe.github.io/tfhe/
  32. 32.
    nGraph-HE: HE transformer for nGraph. https://github.com/NervanaSystems/he-transformer
  33. 33.
    Dathathri, R., et al.: CHET: compiler and runtime for homomorphic evaluation of tensor programs. arXiv preprint arXiv:1810.00845 (2018)
  34. 34.
    Crockett, E., Peikert, C., Sharp, C.: ALCHEMY: a language and compiler for homomorphic encryption made easy. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pp. 1020–1037 (2018)Google Scholar
  35. 35.
    Crockett, E., Peikert, C.: Functional lattice cryptography. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, CCS 2016, pp. 993–1005. Association for Computing Machinery, New York (2016)Google Scholar
  36. 36.
    Çetin, G.S., Doröz, Y., Sunar, B., Martin, W.J.: Arithmetic using word-wise homomorphic encryption (2016)Google Scholar
  37. 37.
    Sanyal, A., Kusner, M.J., Gascon, A., Kanade, V.: TAPAS: tricks to accelerate (encrypted) prediction as a service. arXiv preprint arXiv:1806.03461 (2018)
  38. 38.
    Chirgwin, R.: IBM’s homomorphic encryption accelerated to run 75 times faster (2018). Retrieved from The Register: https://www.theregister.co.uk/2018/03/08/ibm_faster_homomorphic_encryption
  39. 39.
    Halevi, S., Shoup, V.: Faster homomorphic linear transformations in HElib. In: Shacham, H., Boldyreva, A. (eds.) CRYPTO 2018. LNCS, vol. 10991, pp. 93–120. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-96884-1_4 CrossRefzbMATHGoogle Scholar
  40. 40.
    Homomorphic encryption standard (2019). https://eprint.iacr.org/2019/939.pdf
  41. 41.
    Chialva, D., Dooms, A.: Conditionals in homomorphic encryption and machine learning applications. arXiv preprint arXiv:1810.12380 (2018)
  42. 42.
    Chen, H., Laine, K., Player, R.: Simple encrypted arithmetic library - SEAL v2.1. In: Brenner, M., et al. (eds.) FC 2017. LNCS, vol. 10323, pp. 3–18. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-70278-0_1CrossRefGoogle Scholar
  43. 43.
    Brakerski, Z., Gentry, C., Halevi, S.: Packed ciphertexts in LWE-based homomorphic encryption. In: Kurosawa, K., Hanaoka, G. (eds.) PKC 2013. LNCS, vol. 7778, pp. 1–13. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-36362-7_1CrossRefGoogle Scholar
  44. 44.
    Carpov, S., Sirdey, R.: A compression method for homomorphic ciphertexts. IACR Cryptology ePrint Archive, 2015:1199 (2015)Google Scholar
  45. 45.
    Doröz, Y., Sunar, B., Çetin, G.S.: On-the-fly homomorphic batching/unbatching (2015)Google Scholar
  46. 46.
    Smart, N.P., Vercauteren, F.: Fully homomorphic SIMD operations. Designs Codes Cryptography 71(1), 57–81 (2012).  https://doi.org/10.1007/s10623-012-9720-4CrossRefzbMATHGoogle Scholar
  47. 47.
    Jiang, X., Kim, M., Lauter, K., Song, Y.: Secure outsourced matrix computation and application to neural networks (2018)Google Scholar
  48. 48.
    Halevi, S., Shoup, V.: Bootstrapping for HElib. In: Oswald, E., Fischlin, M. (eds.) EUROCRYPT 2015. LNCS, vol. 9056, pp. 641–670. Springer, Heidelberg (2015).  https://doi.org/10.1007/978-3-662-46800-5_25CrossRefGoogle Scholar
  49. 49.
    Chen, H., Han, K.: Homomorphic lower digits removal and improved FHE bootstrapping. In: Nielsen, J.B., Rijmen, V. (eds.) EUROCRYPT 2018. LNCS, vol. 10820, pp. 315–337. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-78381-9_12CrossRefGoogle Scholar
  50. 50.
    Cheon, J.H., Han, K., Kim, A., Kim, M., Song, Y.: Bootstrapping for approximate homomorphic encryption. In: Nielsen, J.B., Rijmen, V. (eds.) EUROCRYPT 2018. LNCS, vol. 10820, pp. 360–384. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-78381-9_14CrossRefGoogle Scholar
  51. 51.
    Chen, H., Chillotti, I., Song, Y.: Improved bootstrapping for approximate homomorphic encryption. In: Ishai, Y., Rijmen, V. (eds.) EUROCRYPT 2019. LNCS, vol. 11477, pp. 34–54. Springer, Cham (2019).  https://doi.org/10.1007/978-3-030-17656-3_2CrossRefGoogle Scholar
  52. 52.
  53. 53.
    Deb, K.: Multi-objective optimisation using evolutionary algorithms: an introduction. In: Wang, L., Ng, A., Deb, K. (eds.) Multi-objective Evolutionary Optimisation for Product Design and Manufacturing, pp. 3–34. Springer, London (2011).  https://doi.org/10.1007/978-0-85729-652-8_1CrossRefGoogle Scholar
  54. 54.
    Monsef, H., Naghashzadegan, M., Jamali, A., Farmani, R.: Comparison of evolutionary multi objective optimization algorithms in optimum design of water distribution network. Ain Shams Eng. J. 10(1), 103–111 (2019)CrossRefGoogle Scholar
  55. 55.
    Greco, S., Klamroth, K., Knowles, J.D., Rudolph, G.: Understanding complexity in multiobjective optimization (dagstuhl seminar 15031). In: Dagstuhl Reports, vol. 5. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Imtiyazuddin Shaik
    • 1
    Email author
  • Ajeet Kumar Singh
    • 1
  • Harika Narumanchi
    • 1
  • Nitesh Emmadi
    • 1
  • Rajan Mindigal Alasingara Bhattachar
    • 1
  1. 1.Cyber Security and Privacy Research GroupTCS Research and Innovation, Tata Consultancy ServicesHyderabadIndia

Personalised recommendations