Advertisement

APDL: A Practical Privacy-Preserving Deep Learning Model for Smart Devices

  • Xindi Ma
  • Jianfeng Ma
  • Sheng Gao
  • Qingsong Yao
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 747)

Abstract

With the development of sensors on smart devices, many applications usually learn an accurate model based on the collected sensors’ data to provide new services for users. However, the collection of data from users presents obvious privacy issues. Once the companies gather the data, they will keep it forever and the users from whom the data is collected can neither delete it nor control how it will be used.

In this paper, we design, implement, and evaluate a practical privacy-preserving deep learning model that enables multiple participants to jointly learn an accurate model for a given objective. We introduce a light-weight data sanitized mechanism based on differential privacy to perturb participant’s local training data. After that, the service provider will collect all participants’ sanitized data to learn a global accurate model. This offers an attractive point: participants preserve the privacy of their respective data while still benefitting from other participants’ data. Finally, we theoretically prove that our APDL can achieves the \(\varepsilon \)-differential privacy and the evaluation results over a real-word dataset demonstrate that our APDL can perturb participant data effectively.

Notes

Acknowledgment

This work was supported by the National Natural Science Foundation of China (Grant Nos. U1405255, 61672413, 61602537, 61602357, 61303221), National High Technology Research and Development Program (863 Program) (Grant Nos. 2015AA016007), Shaanxi Science & Technology Coordination & Innovation Project (Grant No. 2016TZC-G-6-3), Shaanxi Provincial Natural Science Foundation (Grant Nos. 2015JQ6227, 2016JM6005), China 111 Project (Grant No. B16037), Beijing Municipal Social Science Foundation (Grant No. 16XCC023).

References

  1. 1.
    Hannun, A.Y., Case, C., Casper, J., Catanzaro, B., Diamos, G., Elsen, E., Prenger, R., Satheesh, S., Sengupta, S., Coates, A., Ng, A.Y.: Deep speech: scaling up end-to-end speech recognition, CoRR, vol. abs/1412.5567 (2014)Google Scholar
  2. 2.
    He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: 2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, 7–13 December 2015, pp. 1026–1034 (2015)Google Scholar
  3. 3.
    Shokri, R., Shmatikov, V.: Privacy-preserving deep learning. In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, Denver, CO, USA, 6–12 October 2015, pp. 1310–1321 (2015)Google Scholar
  4. 4.
    Shultz, D.: When your voice betrays you (2015)Google Scholar
  5. 5.
    Bengio, Y.: Learning deep architectures for AI. Found. Trends Mach. Learn. 2(1), 1–127 (2009)CrossRefzbMATHGoogle Scholar
  6. 6.
    Deng, L.: A tutorial survey of architectures, algorithms, and applications for deep learning. APSIPA Trans. Sig. Inf. Process. 3, 1–29 (2014)Google Scholar
  7. 7.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017)CrossRefGoogle Scholar
  8. 8.
    Graves, A., Mohamed, A., Hinton, G.E.: Speech recognition with deep recurrent neural networks. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2013, Vancouver, BC, Canada, 26–31 May 2013, pp. 6645–6649 (2013)Google Scholar
  9. 9.
    Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: DeepFace: closing the gap to human-level performance in face verification. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, Columbus, OH, USA, 23–28 June 2014, pp. 1701–1708 (2014)Google Scholar
  10. 10.
    Xiong, H.Y., Alipanahi, B., Lee, L.J., Bretschneider, H., Merico, D., Yuen, R.K., Hua, Y., Gueroussov, S., Najafabadi, H.S., Hughes, T.R., et al.: The human splicing code reveals new insights into the genetic determinants of disease. Science 347(6218), 1254806 (2015)CrossRefGoogle Scholar
  11. 11.
    Fakoor, R., Ladhak, F., Nazi, A., Huber, M.: Using deep learning to enhance cancer diagnosis and classification. In: Proceedings of the International Conference on Machine Learning (2013)Google Scholar
  12. 12.
    Liang, M., Li, Z., Chen, T., Zeng, J.: Integrative data analysis of multi-platform cancer data with a multimodal deep learning approach. IEEE/ACM Trans. Comput. Biol. Bioinform. 12(4), 928–937 (2015)CrossRefGoogle Scholar
  13. 13.
    Jiang, Q., Zeadally, S., Ma, J., He, D.: Lightweight three-factor authentication and key agreement protocol for internet-integrated wireless sensor networks. IEEE Access 5, 3376–3392 (2017)CrossRefGoogle Scholar
  14. 14.
    Jiang, Q., Ma, J., Yang, C., Ma, X., Shen, J., Chaudhry, S.A.: Efficient end-to-end authentication protocol for wearable health monitoring systems. Comput. Electr. Eng. 63, 182–195 (2017)CrossRefGoogle Scholar
  15. 15.
    Ma, X., Ma, J., Li, H., Jiang, Q., Gao, S.: ARMOR: a trust-based privacy-preserving framework for decentralized friend recommendation in online social networks. Fut. Gener. Comput. Syst. 79, 82–94 (2018)CrossRefGoogle Scholar
  16. 16.
    Gao, S., Ma, J., Sun, C., Li, X.: Balancing trajectory privacy and data utility using a personalized anonymization model. J. Netw. Comput. Appl. 38, 125–134 (2014)CrossRefGoogle Scholar
  17. 17.
    Gao, S., Ma, J., Shi, W., Zhan, G., Sun, C.: TrPF: a trajectory privacy-preserving framework for participatory sensing. IEEE Trans. Inf. Forensics Secur. 8(6), 874–887 (2013)CrossRefGoogle Scholar
  18. 18.
    Ma, X., Li, H., Ma, J., Jiang, Q., Gao, S., Xi, N., Lu, D.: APPLET: a privacy-preserving framework for location-aware recommender system. Sci. Chin. Inf. Sci. 60(9), 092101 (2017)CrossRefGoogle Scholar
  19. 19.
    Ma, X., Ma, J., Li, H., Jiang, Q., Gao, S.: AGENT: an adaptive geo-indistinguishable mechanism for continuous location-based service. Peer-to-Peer Netw. Appl. 11(3), 473–485 (2017)CrossRefGoogle Scholar
  20. 20.
    Gao, S., Ma, X., Zhu, J., Ma, J.: APRS: a privacy-preserving location-aware recommender system based on differentially private histogram. Sci. Chin. Inf. Sci. 60(11), 119103 (2017)CrossRefGoogle Scholar
  21. 21.
    Fu, Z., Huang, F., Sun, X., Vasilakos, A., Yang, C.-N.: Enabling semantic search based on conceptual graphs over encrypted outsourced data. IEEE Trans. Serv. Comput. 12(8), 1874–1884 (2016)Google Scholar
  22. 22.
    Fu, Z., Wu, X., Guan, C., Sun, X., Ren, K.: Toward efficient multi-keyword fuzzy search over encrypted outsourced data with accuracy improvement. IEEE Trans. Inf. Forensics Secur. 11(12), 2706–2716 (2016)CrossRefGoogle Scholar
  23. 23.
    Zhang, Q., Yang, L.T., Chen, Z.: Privacy preserving deep computation model on cloud for big data feature learning. IEEE Trans. Comput. 65(5), 1351–1362 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Mohassel, P., Zhang, Y.: SecureML: a system for scalable privacy-preserving machine learning. In: 2017 IEEE Symposium on Security and Privacy, SP 2017, San Jose, CA, USA, 22–26 May 2017, pp. 19–38 (2017)Google Scholar
  25. 25.
    Li, P., Li, J., Huang, Z., Li, T., Gao, C., Yiu, S., Chen, K.: Multi-key privacy-preserving deep learning in cloud computing. Future Gener. Compt. Syst. 74, 76–85 (2017)CrossRefGoogle Scholar
  26. 26.
    Gilad-Bachrach, R., Dowlin, N., Laine, K., Lauter, K.E., Naehrig, M., Wernsing, J.: CryptoNets: applying neural networks to encrypted data with high throughput and accuracy. In: Proceedings of the 33rd International Conference on Machine Learning, ICML, New York City, NY, USA, pp. 201–210 (2016)Google Scholar
  27. 27.
    Bost, R., Popa, R.A., Tu, S., Goldwasser, S.: Machine learning classification over encrypted data. In: 22nd Annual Network and Distributed System Security Symposium, NDSS, San Diego, California, USA (2015)Google Scholar
  28. 28.
    Abadi, M., Chu, A., Goodfellow, I., McMahan, H.B., Mironov, I., Talwar, K., Zhang, L.: Deep learning with differential privacy. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 308–318. ACM (2016)Google Scholar
  29. 29.
    Phan, N., Wu, X., Hu, H., Dou, D.: Adaptive laplace mechanism: differential privacy preservation in deep learning, CoRR, vol. abs/1709.05750 (2017)Google Scholar
  30. 30.
    Dwork, C.: Differential privacy. In: Bugliesi, M., Preneel, B., Sassone, V., Wegener, I. (eds.) ICALP 2006. LNCS, vol. 4052, pp. 1–12. Springer, Heidelberg (2006).  https://doi.org/10.1007/11787006_1CrossRefGoogle Scholar
  31. 31.
    Dwork, C., McSherry, F., Nissim, K., Smith, A.: Calibrating noise to sensitivity in private data analysis. In: Halevi, S., Rabin, T. (eds.) TCC 2006. LNCS, vol. 3876, pp. 265–284. Springer, Heidelberg (2006).  https://doi.org/10.1007/11681878_14CrossRefGoogle Scholar
  32. 32.
    McSherry, F., Talwar, K.: Mechanism design via differential privacy. In: Proceedings of the 48th Annual IEEE Symposium on Foundations of Computer Science FOCS, Providence, RI, USA, pp. 94–103 (2007)Google Scholar
  33. 33.
    Murphy, K.P.: Machine Learning - A Probabilistic Perspective. Adaptive Computation and Machine Learning Series. MIT Press, Cambridge (2012)zbMATHGoogle Scholar
  34. 34.
    Shen, Y., Jin, H.: EpicRec: towards practical differentially private framework for personalized recommendation. In: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria, pp. 180–191 (2016)Google Scholar
  35. 35.
    Shen, Y., Jin, H.: Privacy-preserving personalized recommendation: an instance-based approach via differential privacy. In: IEEE International Conference on Data Mining, ICDM, Shenzhen, China, pp. 540–549 (2014)Google Scholar
  36. 36.
    Hay, M., Rastogi, V., Miklau, G., Suciu, D.: Boosting the accuracy of differentially private histograms through consistency. PVLDB 3(1), 1021–1032 (2010)Google Scholar
  37. 37.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  38. 38.
    Collobert, R., Kavukcuoglu, K., Farabet, C.: Torch7: a Matlab-like environment for machine learning. In: BigLearn, NIPS Workshop, no. EPFL-CONF-192376 (2011)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  • Xindi Ma
    • 1
  • Jianfeng Ma
    • 1
  • Sheng Gao
    • 2
  • Qingsong Yao
    • 1
  1. 1.School of Cyber EngineeringXidian UniversityXi’anChina
  2. 2.School of InformationCentral University of Finance and EconomicsBeijingChina

Personalised recommendations