Abstract
Due to the rapid development of the cloud computing environment, it is widely accepted that cloud servers are important for users to improve work efficiency. Users need to know servers’ capabilities and make optimal decisions on selecting the best available servers for users’ tasks. We consider the process that users learn servers’ capabilities as a multi-agent Reinforcement learning process. The learning speed and efficiency in Reinforcement learning can be improved by transferring the learning experience among learning agents which is defined as advising. However, existing advising frameworks are limited by a requirement during experience transfer, which all learning agents in a Reinforcement learning environment must have the completely same available choices, also called actions. To address the above limit, this paper proposes a novel differential privacy agent advising approach in Reinforcement learning. Our proposed approach can significantly improve the conventional advising frameworks’ application when agents’ choices are not the completely same. The approach can also speed up the Reinforcement learning by the increase of possibility of experience transfer among agents with different available choices.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Amir, O., Kamar, E., Kolobov, A., Grosz, B.: Interactive teaching strategies for agent training (2016)
Clouse, J.A., Utgoff, P.E.: A teaching method for Reinforcement learning, pp. 92–110 (1992)
da Silva, F., Glatt, R., Costa, A.: Simultaneously learning and advising in multiagent Reinforcement learning. In: Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems, pp. 1100–1108 (2017)
David, M., et al.: Distraction becomes engagement in automated driving. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 59, 1676–1680 (2015)
Dwork, C.: A firm foundation for private data analysis. Commun. ACM 54, 86–95 (2011)
Dwork, C., McSherry, F., Nissim, K., Smith, A.: Calibrating noise to sensitivity in private data analysis. In: Halevi, S., Rabin, T. (eds.) TCC 2006. LNCS, vol. 3876, pp. 265–284. Springer, Heidelberg (2006). https://doi.org/10.1007/11681878_14
Clouse, J.A.: Learning from an automated training agent. In: Adaptation and Learning in Multiagent Systems (1996)
Littman, M.: Reinforcement learning improves behaviour from evaluative feedback. Nature 521, 445–451 (2015)
Maclin, R., Shavlik, J.W.: Creating advice-taking reinforcement learners. Mach. Learn. 22, 251–281 (1996)
Matthew, E.T., Nicholas, C., Anestis, F., Ioannis, V., Lisa, T.: Reinforcement learning agents providing advice in complex video games. Connect. Sci. 26, 45–63 (2014)
Nunes, L., Oliveira, E.: On learning by exchanging advice. arXiv preprint cs/0203010 (2002)
Sun, N., Zhang, J., Rimba, P., Gao, S., Zhang, Y., Xiang, Y.: Data-driven cybersecurity incident prediction: a survey. IEEE Commun. Surv. Tutor. 21, 1744–1772 (2018)
Torrey, L., Walker, T., Shavlik, J., Maclin, R.: Using advice to transfer knowledge acquired in one Reinforcement learning task to another. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 412–424. Springer, Heidelberg (2005). https://doi.org/10.1007/11564096_40
Torrey, L., Taylor, M.: Teaching on a budget: agents advising agents in Reinforcement learning. In: Proceedings of the 2013 International Conference on Autonomous Agents and Multi-agent Systems, pp. 1053–1060 (2013)
Ye, D., He, Q., Wang, Y., Yang, Y.: An agent-based integrated self-evolving service composition approach in networked environments. IEEE Trans. Serv. Comput. 12(6) (2019)
Ye, D., Zhang, M., Vasilakos, A.V.: A survey of self-organization mechanisms in multiagent systems. IEEE Trans. Syst. Man Cybern. Syst. 47(3), 441–461 (2016)
Ye, D., Zhu, T., Zhou, W., Yu, P.: Differentially private malicious agent avoidance in multiagent advising learning. IEEE Trans. Cybern. (2019)
Ye, D., Zhang, M., Sutanto, D.: Cloning, resource exchange, and relationadaptation: an integrative self-organisation mechanism in a distributed agent network. IEEE Trans. Parallel Distrib. Syst. 25(4), 887–897 (2013)
Zhu, T., Li, G., Zhou, W., Yu, P.: Differentially private data publishing and analysis: a survey. IEEE Trans. Knowl. Data Eng. 29, 1619–1638 (2017)
Zhu, T., Xiong, P., Li, G., Zhou, W., Yu, P.: Differentially private model publishing in cyber physical systems. Future Gener. Comput. Syst. (2018)
Zimmer, M., Viappiani, P., Weng, P.: Teacher-student framework: a Reinforcement learning approach. In: AAMAS Workshop Autonomous Robots and Multirobot Systems (2014)
Ackowledgement
This work is supported by an ARC Linkage Project (DP190100981) from Australian Research Council, Australia.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Shen, S., Zhu, T., Ye, D., Yang, M., Liao, T., Zhou, W. (2020). Simultaneously Advising via Differential Privacy in Cloud Servers Environment. In: Wen, S., Zomaya, A., Yang, L. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2019. Lecture Notes in Computer Science(), vol 11944. Springer, Cham. https://doi.org/10.1007/978-3-030-38991-8_36
Download citation
DOI: https://doi.org/10.1007/978-3-030-38991-8_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-38990-1
Online ISBN: 978-3-030-38991-8
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)