Abstract
This paper considers the power control problem in device-to-device (D2D) communication underlaying a cellular network and explores the application of the machine learning (ML) approach in power control for improving the system throughput. Two multi-agent reinforcement learning (MARL) based algorithms are proposed for performing power control of D2D users (DUs): centralized Q-learning algorithm and distributed Q-learning algorithm. In the centralized algorithm, all DU pairs sharing the same RB use a common Q table in the learning process, while in the distributed algorithm each DU pair maintains its own Q table. Simulation results show that both the centralized algorithm and the distributed algorithm can converge to the same optimum Q values, and the distributed algorithm can converge faster than the centralized algorithm. Moreover, both the proposed Q-learning algorithms outperform the random power control algorithm in terms of the system throughput and satisfaction ratio.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Asadi, A., Wang, Q., Mancuso, V.: A survey on device-to-device communication in cellular networks. IEEE Commun. Surv. Tutorials 16(4), 1801–1819 (2014)
Lee, N., Lin, X., Andrews, J.G., Heath, R.W.: Power control for D2D underlaid cellular networks: modeling, algorithms, and analysis. IEEE J. Sel. Areas Commun. 33(1), 1–13 (2015)
Ren, Y., Liu, F., Liu, Z., Wang, C., Ji, Y.: Power control in D2D-based vehicular communication networks. IEEE Trans. Veh. Technol. 64(12), 5547–5562 (2015)
da Silva, J., Fodor, G.: A binary power control scheme for D2D communications. IEEE Wireless Commun. Lett. 4(6), 669–672 (2015)
Sun, P., Shin, K.G., Zhang, H., He, L.: Transmit power control for D2D-underlaid cellular networks based on statistical features. IEEE Trans. Veh. Technol. 66(5), 4110–4119 (2017)
Nie, S., Fan, Z., Zhao, M., Gu, X., Zhang, L.: Q-learning based power control algorithm for D2D communication. In: Proceedings of 2016 IEEE 27th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC 2016), Valencia, Spain, pp. 1–6 (2016)
Khan, M.I., Alam, M.M., Le Moullec, Y., Yaacoub, E.: Cooperative reinforcement learning for adaptive power allocation in device-to-device communication. In: Proceedings of 2018 IEEE 4th World Forum on Internet of Things (WF-IoT), Singapore (2018)
Luo, Y., Shi, Z., Zhou, X., Liu, Q., Yi, Q.: Dynamic resource allocations based on Q-learning for D2D communication in cellular networks. In: Proceedings of 2014 11th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), Chengdu, China, pp. 385–388 (2014)
Lauer, M., Riedmiller, M.: An algorithm for distributed reinforcement learning in cooperative multi-gent aystems. In: Proceedings of the 17th International Conference on Machine Learning, San Francisco, CA, pp. 535–542 (2000)
Jiang, T., Zhao, Q., David, G., Burr, A.G., Clarke, T.: Single-state Q-learning for self-organized radio resource management in dual-hop 5G high capacity density networks. Trans. Emerg. Telecommun. Technol. 27, 1628–1640 (2016). https://doi.org/10.1002/ett.3019
Sutton, R.S., Barto, A.G.: Reinforcement learning: an introduction. IEEE Trans. Neural Networks 9(5), 1054 (1998)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Chen, W., Zheng, J. (2019). A Multi-agent Reinforcement Learning Based Power Control Algorithm for D2D Communication Underlaying Cellular Networks. In: Han, S., Ye, L., Meng, W. (eds) Artificial Intelligence for Communications and Networks. AICON 2019. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 287. Springer, Cham. https://doi.org/10.1007/978-3-030-22971-9_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-22971-9_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-22970-2
Online ISBN: 978-3-030-22971-9
eBook Packages: Computer ScienceComputer Science (R0)