Abstract
Heterogeneous networks (HetNets) balance the traffic load and reduce the cost of cell deployment, which is considered as a promising technology in next generation cellular networks. Due to non-convexity characteristics, it is very difficult to obtain the optimal strategy for user association problem. This paper proposes a new framework to ensure the long-term overall network utility under the premise of guaranteeing the quality of service of downlink user equipment in downlink HetNets. At the same time, a distributed optimization algorithm based on multi-user reinforcement learning is proposed. In order to solve the problem of large computational load of big action space, the optimal strategy is obtained by introducing the method of deep Q-network (DQN). Simulation results show that DQN has better performance than Q-learning method.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Huang, Y., Tan, J., Liang, Y.-C.: Wireless big data: transforming heterogeneous networks to smart networks. J. Commun. Inf. Netw. 2(1), 19–32 (2017)
Zhao, N., Liang, Y.-C., Pei, Y.: Dynamic contract design for cooperative wireless networks. In: IEEE Globe Communication Conference (GLOBECOM), Singapore, pp. 1–6 (2017)
Lien, S.Y., Hung, S.C., Chen, K.C., Liang, Y.-C.: Ultra-low-latency ubiquitous connections in heterogeneous cloud radio access networks. IEEE Wirel. Commun. 22(3), 22–31 (2015)
Wu, W., Du, W., Ruan, G.: Network congestion control methods and theory. Int. J. Grid Util. Comput. 6(3/4), 200–206 (2015)
Wang, J., Che, S., Li, Y., Wang, J.: Optimal design of joint network LDPC codes over orthogonal multiple-access relay channels. Int. J. Grid Util. Comput. 7(1), 68–74 (2016)
Bashar, A.: Graphical modelling approach for monitoring and management of telecommunication networks. Int. J. Space Based Situated Comput. 5(2), 65–75 (2015)
Ye, Q., Rong, B., Chen, Y., Al-Shalash, M., Caramanis, C., Andrews, J.G.: User association for load balancing in heterogeneous cellular networks. IEEE Trans. Wirel. Commun. 12(6), 2706–2716 (2013)
Shen, K., Yu, W.: Distributed pricing-based user association for downlink heterogeneous cellular networks. IEEE J. Sel. Areas Commun. 32(6), 1100–1113 (2014)
Bayat, S., Louie, R., Han, Z., Vucetic, B., Li, Y.: Distributed user association and femtocell allocation in heterogeneous wireless networks. IEEE Trans. Commun. 62(8), 3027–3043 (2014)
Elsherif, A., Chen, W.-P., Ito, A., Ding, Z.: Resource allocation and inter-cell interference management for dual-access small cells. IEEE J. Sel. Areas Commun. 33(6), 1082–1096 (2015)
Chen, M., Liew, S.C., Shao, Z., Kai, C.: Markov approximation for combinatorial network optimization. IEEE Trans. Inf. Theor. 59(10), 6301–6327 (2013)
Ktari, M., Mosbah, M., Ahmed, H.K.: Leader election and computation of a spanning tree in dynamic distributed networks using local computations and mobile agents. Int. J. Space Based Situated Comput. 7(2), 57–71 (2017)
Bylykbashi, K., Spaho, E., Barolli, L., Xhafa, F.: Impact of node density and TTL in vehicular delay tolerant networks: performance comparison of different routing protocols. Int. J. Space Based Situated Comput. 7(3), 136–144 (2017)
Zhao, N., Liu, R., Chen, Y., Wu, M., Jiang, Y., Xiong, W., Liu, C.: Contract design for relay incentive mechanism under dual asymmetric information in cooperative networks. Wirel. Netw. https://doi.org/10.1007/s11276-017-1518-x
Zhao, N., Chen, Y., Liu, R., Wu, M., Xiong, W.: Monitoring strategy for relay incentive mechanism in cooperative communications networks. Comput. Electr. Eng. 60, 14–29 (2017)
Al-Jumeily, D., Hussain, A., Fergus, P.: Using adaptive neural networks to provide self-healing autonomic software. Int. J. Space Based Situated Comput. 5(3), 129–140 (2015)
Mardeni, R., Priya, T.: Optimised cost-231 hata models for WiMAX path loss prediction in suburban and open urban environments. Modern Appl. Sci. 4(9), 75–89 (2010)
Tieleman, T., Hinton, G.: Lecture 6.5łRmsProp: divide the gradient by a running average of its recent magnitude. COURSERA: Neural Netw. Mach. Learn. 4(2), 26–31 (2012)
Acknowledgements
This work was supported by the National Natural Science Foundation of China (No. 61501178, No. 61471162), Project Funded by China Postdoctoral Science Foundation (2017M623004), and the Natural Science Foundation of Hubei Province (no. 2018CFB698).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Zhao, N., He, X., Wu, M., Fan, P., Fan, M., Tian, C. (2019). Deep Q-Network for User Association in Heterogeneous Cellular Networks. In: Barolli, L., Javaid, N., Ikeda, M., Takizawa, M. (eds) Complex, Intelligent, and Software Intensive Systems. CISIS 2018. Advances in Intelligent Systems and Computing, vol 772. Springer, Cham. https://doi.org/10.1007/978-3-319-93659-8_35
Download citation
DOI: https://doi.org/10.1007/978-3-319-93659-8_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-93658-1
Online ISBN: 978-3-319-93659-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)