Global Q-Learning Approach for Power Allocation in Femtocell Networks

  • Abdulmajeed M. AleneziEmail author
  • Khairi Hamdi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11871)


In dense femtocell network, the complexity of the resource allocation increases significantly as the network becomes denser, which limits the performance of the network. The usage of reinforcement learning to solve the resource allocation problem showed promising results compared to conventional methods. In this work, we use global Q-learning approach on the macro base station to solve the resource allocation problem in a dense and complex network. We propose a new reward function that can be implemented on a centralized Q-learning and achieve good results in terms of maintaining the quality of service for the macro user and maximizing the sum capacity of the femtocell users. In comparison to other reward functions, the proposed reward function maintained both the QoS for the macro user and fairness among all femtocell users.


Femtocell Q-learning Resource allocation 


  1. 1.
    Abdelnasser, A., Hossain, E., Kim, D.I.: Clustering and resource allocation for dense femtocells in a two-tier cellular ofdma network. IEEE Trans. Wirel. Commun. 13(3), 1628–1641 (2014)CrossRefGoogle Scholar
  2. 2.
    Amiri, R., Mehrpouyan, H., Fridman, L., Mallik, R.K., Nallanathan, A., Matolak, D.: A machine learning approach for power allocation in hetnets considering QOS. In: 2018 IEEE International Conference on Communications (ICC), pp. 1–7. IEEE (2018)Google Scholar
  3. 3.
    Bleicher, A.: A surge in small cells [2013 tech to watch]. IEEE Spectr. 50(1), 38–39 (2012)CrossRefGoogle Scholar
  4. 4.
    Galindo-Serrano, A., Giupponi, L.: Distributed q-learning for interference control in OFDMA-based femtocell networks. In: 2010 IEEE 71st Vehicular Technology Conference, pp. 1–5. IEEE (2010)Google Scholar
  5. 5.
    Ghosh, S., De, D., Deb, P.: Energy and spectrum optimization for 5G massive MIMO cognitive femtocell based mobile network using auction game theory. Wirel. Pers. Commun. 106, 1–22 (2019)CrossRefGoogle Scholar
  6. 6.
    Heegard, C.: Range versus rate in IEEE 802.11 G wireless local area networks. In: Proceedings of IEEE, vol. 802, September 2001 Google Scholar
  7. 7.
    Pyun, S.Y., Lee, W., Jo, O.: Uplink resource allocation for interference mitigation in two-tier femtocell networks. Mobile Inf. Syst. 2018, 1–6 (2018)CrossRefGoogle Scholar
  8. 8.
    Raheem, R., Lasebae, A., Aiash, M., Loo, J.: Interference management for co-channel mobile femtocells technology in LTE networks. In: 2016 12th International Conference on Intelligent Environments (IE), pp. 80–87. IEEE (2016)Google Scholar
  9. 9.
    Tefft, J.R., Kirsch, N.J.: A proximity-based q-learning reward function for femtocell networks. In: 2013 IEEE 78th Vehicular Technology Conference (VTC Fall), pp. 1–5. IEEE (2013)Google Scholar
  10. 10.
    Yu, J., Han, S., Li, X.: A robust game-based algorithm for downlink joint resource allocation in hierarchical ofdma femtocell network system. IEEE Trans. Syst. Man Cybern. Syst. 99, 1–11 (2018)Google Scholar
  11. 11.
    Zhang, H., Li, H., Lee, J.H., Dai, H.: Qos-based interference alignment with similarity clustering for efficient subchannel allocation in dense small cell networks. IEEE Trans. Commun. 65(11), 5054–5066 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Electrical and Electronic EngineeringThe University of ManchesterManchesterUK
  2. 2.Islamic University of MedinahMadinahKingdom of Saudi Arabia

Personalised recommendations