Deep Learning Opportunities for Resource Management in Cognitive Radio Networks



This chapter discusses the use of machine and deep learning in spectrum and resource optimisation for the cognitive radio networks. Indeed, modern cognitive radio networks are being developed to employ artificial intelligence strategies for dynamic spectrum sensing and decision-making processes in order to achieve optimum and accurate decisions. Contrary to traditional wireless communications, learning goal-directed behaviour in dynamic and distributed environments like the cognitive radio networks still pose major challenges for the machine learning strategies being developed. The key difficulty lies in insufficient exploration of the state space, which results in agents being unable to learn robust policies. However, through the use of deep architecture such as deep learning and deep reinforcement learning, agents can explore new behaviour and could eventually help the agent solve tasks posed by the environment. The chapter explores and investigates deep architecture being applied in addressing spectrum management problems in the cognitive radio networks. The most integral part of this chapter is the discussion on how these deep architecture are orchestrated or tailored to solve different problems in the cognitive radio networks.


Cognitive radio networks Resource allocation Artificial intelligence Deep learning Machine learning Deep reinforcement learning Deep architecture 


  1. 1.
    E. Meshkova, J. Riihijarvi, A. Achtzehn, P. Mahonen, Exploring simulated annealing and graphical models for optimization in cognitive wireless networks, in Proceedings of the IEEE GLOBECOM (2009), pp. 1–8Google Scholar
  2. 2.
    B. Awoyemi, B. Maharaj, A. Alfa, Optimal resource allocation solutions for heterogeneous cognitive radio networks. Digital Commun. Netw. 3(2), 129–139 (2017). CrossRefGoogle Scholar
  3. 3.
    G.I. Tsiropoulos, O.A. Dobre, M.H. Ahmed, K.E. Baddour, Radio resource allocation techniques for efficient spectrum access in cognitive radio networks. IEEE Commun. Surv. Tutorials 18(1), 824–847 (2016)CrossRefGoogle Scholar
  4. 4.
    G. Soós, D. Ficzere, P. Varga, User group behavioural pattern in a cellular mobile network for 5G use-cases, in NOMS 2020 - 2020 IEEE/IFIP Network Operations and Management Symposium (2020), pp. 1–7Google Scholar
  5. 5.
    L. Dai, R. Jiao, F. Adachi, H.V. Poor, L. Hanzo, Deep learning for wireless communications: an emerging interdisciplinary paradigm. IEEE Commun. Mag., 05 (2020)Google Scholar
  6. 6.
    T. O’Shea, J. Hoydis, An introduction to deep learning for the physical layer. IEEE Trans. Cognitive Commun. Netw. 3(4), pp. 563–575 (2017)CrossRefGoogle Scholar
  7. 7.
    i Scoop, 5G and IoT: the mobile broadband future of IoT (2020).
  8. 8.
    Ericsson, Discussing challenges in the internet of security (2020).
  9. 9.
    B. Blanco-Filgueira, D. García-Lesta, M. Fernández-Sanjurjo, V.M. Brea, P. López, Deep learning-based multiple object visual tracking on embedded system for IoT and mobile edge computing applications. IEEE Int. Things J. 6(3), 5423–5431 (2019)CrossRefGoogle Scholar
  10. 10.
    Y. Bengio, Learning deep architectures for AI. Found. Trends Mach. Learn. 2(01), 1–55 (2009)CrossRefGoogle Scholar
  11. 11.
    Y. Roh, G. Heo, S. Whang, A survey on data collection for machine learning: a Big Data - AI integration perspective. IEEE Trans. Knowl. Data Eng. PP(10) (2019)Google Scholar
  12. 12.
    M.C. Hlophe, A model-based deep learning approach to spectrum management in distributed cognitive radio networks. Ph.D. Dissertation, University of Pretoria (2020)Google Scholar
  13. 13.
    M.C. Hlophe, B.T. Maharaj, Qos provisioning and energy saving scheme for distributed cognitive radio networks using deep learning. J. Commun. Netw. 22(3), 185–204 (2020)CrossRefGoogle Scholar
  14. 14.
    R. Greve, E. Jacobsen, S. Risi, Evolving neural turing machines for reward-based learning, in Proceedings of the Genetic and Evolutionary Computation Conference (2016), pp. 117–124Google Scholar
  15. 15.
    M. Zhang, L. Wang, Y. Feng, H. Yin, A spectrum sensing algorithm for OFDM signal based on deep learning and covariance matrix graph. IEICE Trans. Commun. E101.B(05), 2435–2444 (2018)Google Scholar
  16. 16.
    K. Yang, Z. Huang, X. Wang, X. Li, A blind spectrum sensing method based on deep learning. Sensors 19(10), 2270.
  17. 17.
    M.C. Hlophe, S.B.T. Maharaj, Spectrum occupancy reconstruction in distributed cognitive radio networks using deep learning. IEEE Access 7, 14294–14307 (2019)CrossRefGoogle Scholar
  18. 18.
    S. Liu, K. Hu, W. Ni, Z. Xu, F. Wang, Z. Wan, A cognitive relay network throughput optimization algorithm based on deep reinforcement learning. Wirel. Commun. Mob. Comput. 2019(2731485), 1–8 (2019)Google Scholar
  19. 19.
    H. Li, T. Wei, A. Ren, Q. Zhu, Y. Wang, Deep reinforcement learning: framework, applications, and embedded implementations, in Proceedings of the IEEE/ACM International Conference on Computer-Aided Design (ICCAD) (2017), pp. 847–854Google Scholar
  20. 20.
    W. Zhou, J. Li, Y. Chen, L. Shen, Strategic interaction multi-agent deep reinforcement learning. IEEE Access 8, 119000–119009 (2020)CrossRefGoogle Scholar
  21. 21.
    Y. Gai, B. Krishnamachari, R. Jain, Learning multiuser channel allocations in cognitive radio networks: a combinatorial multi-armed bandit formulation, in 2010 IEEE Symposium on New Frontiers in Dynamic Spectrum (DySPAN) (2010), pp. 1–9Google Scholar
  22. 22.
    Y. Du, Y. Xu, L. Xue, L. Wang, F. Zhang, An energy-efficient cross-layer routing protocol for cognitive radio networks using apprenticeship deep reinforcement learning. Energies 12(07), 2829 (2019)Google Scholar
  23. 23.
    N. Modi, P. Mary, C. Moy, Qos driven channel selection algorithm for cognitive radio network: Multi-user multi-armed bandit approach. IEEE Trans. Cognitive Commun. Netw. 3(1), 49–66 (2017)CrossRefGoogle Scholar
  24. 24.
    Y. Chen, H. Zhou, R. Kong, L. Zhu, H. Mao, Decentralized blind spectrum selection in cognitive radio networks considering handoff cost. Fut. Int. 9(03), 10 (2017)Google Scholar
  25. 25.
    E. Boursier, V. Perchet, SIC-MMAB: synchronisation involves communication in multiplayer multi-armed bandits. CoRR, vol. abs/1809.08151 (2018).
  26. 26.
    V. Mnih, K. Kavukcuoglu, D. Silver, A. Rusu, J. Veness, M. Bellemare, A. Graves, M. Riedmiller, A. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, D. Hassabis, Human-level control through deep reinforcement learning. Nature 518(02), 529–33 (2015)CrossRefGoogle Scholar
  27. 27.
    S. Srinivasan, M. Lanctot, V.F. Zambaldi, J. Pérolat, K. Tuyls, R. Munos, M. Bowling, Actor-critic policy optimization in partially observable multiagent environments. CoRR, abs/1810.09026 (2018).
  28. 28.
    T.D. Kulkarni, K. Narasimhan, A. Saeedi, J.B. Tenenbaum, Hierarchical deep reinforcement learning: integrating temporal abstraction and intrinsic motivation. CoRR, abs/1604.06057 (2016).
  29. 29.
    P. Si, H. Liang, W. Wu, Y. Zhang, Joint resource management in cognitive radio and edge computing based industrial wireless networks, in GLOBECOM 2017 - 2017 IEEE Global Communications Conference (2017), pp. 1–6Google Scholar
  30. 30.
    F.S. Mohammadi, A. Kwasinski, QoE-driven integrated heterogeneous traffic resource allocation based on cooperative learning for 5G cognitive radio networks, in 2018 IEEE 5G World Forum (5GWF), Silicon Valley, CA, USA (2018), pp. 244–249.
  31. 31.
    Z. Yang, K. Merrick, L. Jin, H.A. Abbass, Hierarchical deep reinforcement learning for continuous action control. IEEE Trans. Neural Netw. Learn. Syst. 29(11), 5174–5184 (2018)MathSciNetCrossRefGoogle Scholar

Copyright information

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022

Authors and Affiliations

  1. 1.University of PretoriaPretoriaSouth Africa

Personalised recommendations