Wireless Networks

, Volume 25, Issue 1, pp 201–213 | Cite as

A hierarchical learning approach to anti-jamming channel selection strategies

  • Fuqiang Yao
  • Luliang JiaEmail author
  • Youming Sun
  • Yuhua Xu
  • Shuo Feng
  • Yonggang Zhu


This paper investigates the channel selection problem for anti-jamming defense in an adversarial environment. In our work, we simultaneously consider malicious jamming and co-channel interference among users, and formulate this anti-jamming defense problem as a Stackelberg game with one leader and multiple followers. Specifically, the users and jammer independently and selfishly select their respective optimal strategies and obtain the optimal channels based on their own utilities. To derive the Stackelberg Equilibrium, a hierarchical learning framework is formulated, and a hierarchical learning algorithm (HLA) is proposed. In addition, the convergence performance of the proposed HLA algorithm is analyzed. Finally, we present simulation results to validate the effectiveness of the proposed algorithm.


Anti-jamming Stackelberg game Channel selection Stochastic learning 



This work was supported in part by the Natural Science Foundation for Distinguished Young Scholars of Jiangsu Province under Grant BK20160034, in part by the National Science Foundation of China under Grant 61631020, Grant 61671473, Grant 61401508, and Grant 61401505, in part by Jiangsu Provincial Natural Science Foundation of China Grant BK20130069, and Grant BK20151450, and in part by the Open Research Foundation of Science and Technology in Communication Networks Laboratory.


  1. 1.
    International Telecommunication Union (2015). Technical and operational principles for HF sky-wave communication stations to improve the man-made noise HF environment(ITU-R 258/5). ITU.Google Scholar
  2. 2.
    Zou, Y., Zhu, J., Wang, X., & Hanzo, L. (2016). A survey on wireless security: Technical challenges, recent advances, and future trends. Proceedings of the IEEE, 104(9), 1727–1765.CrossRefGoogle Scholar
  3. 3.
    Sagduyu, Y. E., Berry, R. A., & Ephremides, A. (2011). Jamming games in wireless networks with incomplete information. IEEE Communications Magazine, 49(8), 112–118.CrossRefGoogle Scholar
  4. 4.
    Chen, C., Song, M., Xin, C., & Backens, J. (2013). A game-theoretical anti-jamming scheme for cognitive radio networks. IEEE Network, 27(3), 22–27.CrossRefGoogle Scholar
  5. 5.
    Zhang, L., Guan, Z., & Melodia, T. (2016). United against the enemy: anti-jamming based on cross-layer cooperation in wireless networks. IEEE Transactions on Wireless Communications, 15(8), 5733–5747.CrossRefGoogle Scholar
  6. 6.
    Zhu, H., Fang, C., Liu, Y., et al. (2016). You can jam but you can’t hide: defending against jamming attacks for Geo-location database driven spectrum sharing. IEEE Journal on Selected Areas in Communications,. doi: 10.1109/JSAC.2016.2605799.Google Scholar
  7. 7.
    Pietro, R. D., & Oligeri, G. (2013). Jamming mitigation in cognitive radio networks. IEEE Network, 27(3), 10–15.CrossRefGoogle Scholar
  8. 8.
    Wu, Y., Wang, B., Liu, K., & Clancy, T. (2012). Anti-jamming games in multi-channel cognitive radio networks. IEEE Journal on Selected Areas in Communications, 30(1), 4–15.CrossRefGoogle Scholar
  9. 9.
    Han, Z., Niyato, D., Saad, W., Basar, T., et al. (2012). Game theory in wireless and communication networks. Cambridge: Cambridge University Press.zbMATHGoogle Scholar
  10. 10.
    Xu, Y., Wang, J., Wu, Q., et al. (2015). A game-theoretic perspective on self-organizing optimization for cognitive small cells. IEEE Communications Magazine, 53(7), 100–108.CrossRefGoogle Scholar
  11. 11.
    Sun, Y., Wu, Q., Wang, J., Xu, Y., & Anpalagan, A. (2016). Veracity: Overlapping coalition formation based double auction for heterogeneous demand and spectrum reusability. IEEE Journal on Selected Areas in Communications, 34(10), 2690–2705.CrossRefGoogle Scholar
  12. 12.
    Shao, H., Sun, Y., Zhao, H., et al. (2016). Locally cooperative traffic-offloading in multi-mode small cell networks via potential games. Transactions on Emerging Telecommunications Technologies, 27(7), 968–981.CrossRefGoogle Scholar
  13. 13.
    Sun, Y., Wang, J., et al. (2016). Local altruistic coalition formation game for spectrum sharing and interference management in hyper-dense cloud-RANs. IET Communications, 10(15), 1914–1921.CrossRefGoogle Scholar
  14. 14.
    Sharma, R. K., & Rawat, D. B. (2015). Advances on security threats and countermeasures for cognitive radio networks: A survey. IEEE Communications Surveys & Tutorials, 17(2), 1023–1043. (Second Quarter) .CrossRefGoogle Scholar
  15. 15.
    Wang, B., Wu, Y., Liu, K., & Clancy, T. (2011). An anti-jamming stochastic game for cognitive radio networks. IEEE Journal on Selected Areas in Communications, 29(4), 877–889.CrossRefGoogle Scholar
  16. 16.
    Li, H., & Han, Z. (2010). Dogfight in spectrum: Combating primary user emulation attacks in cognitive radio systems, part I: Known channel statistics. IEEE Transactions on Wireless Communications, 9(11), 3566–3577.CrossRefGoogle Scholar
  17. 17.
    Oskoui, M. G., Khorramshahi, P., & Salehi, T. A. (2016). Using game theory to battle jammer in control channels of cognitive radio ad hoc networks. In Proceedings of the IEEE ICC (pp. 1–5).Google Scholar
  18. 18.
    El-Bardan, R., Brahma, S., & Varshney, P. K. (2016). Strategic power allocation with incomplete information in the presence of jammer. IEEE Transactions on Communications, 64(8), 3467–3479.CrossRefGoogle Scholar
  19. 19.
    Song, T., Stark, W. E., Li, T., & Tugnait, J. K. (2016). Optimal multiband transmission under hostile jamming. IEEE Transactions on Communications, 64(9), 4013–4027.CrossRefGoogle Scholar
  20. 20.
    Hanawal, M. K., Abdel-Rahman, M. J., & Krunz, M. (2014). Game theoretic anti-jamming dynamic frequency hopping and rate adaptation in wireless systems. In Proceedings of the WiOpt Conference (pp. 247–254).Google Scholar
  21. 21.
    Hanawal, M. K., Abdel-Rahman, M. J., & Krunz, M. (2016). Joint adaptation of frequency hopping and transmission rate for anti-jamming wireless systems. IEEE Transactions on Mobile Computing, 15(9), 2247–2259.CrossRefGoogle Scholar
  22. 22.
    Abdel-Rahman, M. J., & Krunz, M. (2014). Game-theoretic quorum-based frequency hopping for anti-jamming rendezvous in DSA networks. In Proceedings of the IEEE DYSPAN (pp. 248–258).Google Scholar
  23. 23.
    Sun, Y., Wang, J., Sun, F., & Zhang, J. (2016). Energy-aware joint user scheduling and power control for two-tier femtocell networks: A hierarchical game approach. IEEE Systems Journal,. doi: 10.1109/JSYST.2016.2580560.Google Scholar
  24. 24.
    Kang, X., Zhang, R., & Motani, M. (2012). Price-based resource allocation for spectrum-sharing femtocell networks: A Stackelberg game approach. IEEE Journal on Selected Areas in Communications, 30(3), 538–549.CrossRefGoogle Scholar
  25. 25.
    Yang, D., Zhang, J., Fang, X., & Richa, A., et al. (2012). Optimal transmission power control in the presence of a smart jammer. In Proceedings of the IEEE Globecom (pp. 5506–5511).Google Scholar
  26. 26.
    Yang, D., Xue, G., Zhang, J., Richa, A., & Fang, X. (2013). Coping with a smart jammer in wireless networks: A stackelberg game approach. IEEE Transactions on Wireless Communications, 12(8), 4038–4047.CrossRefGoogle Scholar
  27. 27.
    Xiao, L., Chen, T., Liu, J., & Dai, H. (2015). Anti-jamming transmission stackelberg game with observation errors. IEEE Communications Letters, 19(6), 949–952.CrossRefGoogle Scholar
  28. 28.
    Jia, L., Yao, F., Sun, Y., et al. (2016). Bayesian Stackelberg game for anti-jamming with incomplete information. IEEE Communications Letters, 20(10), 1991–1994.CrossRefGoogle Scholar
  29. 29.
    Tang, X., Ren, P., Wang, Y., et al. (2015). Securing wireless transmission against reactive jamming: A Stackelberg game framework. In Proceedings of the IEEE GLOBECOM (pp. 1–6).Google Scholar
  30. 30.
    Xiao, L., Xie, C., Chen, T., et al. (2016). Mobile offloading game against smart attacks. In Proceedings of the IEEE INFOCOM (pp. 403–408).Google Scholar
  31. 31.
    Xiao, L., Xie, C., Chen, T., et al. (2016). A mobile offloading game against smart attacks. IEEE Access, 4, 2281–2291.CrossRefGoogle Scholar
  32. 32.
    Wu, Q., Xu, Y., Wang, J., et al. (2013). Distributed channel selection in time-varying radio environment: Interference mitigation game with uncoupled stochastic learning. IEEE Transactions on Vehicular Technology, 62(9), 4524–4538.CrossRefGoogle Scholar
  33. 33.
    Zheng, J., Cai, Y., Yang, W., et al. (2013). A fully distributed algorithm for dynamic channel adaptation in canonical communication networks. IEEE Wireless Communications Letters, 2(5), 491–494.CrossRefGoogle Scholar
  34. 34.
    Zheng, J., Cai, Y., Xu, Y., & Anpalagan, A. (2014). Distributed channel selection for interference mitigation in dynamic environment: A game theoretic stochastic learning solution. IEEE Transactions on Vehicular Technology, 63(9), 4757–4762.CrossRefGoogle Scholar
  35. 35.
    Chen, X., Zhang, H., Chen, T., & Lasanen, M.: Improving energy efficiency in green femtocell networks: a hierarchical reinforcement learning framework. In Proceedings of the IEEE ICC 2013 (pp. 2241–2245).Google Scholar
  36. 36.
    Sun, Y., Shao, H., Liu, X., et al. (2015). Traffic offloading in two-tier multi-mode small cell networks over unlicensed bands: A hierarchical learning framework. KSII Transactions on Internet and Information Systems, 9(11), 4291–4310.Google Scholar
  37. 37.
    Monderer, D., & Shapley, L. S. (1996). Potential games. Games and Economic Behavior, 14(1), 124–143.MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Sastry, P., Phansalkar, V., & Thathachar, M. (1994). Decentralized learning of Nash equilibria in multi-person stochastic games with incomplete information. IEEE Transactions on Systems, Man, and Cybernetics, 24(5), 769–777.MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Zhong, W., Xu, Y., Tao, M., et al. (2010). Game theoretic multimode precoding strategy selection for MIMO multiple access channels. IEEE Signal Processing Letters, 17(6), 563–566.CrossRefGoogle Scholar
  40. 40.
    Zheng, J., Cai, Y., Lu, N., et al. (2015). Stochastic game-theoretic spectrum access in distributed and dynamic environment. IEEE Transactions on Vehicular Technology, 64(10), 4807–4820.CrossRefGoogle Scholar
  41. 41.
    Xu, Y., Wang, J., Wu, Q., et al. (2012). Opportunistic spectrum access in unknown dynamic environment: A game-theoretic stochastic learning solution. IEEE Transactions on Wireless Communications, 11(4), 1380–1391.CrossRefGoogle Scholar
  42. 42.
    Xu, Y., Xu, Y., & Anpalagan, A. (2015). Database-assisted spectrum access in dynamic networks: A distributed learning solution. IEEE Access, 3, 1071–1078.CrossRefGoogle Scholar
  43. 43.
    Watkins, C. J. C. H., & Dayan, P. (1992). Q-learning. Machine Learning, 8, 279–292.zbMATHGoogle Scholar
  44. 44.
    Kianercy, A., & Galstyan, A. (2012). Dynamics of Boltzmann Q-learning in two-player two-action games. Physical Review E, 85(4), 1–10.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Nanjing Telecommunication Technology InstituteNanjingChina
  2. 2.College of Communication EngineeringPLA University of Science and TechnologyNanjingChina
  3. 3.Science and Technology on Communication Networks LaboratoryShijiazhuangChina
  4. 4.Computational Science and Engineering at McMaster UniversityHamiltonCanada

Personalised recommendations