Abstract
In this paper, we present an algorithm based on the reinforcement learning scheme with the help of Shapley value of game for power allocation in the cognitive radio networks. The goal is to optimize the achievable transmission rates for secondary users and simultaneously to maximize their usefulness in the coalition. A performance measure is formed as a weighted linear function of the probability of the idle channel amongst N cooperating secondary users. Then, the problem is formulated as a semi-Markov decision process with an average cost criterion and reinforcement learning algorithm is developed to an approximate optimal control policy. The proposed scheme is driven by an estimated dynamic model of cognitive radio network learning simultaneously with the use of the Shapley value of games, to form the best coalition. The simulations are provided to compare the effectiveness of the proposed method against other methods under a variety of traffic conditions with some well-known policies.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Akyildiz, I.F., Lee, W.-Y., Vuran, M.C., Mohanty, S.: Next Generation/Dynamic Spectrum Access/Cognitive Radio Wireless Networks: a Survey. Computer Networks (2006)
Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press (2004)
Cover, T.M., El Gamai, A., Salehi, M.: Multiple Access Channels with Arbitrarily Correlated Sources. IEEE Trans. on Information Theory 26(6), 648–657 (1980)
Etkin, R., Tse, D., Wang, H.: Gaussian Interference Channel Capacity to Within One Bit. IEEE Trans. Information Theory 54(12), 5534–5562 (2008)
Fudenberg, D., Tirole, J.: Game Theory. The MIT Press, Cambridge (1991)
Han, T., Kobayashi, K.: A New Achievable Rate Region for the Interference Channel. IEEE Trans. on Information Theory 27(1), 49–60 (1981)
Hu, J., Wellman, M.P.: Nash Q-Learning for General-Sum Stochastic Games. Journal of Machine Learning Research 4, 1039–1069 (2003)
Laneman, J.N., Wornell, G.W.: Distributed Space-Time Coded Protocols for Exploiting Cooperative Diversity in Wireless Networks. IEEE Trans. on Information Theory 49(10), 2415–2425 (2003)
Lasaulce, S., Debbah, M., Altman, E.: Methodologies for Analyzing Equilibria in Wireless Games. IEEE Signal Processing Magazine, Special Issue on Game Theory for Signal Processing 26(5), 41–52 (2009)
Martyna, J.: A Mathematical Framework for the Multidimensional QoS in Cognitive Radio Networks. In: Balandin, S., Koucheryavy, Y., Hu, H. (eds.) NEW2AN/ruSMART 2011. LNCS, vol. 6869, pp. 440–449. Springer, Heidelberg (2011)
Mitola III, J.: The Software Radio Architecture. IEEE Comm. Mag. 33(5), 26–38 (1995)
Shapley, L.S.: Rand Corporation Research Memorandum. Notes on the N-Person Game: Some Variants of the von Neumann-Morgenstern Definition of Solution, RM-812 (1952)
Shapley, L.S.: A Value for N-Person Games. In: Kuhn, H.W., Tucker, A.W. (eds.) Contribution to the Theory of Games. Princeton University Press, Princeton (1953)
Stackelberg, V.H.: Marketform und Gleichgewicht. Oxford University Press, Oxford (1934)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. The MIT Press, Cambridge (1998)
Thomas, R.W., Friend, D.H., DaSilva, L.A., MacKenzie, A.B.: Cognitive Networks: Adaptation and Learning to Achieve End-to-End Performance Objectives. IEEE Communication Magazine 44(12), 51–57 (2006)
Watkins, C.J.C.H.: Learning from Delayed Rewards. Ph.D. Thesis, University of Cambridge, Cambridge, UK (1989)
Yau, K.-L.A., Komisarczuk, P., Teal, P.D.: Reinforcement Learning for Context Awareness and Intelligence in Wireless Networks: Review, New Features and Open Issues. Journal of Network and Computer Applications 35, 253–267 (2012)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Martyna, J. (2012). Power Allocation in Cognitive Radio Networks by the Reinforcement Learning Scheme with the Help of Shapley Value of Games. In: Andreev, S., Balandin, S., Koucheryavy, Y. (eds) Internet of Things, Smart Spaces, and Next Generation Networking. ruSMART NEW2AN 2012 2012. Lecture Notes in Computer Science, vol 7469. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32686-8_29
Download citation
DOI: https://doi.org/10.1007/978-3-642-32686-8_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-32685-1
Online ISBN: 978-3-642-32686-8
eBook Packages: Computer ScienceComputer Science (R0)