Opportunistic Transmission over Randomly Varying Channels

  • Vivek S. Borkar
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5425)


This article briefly surveys a connected body work of the author and his collaborators on opportunistic transmission over a randomly varying channel. Both single user and multi-user scenarios are considered and a reinforcement learning based scheme proposed for both, with provable convergence properties.


Randomly varying channels opportunistic transmission Markov decision processes structural properties reinforcement learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abounadi, J., Bertsekas, D.P., Borkar, V.S.: Learning algorithms for Markov decision processes with average cost. SIAM Journal of Control and Optimization 40(3), 681–698 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Altman, E.: Constrained Markov Decision Processes. Chapman and Hall / CRC, Boca Raton (1999)zbMATHGoogle Scholar
  3. 3.
    Agarwal, M., Borkar, V.S., Karandikar, A.: Structural properties of optimal transmission policies over a randomly varying channel. IEEE Transactions on Automatic Control 53(6), 1476–1491 (2008)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Berry, R.A., Gallager, R.G.: Communication over fading channels with delay constraints. IEEE Trans. on Information Theory 48(5), 1135–1149 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Bertsekas, D.P., Tsitsiklis, J.N.: Neuro-Dynamic Programming. Athena Scientific, Belmont (1996)zbMATHGoogle Scholar
  6. 6.
    Borkar, V.S.: Convex analytic methods in Markov decision processes. In: Shwartz, A., Feinberg, E. (eds.) Handbook of Markov Decision Processes, pp. 347–375. Kluwer Academic, Dordrecht (2002)CrossRefGoogle Scholar
  7. 7.
    Borkar, V.S.: An actor-critic algorithm for constrained Markov decision processes. Systems and Control Letters 54(3), 207–213 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Borkar, V.S.: Cooperative dymanics and Wardrop equilibria. Systems and Control Letters 58(2), 91–93 (to appear, 2009)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Borkar, V.S.: Stochastic Approximation – A Dynamical Systems Viewpoint. Hindustan Publ. Agency/Cambridge Uni. Press, New Delhi/Cambridge (2008)zbMATHGoogle Scholar
  10. 10.
    Djonin, D.V., Krishnamurthy, V.: Structural results on the optimal transmission scheduling policies and costs for correlated sources and channels. In: IEEE Conference on Decision and Control, pp. 3231–3236 (2005)Google Scholar
  11. 11.
    Hirsch, M.W.: Systems of differential equations that are competitive or cooperative II: convergence almost everywhere. SIAM Journal of Mathematical Analysis 16(3), 423–439 (1985)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Powell, W.: Approximate Dynamic Programming. Wiley Interscience, Hoboken (2007)CrossRefzbMATHGoogle Scholar
  13. 13.
    Puterman, M.: Markov Decision Processes. John Wiley, New York (1994)CrossRefzbMATHGoogle Scholar
  14. 14.
    Salodkar, N., Bhorkar, A., Karandikar, N., Borkar, V.S.: An on-line learning algorithm for energy efficient delay constrained scheduling over a fading channel. IEEE Trans. on Selected Areas in Communications 26(4), 732–742 (2008)CrossRefGoogle Scholar
  15. 15.
    Salodkar, N., Karandikar, A., Borkar, V.S.: A stable on-line algorithm for energy efficient multi-user scheduling (in preparation)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Vivek S. Borkar
    • 1
  1. 1.School of Technology and Computer ScienceTata Institute of Fundamental ResearchMumbaiIndia

Personalised recommendations