Optimal Control of Data Transmission over a Fluctuating Channel with Unknown State

  • N. A. KuznetsovEmail author
  • D. V. MyasnikovEmail author
  • K. V. SemenikhinEmail author

Abstract—Optimization problem for data packet transmission over a communication channel governed by a hidden Markov process is considered. The transmitter is modeled as a single-channel finite-buffer queuing system with non-stationary Poisson arrivals. The service rate is proportional to the controlled transmission rate with channel-dependent factor. Buffer overflow leads to packet losses, whereas channel state worsening results in lower service rate. The goal of the optimization problem is to minimize average losses under constraint on the transmitter energy consumption. The exact form of the optimal policy is presented for the augmented control problem. Several control policies with incomplete information are proposed on the basis of the optimal control and hidden state estimates. We consider two estimates based on the optimal filtering equations and the current queue state. Results of computer simulation are presented to compare the control policies under consideration.


queuing system data transmission optimal control augmented problem control with incomplete information filtering estimate 



This work was supported by the Russian Science Foundation (project no. 16-11-00063).


  1. 1.
    A. V. Borisov, “Monitoring remote server accessibility: the optimal filtering approach,” Informatics and Applications 8 (3), 53–69 (2014).Google Scholar
  2. 2.
    A. V. Borisov, A. V. Bosov, and G. B. Miller, “Modeling and monitoring of VoIP connection,” Informatics and Applications 10 (2), 2–13 (2016).Google Scholar
  3. 3.
    M. Welzl, Network Congestion Control: Managing Internet Traffic (Wiley, New York, 2005).CrossRefGoogle Scholar
  4. 4.
    B. M. Miller, K. E. Avrachenkov, K. V. Stepanyan, and G. B. Miller, “Flow control as a stochastic optimal control problem with incomplete information,” Problems of Information Transmission 41 (2), 150–170 (2005).MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    B. M. Miller, G. B. Miller, and K. V. Semenikhin, “Optimal channel choice for lossy data flow transmission,” Automation and Remote Control 79 (1), 66–77 (2018).Google Scholar
  6. 6.
    U. Rieder and J. Winter, “Optimal control of Markovian jump processes with partial information and applications to a parallel queueing model,” Math. Meth. Oper. Res. 70, 567–596 (2009).MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    N. A. Kuznetsov, D. V. Myasnikov, and K. V. Semenikhin, “Optimization of two-phase queuing system and its application to the control of data transmission between two robotic agents,” Journal of Communications Technology and Electronics 62 (12), 1484–1498 (2017).CrossRefGoogle Scholar
  8. 8.
    A. Hordijk and F. Spieksma, “Constrained admission control to a queueing system,” Adv. Appl. Probab. 21, 409–431 (1989).MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    M. Y. Kitaev and V. V. Rykov, Controlled Queueing Systems (CRC, Boca Raton, 1995).zbMATHGoogle Scholar
  10. 10.
    E. Altman, Constrained Markov Decision Processes (Chapman & Hall/CRC, Boca Raton, 1999).zbMATHGoogle Scholar
  11. 11.
    R. J. Elliott, L. Aggoun, and J. B. Moore, Hidden Markov Models. Estimation and Control (Springer-Verlag, New York, 2008).zbMATHGoogle Scholar
  12. 12.
    B. M. Miller, G. B. Miller, and K. V. Semenikhin, “Methods to design optimal control of Markov process with finite state set in the presence of constraints,” Automation and Remote Control 72 (2), 323‒341 (2011).Google Scholar
  13. 13.
    D.-T. Ho, E. I. Grotli, P. B. Sujit, T. A. Johansen, and J. B. Sousa, “Optimization of wireless sensor network and UAV data acquisition,” J. Intell. & Robotic Syst. 78 (1), 159–179 (2015).CrossRefGoogle Scholar
  14. 14.
    B. Miller, G. Miller, and K. Semenikhin, “Optimization of the data transmission flow from moving object to nonhomogeneous network of base stations,” in Proc. 20th IFAC World Congress (IFAC’2017), Toulouse, France, 2017 (IFAC, 2017).Google Scholar
  15. 15.
    T. M. Cover and J. A. Thomas, Elements of Information Theory (Wiley, New York, 1991).Google Scholar
  16. 16.
    T. S. Rappaport, Wireless Communications: Principles and Practice, 2nd Ed. (Prentice Hall, Upper Saddle River, 2002).zbMATHGoogle Scholar
  17. 17.
    B. M. Miller, G. B. Miller, and K. V. Semenikhin, “Optimal control problem regularization for the Markov process with finite number of states and constraints,” Automation and Remote Control 77 (9), 1589–1611 (2016).Google Scholar

Copyright information

© Pleiades Publishing, Inc. 2018

Authors and Affiliations

  1. 1.Kotel’nikov Institute of Radio Engineering and Electronics, Russian Academy of SciencesMoscowRussia
  2. 2.Moscow Institute of Physics and TechnologyDolgoprudnyiRussia
  3. 3.Moscow Aviation Institute (National Research University)MoscowRussia

Personalised recommendations