Telecommunication Systems

, Volume 68, Issue 1, pp 89–104 | Cite as

A stochastic approximation approach to active queue management

  • Shalabh Bhatnagar
  • Sanjeev Patel
  • Karmeshu


Recently, a dynamic adaptive queue management with random dropping (AQMRD) scheme has been developed to capture the time-dependent variation of average queue size by incorporating the rate of change of average queue size as a parameter. A major issue with AQMRD is the choice of parameters. In this paper, a novel online stochastic approximation based optimization scheme is proposed to dynamically tune the parameters of AQMRD and which is also applicable for other active queue management (AQM) algorithms. Our optimization scheme significantly improves the throughput, average queue size, and loss-rate in relation to other AQM schemes.


Active queue management (AQM) Stochastic approximation Dropping probability Heavy traffic conditions Stochastic simulation Traffic control 


  1. 1.
    Floyd, S., & Jacobson, V. (1993). Random early detection gateways for congestion avoidance. IEEE/ACM Transactions on Networking, 1(4), 397–413.CrossRefGoogle Scholar
  2. 2.
    Athuraliya, S., Li, V. H., Low, S. H., & Elissa, Q. Y. (2002). REM: Active queue management. IEEE Network, 15(3), 48–53.CrossRefGoogle Scholar
  3. 3.
    Meckenney, P. E. (1990). Stochastic fair queuing. Proceedings IEEE INFOCOM, 2, 733–740.Google Scholar
  4. 4.
    Hollot, C. V., Misra, V., Towsley, D., Gong, W. (2001). On designing improved controllers for AQM routers supporting TCP flows. In Proceedings of the IEEE INFOCOM (pp. 1726–1734).Google Scholar
  5. 5.
    Feng, W., Shin, K. G., Kandlur, D. D., & Saha, D. (2002). The BLUE active queue management algorithms. IEEE/ACM Transactions on Networking, 10(4), 513–528.CrossRefGoogle Scholar
  6. 6.
    Feng, G., Agarwal, A . K., Jayaraman, A., & Siew, C . K. (2004). Modified RED gateways under bursty traffic. IEEE Communications Letter, 8(5), 323–325.CrossRefGoogle Scholar
  7. 7.
    Floyd, S., Gummadi, R., Shenker, S. (2001). Adaptive RED: An algorithm for increasing the robustness of RED’s active queue management. Technical Report, UC, Berkeley, CA, (Online)
  8. 8.
    Feng, C. W., Huang, L. F., Xu, C., & Chang, Y. C. (2017). Congestion control scheme performance analysis based on nonlinear RED. IEEE System Journal, 99, 1–8.Google Scholar
  9. 9.
    Verma, R., Iyer, A., & Karandikar, A. (2003). Active queue management using adaptive RED. Journal of Communications and Networks, 5(3), 275–281.CrossRefGoogle Scholar
  10. 10.
    Zadeh, H. Y., Habibi, A., Li, X., Jafarkhani, H., & Bauer, C. (2012). A statistical study of loss-delay tradeoff for RED queues. IEEE Transactions on Communications, 60(7), 1966–1974.CrossRefGoogle Scholar
  11. 11.
    Xu, Q., & Sun, J. (2012). New active queue management scheme based on statistical analysis. In Proceedings of WCICA ’12, (pp. 2562–2565). Beijing, China.Google Scholar
  12. 12.
    Kulatungay, C., Kuhnz, N., Fairhursty, G., & Ros, D. (2015). Tackling Bufferbloat in capacity-limited networks. In EuCNC’15 (pp. 381-385). doi: 10.1109/EuCNC.2015.7194103.
  13. 13.
    Wang, J., Rong, L., & Liu, Y. (2008). A robust proportional controller for AQM based on optimized second-order system model. Computer Communications, 31(10), 2468–2477.CrossRefGoogle Scholar
  14. 14.
    Chavan, K., Kumar, R. G., Belur, M. N., & Karandikar, A. (2011). Robust active queue management for wireless networks. IEEE Transactions on Control Systems Technology, 19(6), 1630–1638.CrossRefGoogle Scholar
  15. 15.
    Tan, L., Zhang, W., Peng, G., & Chen, G. (2006). Stability of TCP/RED systems in AQM routers. IEEE Transactions on Automatic Control, 51(8), 1393–1398.CrossRefGoogle Scholar
  16. 16.
    Woo, S., & Kim, K. (2010). Tight upper bound for stability of TCP/RED systems in AQM routers. IEEE Communications Letters, 14(7), 682–684.CrossRefGoogle Scholar
  17. 17.
    Qazi, I. A., Andrew, L. L. H., & Znati, T. K. (2012). Congestion control with multipacket feedback. IEEE/ACM Transactions on Networking, 20(6), 1721–1733.CrossRefGoogle Scholar
  18. 18.
    Poojary, S., & Sharma, V. (2016). Analysis of multiple flows using different high speed TCP protocols on a general network. Performance Evaluation, 104, 4262.CrossRefGoogle Scholar
  19. 19.
    Akyildiz, I. F., Lee, A., Wang, P., Luo, M., & Chou, W. (2015). Research challenges for traffic engineering in software defined networks. IEEE Network, 30(3), 52–58.CrossRefGoogle Scholar
  20. 20.
    Akyildiz, I. F., Nie, S., Lin, S.-C., & Chandrasekaran, M. (2016). 5G roadmap: 10 key enabling technologies. Computer Networks, 106(4), 17–48.CrossRefGoogle Scholar
  21. 21.
    Harel, A., Namn, S., & Sturm, J. (1999). Simple bounds for closed queuing networks. Queueing Systems, 31(1), 125–135.CrossRefGoogle Scholar
  22. 22.
    Padhye, J., Firoiu, V., Towsley, D. F., & Kurose, J. F. (2000). Modeling TCP Reno performance: A simple model and its empirical validation. IEEE/ACM Transactions on Networking, 8, 133–145.CrossRefGoogle Scholar
  23. 23.
    Yan, J., Muhlbauer, W., & Plattner, B. (2011). An analytical model for streaming over TCP. In NEW2AN (pp. 370–381).Google Scholar
  24. 24.
    Yan, J., & Plattner, B. (2013). A simple solution to find the distribution of TCP window sizes. IEEE Communications Letters, 17(2), 417–419.CrossRefGoogle Scholar
  25. 25.
    Gemikonakli, E., Ever, E., Mapp, G., & Gemikonaklii, O. (2017). Admission control and buffer management of wireless communication systems with mobile stations and integrated voice and data services. Telecommunication Systems, 65(4), 663–675.CrossRefGoogle Scholar
  26. 26.
    Salah, K., & Kafhali, S. E. (2017). Performance modeling and analysis of hypoexponential network servers. Telecommunication Systems, 65(4), 717–728.CrossRefGoogle Scholar
  27. 27.
    Prashanth, L. A., Bhatnagar, S., Fu, M., & Marcus, S. (2017). Adaptive system optimization using random directions stochastic approximation. IEEE Transactions on Automatic Control, 62(5), 2223–2238.CrossRefGoogle Scholar
  28. 28.
    Spall, J. C. (1992). Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Transactions on Automatic Control, 37(3), 332–341.CrossRefGoogle Scholar
  29. 29.
    Bhatnagar, S. (2007). Adaptive Newton-based smoothed functional algorithms for simulation optimization. ACM Transactions on Modeling and Computer Simulation, 18(1), 2:1–2:35.CrossRefGoogle Scholar
  30. 30.
    Karmeshu, Bhatnagar, S., & Mishra, V. K. (2011). An optimized SDE model for slotted aloha. IEEE Transactions on Communications, 59(6), 1502–1508.Google Scholar
  31. 31.
    Bhatnagar, S., Prasad, H. L., & Prashanth, L. A. (2013). Stochastic recursive algorithms for optimization: Simultaneous perturbation methods., Lecture notes in control and information sciences London: Springer.CrossRefGoogle Scholar
  32. 32.
    Patro, R. K., & Bhatnagar, S. (2009). A probabilistic constrained nonlinear optimization framework to optimize RED parameters. Performance Evaluation, 66(2), 81–104.CrossRefGoogle Scholar
  33. 33.
    Karmeshu, Patel, S., & Bhatnagar, S. (2017). Adaptive mean queue size and its rate of change: Queue management with random dropping. Telecommunication Systems, 65(2), 281–295.Google Scholar
  34. 34.
    Kushner, H. J., & Clark, D. S. (1978). Stochastic approximation methods for constrained and unconstrained systems. New York: Springer.CrossRefGoogle Scholar
  35. 35.
    Wang, H., & Shin, K. G. (1999). Refined design of random early detection gateways. In Proceedings of IEEE GLOBECOM (pp. 769–775).Google Scholar
  36. 36.
    Hollot, C. V., Misra, V., Towsley, D., & Gong, W. (2002). Analysis and design of controllers for AQM routers supporting TCP flows. IEEE Transactions on Automatic Control, 47(6), 945–959.CrossRefGoogle Scholar
  37. 37.
    Low, S. H., Paganini, F., Wang, J., & Doyle, J. C. (2003). Linear stability of TCP/RED and a scalable control. Computer Networks Journal, 43(5), 633–647.CrossRefGoogle Scholar
  38. 38.
    Bhatnagar, S., & Patro, R. K. (2009). A proof of convergence of the B-RED and P-RED algorithms for random early detection. IEEE Communications Letters, 13(10), 809–811.CrossRefGoogle Scholar
  39. 39.
    Adams, R. (2013). Active queue management: A survey. IEEE Communations Surveys & Tutorials, 15(3), 1425–1476.CrossRefGoogle Scholar
  40. 40.
    Wu, Y., Min, G., & Yang, L. T. (2013). Performance analysis of hybrid wireless networks under bursty and correlated traffic. IEEE Transactions on Vehicular Technology, 62(1), 449–454.CrossRefGoogle Scholar
  41. 41.
    Wang, C., Li, B., Thomas Hou, Y., Sohraby, K. & Lin, Y. (2004). LRED: A robust active queue management scheme based on packet loss ratio. In Proceedings on 23rd Annual Joint Conference on IEEE INFOCOM (vol. 1, pp. 112).Google Scholar
  42. 42.
    Bhatnagar, S., Fu, M. C., Marcus, S. I., & Wang, I. J. (2003). Two-timescale simultaneous perturbation stochastic approximation using deterministic perturbation sequences. ACM Transactions on Modelling and Computer Simulation, 13(2), 180–209.CrossRefGoogle Scholar
  43. 43.
    Bhatnagar, S. (2005). Adaptive multivariate three-timescale stochastic approximation algorithms for simulation based optimization. ACM Transactions on Modeling and Computer Simulation, 15(1), 74–107.CrossRefGoogle Scholar
  44. 44.
    Borkar, V. S. (2008). Stochastic approximation: A dynamical systems viewpoint. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  45. 45.
    Bhatnagar, S., & Borkar, V. S. (1997). Multiscale stochastic approximation for parametric optimization of hidden Markov models. Probability in the Engineering and Informational Sciences, 11, 509–522.CrossRefGoogle Scholar
  46. 46.
    Bhatnagar, S., Fu, M. C., Marcus, S. I., & Bhatnagar, S. (2001). Two timescale algorithms for simulation optimization of hidden Markov models. IIE Transactions (Pritsker special issue on simulation), 3, 245–258.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Department of Computer Science and AutomationIndian Institute of ScienceBangaloreIndia
  2. 2.The School of Computer and Systems SciencesJawaharlal Nehru UniversityNew DelhiIndia

Personalised recommendations