Wireless Networks

, Volume 25, Issue 7, pp 4097–4113 | Cite as

To overhear or not to overhear: a dilemma between network coding gain and energy consumption in multi-hop wireless networks

  • Nastooh Taheri Javan
  • Masoud SabaeiEmail author
  • Mehdi Dehghan


Any properly designed network coding technique can result in increased throughput and reliability of multi-hop wireless networks by taking advantage of the broadcast nature of wireless medium. In many inter-flow network coding schemes nodes are encouraged to overhear neighbour’s traffic in order to improve coding opportunities at the transmitter nodes. A study of these schemes reveal that some of the overheard packets are not useful for coding operation and thus this forced overhearing increases energy consumption dramatically. In this paper, we formulate network coding aware sleep/wakeup scheduling as a semi Markov decision process (SMDP) that leads to an optimal node operation. In the proposed solution for SMDP, the network nodes learn when to switch off their transceiver in order to conserve energy and when to stay awake to overhear some useful packets. One of the main challenges here is the delay in obtaining reward signals by nodes. We employ a modified reinforcement learning method based on continuous-time Q-learning to overcome this challenge in the learning process. Our simulation results confirm the optimality of the new methodology.


Wireless multi-hop networks Network coding Energy consumption Coding gain SMDP Q-learning 


  1. 1.
    Ahlswede, R., Cai, N., Li, S. Y. L., & Yeung, R. W. (2000). Network information flow. IEEE Transactions on Information Theory, 46(4), 1204–1216.MathSciNetCrossRefGoogle Scholar
  2. 2.
    Farooqi, M. Z., Tabassum, S. M., Rehmani, M. H., & Saleem, Y. (2014). A survey on network coding: From traditional wireless networks to emerging cognitive radio networks. Journal of Network and Computer Applications, 46, 166–181.CrossRefGoogle Scholar
  3. 3.
    Omiwade, S., Zheng, R., & Hua, C. (2008). Butterflies in the mesh: Lightweight localized wireless network coding. In Proceedings of 4th workshop on network coding theory and applications (NetCod’08), Hong Kong, China, January 2008, pp. 1–6.Google Scholar
  4. 4.
    Eryilmaz, A., Lun, D., & Swapna, B. T. (2011). Control of multi-hop communication networks for inter-session network coding. IEEE Transactions on Information Theory, 57(2), 1092–1110.MathSciNetCrossRefGoogle Scholar
  5. 5.
    Fragouli, C., Widmer, J., & Boudec, J. L. (2008). Efficient broadcasting using network coding. IEEE/ACM Transactions on Networking, 16(2), 450–463.CrossRefGoogle Scholar
  6. 6.
    Katti, S., Rahul, H., Hu, W., Katabi, D., Medard, M., & Crowcroft, J. (2008). Xors in the air: Practical wireless network coding. IEEE/ACM Transactions on Networking, 16, 497–510.CrossRefGoogle Scholar
  7. 7.
    Le, J., Lui, J. C. S., & Chiu, D. M. (2010). DCAR: Distributed coding-aware routing in wireless networks. IEEE Transaction on Mobile Computing, 9(4), 596–607.CrossRefGoogle Scholar
  8. 8.
    Paschos, G., Georgiadis, L., & Tassiulas, L. (2012). Scheduling with pairwise XORing of packets under statistical overhearing information and feedback. Queueing Systems, 72, 361–395.MathSciNetCrossRefGoogle Scholar
  9. 9.
    Feng, D., Jiang, C., Lim, G., Cimini, L. J., Feng, G., & Li, G. Y. (2013). A survey of energy-efficient wireless communications. IEEE Communication Surveys & Tutorials, 15(1), 167–187.CrossRefGoogle Scholar
  10. 10.
    IEEE Standard for Wireless LAN-Medium Access Control and Physical Layer Specification, P802.11, November 1997.Google Scholar
  11. 11.
    Shwe, H. Y., & Jiang, X. (2011). Efficient power savings in wireless sensor networks with network coding and overhearing avoidance. International Journal of Computer Engineering Research, 2(2), 34–44.Google Scholar
  12. 12.
    Samarasinghe, K., Voigt, T., Mottola, L., & Roeding, U. (2011). Network coding with limited overhearing. In Proceedings of 8th European conference on wireless sensor networks (EWSN’11), Bonn, Germany, February 2011.Google Scholar
  13. 13.
    Guo, X., & Hernndez-Lerma, O. (2009). Continuous-time Markov decision processes: Theory and applications. Berlin: Springer.CrossRefGoogle Scholar
  14. 14.
    Puterman, M. L. (2005). Markov decision processes: Discrete stochastic dynamic programming. New York: Wiley.zbMATHGoogle Scholar
  15. 15.
    Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: Introduction. Cambridge, MA: MIT Press.zbMATHGoogle Scholar
  16. 16.
    Watkins, C., & Dayan, P. (1992). Q-learning. Machine learning, 8(3–4), 279–292.zbMATHGoogle Scholar
  17. 17.
    NS2. Accessed 07 Jul 2014.
  18. 18.
    Johnson, D. B., & Maltz, D. A. (1996). Dynamic source routing in ad hoc wireless networks. In T. Imielinski & H. Korth (Eds.), Mobile computing (pp. 153–181). Dordrecht: Kluwer Academic Publishers.CrossRefGoogle Scholar
  19. 19.
    Vazifehdan, J., Prasad, R. V., & Niemegeers, I. (2014). Energy-efficient reliable routing considering residual energy in wireless ad hoc networks. IEEE Transactions on Mobile Computing, 13(2), 434–447.CrossRefGoogle Scholar
  20. 20.
    Park, I., Kim, D., & Har, D. (2015). Mac achieving low latency and energy efficiency in hierarchical M2M networks with clustered nodes. IEEE Sensors Journal, 15(3), 1657–1661.CrossRefGoogle Scholar
  21. 21.
    Wang, J., Cao, Z., Mao, X., Li, X.-Y., & Liu, Y. (2016). Towards energy efficient duty-cycled networks: Analysis, implications and improvement. IEEE Transactions on Computers, 65(1), 270–280.MathSciNetCrossRefGoogle Scholar
  22. 22.
    Spenza, D., Magno, M., Basagni, S., Benini, L., Paoli, M., & Petrioli, C. (2015). Beyond duty cycling: Wake-up radio with selective awakenings for long-lived wireless sensing systems. In Proceedings of the IEEE conference on computer communications (INFOCOM’15), Kowloon, Hong Kong, pp. 522–530.Google Scholar
  23. 23.
    Zhang, X. M., Zhang, Y., Yan, F., & Vasilakos, A. V. (2015). Interference-based topology control algorithm for delay-constrained mobile ad hoc networks. IEEE Transactions on Mobile Computing, 14(4), 742–754.CrossRefGoogle Scholar
  24. 24.
    Asemani, M., & Esnaashari, M. (2015). Learning automata based energy efficient data aggregation in wireless sensor networks. Wireless Networks, 21(6), 2035–2053.CrossRefGoogle Scholar
  25. 25.
    Zordan, D., Melodia, T., & Rossi, M. (2016). On the design of temporal compression strategies for energy harvesting sensor networks. IEEE Transactions on Wireless Communications, 15(2), 1336–1352.CrossRefGoogle Scholar
  26. 26.
    Mohandespour, M., Govindarasu, M., & Wang, Zh. (2016). Rate, energy, and delay tradeoffs in wireless multicast: network coding vs. routing. IEEE Transactions on Mobile Computing, 15(5), 952–963.CrossRefGoogle Scholar
  27. 27.
    Cui, T., Chen, L., & Ho, T. (2008). Energy efficient opportunistic network coding for wireless networks. In Proceedings of IEEE conference on computer communications (INFOCOM’08), Phoenix, USA, pp. 361–365.Google Scholar
  28. 28.
    Widmer, J., Fragouli, C., & Le Boudec, J.-Y. (2005). Low-complexity energy-efficient broadcasting in wireless ad-hoc networks using network coding. In Proceedings of first workshop on network coding theory and applications (NetCod’05), Grada, Italy.Google Scholar
  29. 29.
    Brown, J., McCarthy, B., Roedig, U., Voigt, T., & Sreenan, C. J. (2011). Burstprobe: Debugging time-critical data delivery in wireless sensor networks. In Proceedings of 8th European conference on wireless sensor networks (EWSN’11), Bonn, Germany.Google Scholar
  30. 30.
    Ning, H., Ling, C., & Leung, K. (2010). Wireless network coding with imperfect overhearing. Arxiv preprint arXiv:1003.4270.
  31. 31.
    Hwang, C. S., & Moon, H. (2009). Rate adaptation for wireless network coding using partial overhearing. IEEE Communications Letters, 13(12), 890–892.CrossRefGoogle Scholar
  32. 32.
    Paschos, G. S., Fragiadakis, C., Georgiadis, L., & Tassiulas, L. (2013). Wireless network coding with partial overhearing information. In Proceedings of the IEEE conference on computer communications (INFOCOM’13), Turin, Italy, pp. 14–19.Google Scholar
  33. 33.
    Xie, L. F., Chong, P. H. J., & Guan, Y. L. (2015). Performance analysis of network coding with virtual overhearing in wireless networks. IEEE Transactions on Vehicular Technology, 64(5), 2051–2061.CrossRefGoogle Scholar
  34. 34.
    Xie, L. F., Chong, P. H. J., Ho, I. W. H., & Chan, H. C. B. (2016). Virtual overhearing: An effective way to increase network coding opportunities in wireless ad-hoc networks. Computer Networks, 105, 111–123.CrossRefGoogle Scholar
  35. 35.
    Routand, R. R., & Ghosh, S. K. (2013). Enhancement of lifetime using duty cycle and network coding in wireless sensor networks. IEEE Transactions on Wireless Communications, 12(2), 656–667.CrossRefGoogle Scholar
  36. 36.
    Lee, K. H., Kima, J. H., & Cho, S. (2016). Power saving mechanism with network coding in the bottleneck zone of multimedia sensor networks. Computer Networks, 96(1), 58–68.CrossRefGoogle Scholar
  37. 37.
    Chandanala, R., & Stoleru, R. (2010). Network coding in duty-cycled sensor networks. In Proceedings of 7th international conference on networked sensing systems (INSS’10), Kassel, Germany, pp. 203–210.Google Scholar
  38. 38.
    Chandanala, R., Zhang, W., Stoleru, R., & Won, M. (2013). On combining network coding with duty-cycling in flood-based wireless sensor networks. Ad Hoc Networks, 11(1), 490–507.CrossRefGoogle Scholar
  39. 39.
    Trujilloa, R. P., Zarateb, J. A., Granellic, F., Fitzekd, F. H. P., & Fonsecaa, N. L. S. (2015). Network coding and duty cycling in IEEE 802.11 wireless networks with bidirectional transmissions and sleeping periods. In Proceedings of IEEE global communication conference (GLOBECOM’15), San Diego, USA, pp. 1–7.Google Scholar
  40. 40.
    Mao, G. (2012). Wireless multi-hop networks: Current research and future challenges. Journal of Communications, 7(5), 357–364.CrossRefGoogle Scholar
  41. 41.
    Wang, X., Leng, S., Ou, S., Zhao, Q., & Qiao, G. (2017). Energy harvesting and computing enabled data broadcasting in mobile social networks. International Journal of Communication Systems (IJCS). Scholar
  42. 42.
    Ku, M. L., Li, W., Chen, Y., & Liu, K. J. R. (2016). Advances in energy harvesting communications: Past, present, and future challenges. IEEE Communications Surveys Tutorials, 18(2), 1384–1412.CrossRefGoogle Scholar
  43. 43.
    Bradtke, S. J., & Duff, M. O. (1995). Reinforcement learning methods for continuous-time Markov decision problems. Advances in neural information processing systems (Vol. 7, pp. 393–400). Cambridge, MA: MIT Press.Google Scholar
  44. 44.
    Ghavamzadeh, M., & Mahadevan, S. (2001). Continuous-time hierarchical reinforcement learning. In Proceedings of 18th international conference on machine learning, San Francisco, USA, pp. 186–193.Google Scholar
  45. 45.
    Walsh, T. J., Nouri, A., Li, L., & Littman, M. L. (2009). Learning and planning in environments with delayed feedback. Autonomous Agents and Multi-Agent Systems, 18(1), 83–105.CrossRefGoogle Scholar
  46. 46.
    Katsikopoulos, K. V., & Engelbrecht, S. E. (2003). Markov decision processes with delays and asynchronous cost collection. IEEE Transactions on Automatic Control, 48(4), 568–574.MathSciNetCrossRefGoogle Scholar
  47. 47.
    Campbell, J. S., Givigi, S. N., & Schwartz, H. M. (2016). Multiple model Q-learning for stochastic asynchronous rewards. Journal of Intelligent and Robotic Systems, 81(3), 407–422.CrossRefGoogle Scholar
  48. 48.
    Jaakkola, T., Jordan, M. I., & Singh, S. P. (1994). On the convergence of stochastic iterative dynamic programming algorithms. Neural Computation, 6(6), 1185–1201.CrossRefGoogle Scholar
  49. 49.
    Campbell, J. S. (2014). Multiple model reinforcement learning for environments with Poissonian time delays. M.S Thesis, Faculty of Graduate Studies and Research, Carleton University, Canada.Google Scholar
  50. 50.
    Rappaport, T. S. (1996). Wireless communications: Principles and practice. Upper Saddle River: Prentice Hall.zbMATHGoogle Scholar
  51. 51.
    Feeney, L. M., & Nilsson, M. (2003). Investigating the energy consumption of a wireless network interface in an ad hoc networking environment. In Proceedings of the IEEE conference on computer communications (INFOCOM’01), Vol. 3, Alaska, USA, pp. 1548–1557.Google Scholar
  52. 52.
    Taheri Javan, N., Sabaei, M., & Dehghan, M. (2018). To-send-or-not-to-send: An optimal stopping approach to network coding in multi-hop wireless networks. International Journal of Communication Systems. Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Computer and Information Technology Engineering DepartmentAmirkabir University of Technology (Tehran Polytechnic)TehranIran

Personalised recommendations