Advertisement

A Reinforcement Learning Approach of Data Forwarding in Vehicular Networks

  • Pengfei Zhu
  • Lejian Liao
  • Xin Li
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 747)

Abstract

As the basis of vehicle ad hoc networks, the method of forwarding data is one of the most important parts which ensures the stability and efficiency of network communication. However, the high-speed mobile vehicle nodes cause frequent changes of network topology and disconnections of network links, casting a big challenge to the performance of network data delivery. Data forwarding methods based on the prior knowledge of vehicle’s trajectory are difficult to adapt to the changing vehicle trajectory in real world applications, while getting destination vehicles’ positions in broadcast way are extremely costly. To solve the above problems, we have proposed an association state based optimized data forwarding method (ASODF) with the assistance of low loaded road side units (RSU). The proposed method maps the urban road network into a directed graph, utilizes the carry-forward mechanism and decomposes the data transmission into decision-making data forwarding at intersections and data delivery on roads. The vehicles carried data combine the destination nodes locations obtained by low loaded road side units and their locations into association states, and the association state optimization problem is formalized as a Reinforcement Learning problem with Markov Decision Process (MDP). We utilized the value iteration scheme to figure out the delay-optimal policy, which is further used to forward data packets to obtain the best delay of data transmission. Experiments based on a real vehicle trajectory data set demonstrate the effectiveness of our model ASODF.

References

  1. 1.
    Du, X., Xiao, Y., Chen, H.-H., Wu, Q.: Secure cell relay routing protocol for sensor networks. Wirel. Commun. Mob. Comput. 6(3), 375–391 (2006)CrossRefGoogle Scholar
  2. 2.
    Xiao, Y., Rayi, V.K., Sun, B., Du, X., Hu, F., Galloway, M.: A survey of key management schemes in wireless sensor networks. Comput. Commun. 30(11), 2314–2341 (2007)CrossRefGoogle Scholar
  3. 3.
    Du, X., Xiao, Y., Guizani, M., Chen, H.-H.: An effective key management scheme for heterogeneous sensor networks. Ad Hoc Netw. 5(1), 24–34 (2007)CrossRefGoogle Scholar
  4. 4.
    Du, X., Guizani, M., Xiao, Y., Chen, H.: Transactions papers a routing-driven elliptic curve cryptography based key management scheme for heterogeneous sensor networks. IEEE Trans. Wirel. Commun. 8(3), 1223–1229 (2009).  https://doi.org/10.1109/TWC.2009.060598CrossRefGoogle Scholar
  5. 5.
    Du, X., Chen, H.-H.: Security in wireless sensor networks. IEEE Wirel. Commun. 15(4) (2008)Google Scholar
  6. 6.
    Du, X., Guizani, M., Xiao, Y., Chen, H.-H.: Secure and efficient time synchronization in heterogeneous sensor networks. IEEE Trans. Veh. Technol. 57(4), 2387–2394 (2008)CrossRefGoogle Scholar
  7. 7.
    Perkins, C.E., Bhagwat, P.: Highly dynamic destination-sequenced distance-vector routing (DSDV) for mobile computers. ACM SIGCOMM Comput. Commun. Rev. 24(4), 234–244 (1994)CrossRefGoogle Scholar
  8. 8.
    Clausen, T., Jacquet, P.: Optimized link state routing protocol (OLSR). Technical report (2003)Google Scholar
  9. 9.
    Lee, S.-J., Gerla, M., Chiang, C.-C.: The dynamic source routing protocol for multi-hop wireless adhoc networksGoogle Scholar
  10. 10.
    Karp, B., Kung, H.-T.: GPSR: greedy perimeter stateless routing for wireless networks. In: Proceedings of the 6th Annual International Conference on Mobile Computing and Networking, pp. 243–254. ACM (2000)Google Scholar
  11. 11.
    Lochert, C., Hartenstein, H., Tian, J., Fussler, H., Hermann, D., Mauve, M.: A routing strategy for vehicular ad hoc networks in city environments. In: Proceedings of the Intelligent Vehicles Symposium, pp. 156–161. IEEE (2003)Google Scholar
  12. 12.
    Ding, Y., Xiao, L.: SADV: static-node-assisted adaptive data dissemination in vehicular networks. IEEE Trans. Veh. Technol. 59(5), 2445–2455 (2010)CrossRefGoogle Scholar
  13. 13.
    Zhao, J., Cao, G.: VADD: vehicle-assisted data delivery in vehicular ad hoc networks. IEEE Trans. Veh. Technol. 57(3), 1910–1922 (2008)CrossRefGoogle Scholar
  14. 14.
    Costa, P., Frey, D., Migliavacca, M., Mottola, L.: Towards lightweight information dissemination in inter-vehicular networks. In: Proceedings of the 3rd International Workshop on Vehicular Ad Hoc Networks, pp. 20–29. ACM (2006)Google Scholar
  15. 15.
    Leontiadis, I., Mascolo, C.: GEOPPS: geographical opportunistic routing for vehicular networks. In: IEEE International Symposium on World of Wireless, Mobile and Multimedia Networks: WoWMoM 2007, pp. 1–6. IEEE (2007)Google Scholar
  16. 16.
    Chen, L., Li, Z.-J., Jiang, S.-X., Feng, C.: MGF: mobile gateway based forwarding for infrastructure-to-vehicle data delivery in vehicular ad hoc networks. Jisuanji Xuebao (Chin. J. Comput.) 35(3), 454–463 (2012)Google Scholar
  17. 17.
    Jeong, J., Guo, S., Gu, Y., He, T., Du, D. TBD: trajectory-based data forwarding for light-traffic vehicular networks. In: 29th IEEE International Conference on Distributed Computing Systems, ICDCS 2009, pp. 231–238. IEEE (2009)Google Scholar
  18. 18.
    Jeong, J., Guo, S., Gu, Y., He, T., Du, D.H.: TSF: trajectory-based statistical forwarding for infrastructure-to-vehicle data delivery in vehicular networks. In: 2010 IEEE 30th International Conference on Distributed Computing Systems (ICDCS), pp. 557–566. IEEE (2010)Google Scholar
  19. 19.
    Xu, F., Guo, S., Jeong, J., Gu, Y., Cao, Q., Liu, M., He, T.: Utilizing shared vehicle trajectories for data forwarding in vehicular networks. In: 2011 Proceedings of IEEE INFOCOM, pp. 441–445. IEEE (2011)Google Scholar
  20. 20.
    Wu, Y., Zhu, Y., Li, B.: Trajectory improves data delivery in vehicular networks. In: 2011 Proceedings of IEEE INFOCOM, pp. 2183–2191. IEEE (2011)Google Scholar
  21. 21.
    Choi, O., Kim, S., Jeong, J., Lee, H.-W., Chong, S.: Delay-optimal data forwarding in vehicular sensor networks. IEEE Trans. Veh. Technol. 65(8), 6389–6402 (2016)CrossRefGoogle Scholar
  22. 22.
    Mershad, K., Artail, H.: Performance analysis of routing in VANETs using the RSU network. In: 2011 IEEE 7th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob), pp. 89–96. IEEE (2011)Google Scholar
  23. 23.
    Bellman, R.: A Markovian decision process. J. Math. Mech. 6, 679–684 (1957)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Sutton, R.S., Barto, A.G., Reinforcement Learning: An Introduction, vol. 1, no. 1. MIT Press, Cambridge (1998)Google Scholar
  25. 25.
    Pineau, J., Gordon, G., Thrun, S., et al.: Point-based value iteration: an anytime algorithm for POMDPs. In: IJCAI, vol. 3, pp. 1025–1032 (2003)Google Scholar
  26. 26.
    Huang, H.-Y., Luo, P.-E., Li, M., Li, D., Li, X., Shu, W., Wu, M.-Y.: Performance evaluation of SUVnet with real-time traffic data. IEEE Trans. Veh. Technol. 56(6), 3381–3396 (2007)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.School of Computer ScienceBeijing Institute of TechnologyBeijingChina

Personalised recommendations