The vehicular ad hoc network (VANET) has recently emerged as a promising networking technique attracting both the vehicular manufacturing industry and the academic community. Therefore, the design of next generation VANET management schemes becomes an important issue to satisfy the new demands. However, it is difficult to adapt traditional control approaches, which have already proven reliable in ad-hoc wireless networks, directly. In this study, we focus on the development of vehicular crowdsensing and routing algorithms in VANETs. The proposed scheme, which is based on reinforcement learning and game theory, is designed as novel vertical and horizontal game models, and provides an effective dual-plane control mechanism. In a vertical game, network agent and vehicles work together toward an appropriate crowdsensing process. In a horizontal game, vehicles select their best routing route for the VANET routing. Based on the decentralized, distributed manner, our dual-plane game paradigm captures the dynamics of the VANET system. Simulations and performance analysis verify the efficiency of the proposed scheme, showing that our approach can outperform existing schemes in terms of RSU’s task success ratio, normalized routing throughput, and end-to-end packet delay.
Vehicular ad-hoc network Vertical–horizontal game model Mobile crowdsensing Reinforcement learning Game theory
This is a preview of subscription content, log in to check access.
This research was supported by the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2017-2014-0-00636) supervised by the IITP (Institute for Information and Communications Technology Promotion) and was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2015R1D1A1A01060835).
Compliance with ethical standards
Conflict of interest
The author, Sungwook Kim, declares that there is no competing interests regarding the publication of this paper.
Wang, Y., Liu, Y., Zhang, J., Ye, H., & Tan, Z. (2017). Cooperative store–carry–forward scheme for intermittently connected vehicular networks. IEEE Transactions on Vehicular Technology,66(1), 777–784.Google Scholar
Morales, M. M. C., Haw, R., Cho, E. J., Hong, C. S., & Lee, S. (2012). An adaptable destination-based dissemination algorithm using a publish/subscribe model in vehicular networks. JCSE,6(3), 227–242.CrossRefGoogle Scholar
Chen, J., Mao, G., Li, C., Zafar, A., & Zomaya, A. Y. (2017). Throughput of infrastructure-based cooperative vehicular networks. IEEE Transactions on Intelligent Transportation Systems, 18(11), 2964–2979.CrossRefGoogle Scholar
Su, Z., Hui, Y., & Yang, Q. (2017). The next generation vehicular networks: A content-centric framework. IEEE Wireless Communications,24(1), 60–66.CrossRefGoogle Scholar
Wang, C., Zhang, Z., Shao, L., & Zhou, M. (2016). Estimating travel speed via sparse vehicular crowdsensing data. IEEE World Forum on Internet of Things (WF-IoT), 1, 643–648.Google Scholar
Chalkiadakis, G. (2007). A Bayesian approach to multiagent reinforcement learning and coalition formation under uncertainty. Doctoral dissertation, University of Toronto.Google Scholar
Hu, T., & Fei, Y. (2010). QELAR: A machine-learning-based adaptive routing protocol for energy-efficient and lifetime-extended underwater sensor networks. IEEE Transactions on Mobile Computing,9(6), 796–809.CrossRefGoogle Scholar
Galindo-Serrano, A., & Giupponi, L. (2010). Distributed Q-learning for aggregated interference control in cognitive radio networks. IEEE Transactions on Vehicular Technology,59(4), 1823–1834.CrossRefGoogle Scholar
Bai, F., & Krishnamachari, B. (2010). Exploiting the wisdom of the crowd: Localized, distributed information-centric VANETs. IEEE Communications Magazine,48(5), 138–146.CrossRefGoogle Scholar
Wu, D., Arkhipov, D. I., Zhang, Y., Liu, C. H., & Regan, A. C. (2015). Online war-driving by compressive sensing. IEEE Transactions on Mobile Computing,14(11), 2349–2362.CrossRefGoogle Scholar
Wu, D., Liu, Q., Li, Y., McCann, J. A., Regan, A. C., & Venkatasubramanian, N. (2016). Adaptive lookup of open WiFi using crowdsensing. IEEE/ACM Transactions on Networking,24(6), 3634–3647.CrossRefGoogle Scholar
Talebifard, P., & Leung, V. C. M. (2013). Towards a content-centric approach to crowd-sensing in vehicular clouds. Journal of Systems Architecture,59, 976–984.CrossRefGoogle Scholar
Bazzi, A., & Zanella, A. (2016). Position based routing in crowd sensing vehicular networks. Ad Hoc Networks,36, 409–424.CrossRefGoogle Scholar
Wan, J., Liu, J., Shao, Z., Vasilakos, A. V., Imran, M., & Zhou, K. (2016). Mobile crowd sensing for traffic prediction in internet of vehicles. Sensors,16(1), 1–15.CrossRefGoogle Scholar
Wu, D., Zhang, Y., Luo, J., & Li, R. (2014). Efficient data dissemination by crowdsensing in vehicular networks. In IEEE IWQoS’2014 (pp. 314–319).Google Scholar
Petrov, V., Samuylov, A., Begishev, V., Moltchanov, D., Andreev, S., Samouylov, K., & Koucheryavy, Y. (2017). Vehicle-based relay assistance for opportunistic crowdsensing over narrowband IoT (NB-IoT). IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2017.2670363.Google Scholar
Han, K., Chen, C., Zhao, Q., & Guan, X. (2015). Trajectory-based node selection scheme in vehicular crowdsensing. In IEEE/CIC international conference on communications in China (pp. 1–6).Google Scholar
Sun, J., Hou, F., Ma, S., & Shan, H. (2016). Social-aware incentive mechanism for participatory sensing. In IEEE GLOBECOM’2016 (pp. 1–6).Google Scholar
Nzouonta, J., Rajgure, N., Wang, G., & Borcea, C. (2009). VANET routing on city roads using real-time vehicular traffic information. IEEE Transactions on Vehicular Technology,58(7), 3609–3626.CrossRefGoogle Scholar
Hieu, C. T., & Hong, C. S. (2010). A connection entropy-based multi-rate routing protocol for mobile ad hoc networks. JCSE,4(3), 225–239.CrossRefGoogle Scholar
Jeon, M., Kim, S.-K., Yoon, J.-H., Lee, J., & Yang, S.-B. (2014). A direction entropy-based forwarding scheme in an opportunistic network. JCSE,8(3), 173–179.CrossRefGoogle Scholar
Kim, S. (2012). Adaptive ad hoc network routing scheme by using incentive-based model. Ad Hoc & Sensor Wireless Networks,15, 1–19.Google Scholar
Khaliq-ur-Rahman Raazi, S. M., & Lee, S. (2010). A survey on key management strategies for different applications of wireless sensor networks. JCSE,4(1), 23–51.CrossRefGoogle Scholar
Kim, K., Uno, S., & Kim, M. (2010). Adaptive QoS mechanism for wireless mobile network. JCSE,4(2), 153–172.CrossRefGoogle Scholar