Meeting Dynamic User Demand with Transmission Cost Awareness: CT-MAB RL Based Network Selection

Abstract

The access and transmission in wireless networks not only satisfy user demand for service delivery, but also incur cost in terms of fee and energy consumption, which poses the concern of cost-performance ratio. This chapter studies the cost-performance ratio optimization with dynamic traffic types in dynamic and uncertain HWN. To balance the QoE reward and transmission cost for different traffic types on the fly, the problem is formulated as a continuous-time multi-armed bandit (CT-MAB) model. A traffic-aware online network selection algorithm (ONES) is designed to match typical traffic types (user demand) with respective optimal networks in terms of QoE. In addition, we exploit the correlation feature among multiple traffic types to improve the learning capability, which inspires us to propose another two efficient algorithms: decoupled online network selection algorithm (D-ONES) and virtual multiplexing ONES (VM-ONES). Simulation results demonstrate that our online network selection algorithms achieve better QoE reward rate over non- learning-based algorithms and learning-based algorithms without QoE considerations.