Energy-Aware Hybrid RF-VLC Multiband Selection in D2D Communication: A Stochastic Multiarmed Bandit Approach

To handle the exponentially growing service expectations from mobile users and circumvent the band switching slow rate, device-to-device (D2D) communication is receiving much research attention in the Internet of Things (IoT). While the emerging D2D nodes can support heterogeneous frequency bands [radio frequency (RF) including 2.4 GHz/5 GHz wireless local area network (WLAN), 38-GHz millimeter wave (mmWave), and visible light communication (VLC)], the physical constraints (e.g., blocking) require the user devices to dynamically switch between the bands in order to avoid the loss of connectivity and throughput degradation. In this article, we investigate an effective online link selection in hybrid RF-VLC scenarios for direct user data handling. First, we model the multiband selection issue as a multiarmed bandit (MAB) problem. The source/relay node acts as a player who gambles to maximize its long-term feedback/reward via selecting suitable arms, i.e., available bands (WLAN, mmWave, or VLC). Then, we propose an online, energy-aware band selection (EABS) methodology by leveraging three theoretically guaranteed MAB techniques [upper confidence bound (UCB), Thompson sampling (TS), and minimax optimal stochastic strategy (MOSS)] to derive optimal band selection policies. Based on these adopted policies, we propose three algorithms, namely, EABS-UCB, EABS-TS, and EABS-MOSS, to implement the EABS strategy, respectively. Extensive simulations demonstrate our proposed algorithms' superior performance compared to the traditional link selection schemes regarding energy efficiency, average throughput, and convergence rate. In particular, EABS-MOSS emerges as the best algorithm as it exhibits near-optimal performance due to its flexibility to both stochastic and adversarial environments.