WiGig access point selection using non-contextual and contextual multi-armed bandit in indoor environment

Millimeter wave (mmWave) band, i.e., 30 ~ 300 GHz, supports multi-gigabit communication making it a main component of fifth generation (5G) and future six generation (6G) wireless communications. Wireless gigabit (WiGig) is the standardized 60 GHz mmWave band for WLAN applications. MmWave has intermittent short-range transmissions necessitating the installation of multiple WiGig access points (APs) using antenna beamforming training (BT) to fully cover a target indoor area. WiGig user equipment (UE) should select the best AP among the installed ones maximizing its achievable data rate. Conventionally, UE should exhaustively search the best AP having the highest received power using BT with all available APs, which reduces the attainable throughput in consequence. In this paper, the problem of WiGig AP selection is formulated as a multi-armed bandit (MAB) game, where, the UE is considered as the player aiming to maximize its long-term average throughput, i.e., the reward, through playing over the available APs, i.e., the arms of the bandit. Non-contextual MAB algorithms, namely upper confidence bound (UCB) and Thompson sampling (TS) are adopted to address the formulated problem. Moreover, as standardized WiGig devices are multi-band capable containing 2.4/5 GHz Wi-Fi and 60 GHz mmWave bands, Wi-Fi signal characteristics are used as contexts of the mmWave links to further enhance the WiGig AP selection policy. Thus, contextual MAB (CMAB) algorithms, namely linear UCB (LinUCB) and contextual TS (CTS) are also suggested. Simulation analyses confirm the superior performance of the CMAB algorithms over the non-contextual ones in addition to the conventional approaches accompanied with high convergence rates. For example, at no blockage and using too narrow beams of θ-3dB=10∘, the proposed CTS, LinUCB, TS, and UCB schemes obtain 98.7%, 96.8%, 89%, 84% of the optimal performance, while the benchmark schemes obtain 49%, and 1.8%, respectively.