Load Balancing Multi-Player MAB Approaches for RIS-Aided mmWave User Association

In this paper, multiple reconfigurable intelligent surface (RIS) boards are deployed to enhance millimeter wave (mmWave) communication in a harsh blockage environment, where mmWave line-of-sight (LoS) link is completely blocked. Herein, RIS-user association should be considered to maximize the users' achievable data rate while assuring load balance among the installed RIS panels. However, maximum received power (MRP) based RIS-user association will overload some of the RIS boards while keeping others unloaded, which causes RIS load to unbalance and decreases the users' achievable data rate. Instead, in this paper, an online learning methodology using centralized multi-player multi-armed bandit (MP-MAB) with arms' load balancing is proposed. In this formulation, mmWave users, RIS boards, and achievable users' rates act as the bandit game players, arms, and rewards. During the MAB game, the users learn how to avoid associating with the heavily loaded RIS boards, maximizing their achievable data rates, and balancing the RIS loads. Three centralized MP-MAB algorithms with arms' load balancing are proposed from the family of upper confidence bound (UCB) MAB algorithms. These algorithms are UCB1, Kullback-Leibler divergence UCB (KLUCB), and Minimax optimal stochastic strategy (MOSS) with arms' load balancing, i.e., UCB1-LB, KLUCB-LB, and MOSS-LB. Numerical analysis ensures the superior performance of the proposed algorithms over MRP-based RIS-user association and other benchmarks.