Neighbor Discovery and Selection in Millimeter Wave D2D Networks Using Stochastic MAB

The propagation characteristics of millimeter-wave (mmWaves), encourages its use in the device to device (D2D) communications for fifth-generation (5G) and future beyond 5G (B5G) networks. However, due to the use of beamforming training (BT), there is a tradeoff between exploring neighbor devices for best device selection and the required overhead. In this letter, using a tool of machine learning, joint neighbor discovery and selection (NDS) in mmWave D2D networks is formulated as a stochastic budget-constraint multi-armed bandit (MAB) problem. Hence, a modified Thomson sampling (TS) and variants of upper confidence bound (UCB) based algorithms are proposed to address the topic while considering the residual energies of the surrounding devices. Simulation analysis demonstrates the effectiveness of the proposed techniques over the conventional approaches concerning average throughput, energy efficiency, and network lifetime.