Q -Learning Based Selection Strategies for Load Balance and Energy Balance in Heterogeneous Networks

For heterogeneous networks (HetNets), load balance and energy balance have a significant impact on cell range expansion (CRE) and network lifetime extension (NLE). With the rapid growth of intelligent devices, obtaining optimal bias values and routing targets for mobile user equipments (UEs) in HetNets can be challenging by using conventional selection strategies. To alleviate this problem, this paper combines two Q-learning based selection strategies (QSS) in HetNets to relief load imbalance and energy imbalance. Here, bias values and routing decisions are made by distributed Q-learning based selection strategies. In the downlink, energy received from base stations (BSs) and the number of outage UEs are used to design a Q-learning model to select bias values. In the uplink, neighbor energy sorting and varies of neighbors remaining energy of each mobile UE are used to design a Q-learning model to select routing targets. Simulation results demonstrate that the proposed QSS method has better performances of both load balance and energy balance compared to the conventional strategies.