Actor-critic architecture based probabilistic meta-reinforcement learning for load balancing of controllers in software defined networks

摘要

The exponential growth in the complexity of network architecture to accommodate the enormous amount of data has motivated the emergence of software-defined networks (SDN). However, the scaling of network size and related services seriously affects the controller resource utilization in SDN. An optimal load balancing strategy is required to accommodate the dynamic network traffic and load distribution among different controllers. Although reinforcement learning has been used for load balancing in SDN by modelling it as the linear optimization problem, it could not perform well for the real-time load in SDN due to the uncertain and dynamic correlation between the resources. This paper presents a deep meta-reinforcement learning (meta-RL) technique to derive an intelligent optimization framework for load balancing in SDN using actor-critic architecture. Meta-RL is the modified version of conventional reinforcement learning which utilizes a smaller amount of training data comparatively to enable the agent to learn the policies. The proposed technique separates the task's inference and control to deal with the uncertainty while adapting to the new tasks in the sparse reward problem. It utilizes the probabilistic interpretation of task variables to solve the new task through sparse experience space. The simulation analysis supports the theoretical analysis of SDN's optimal load balancing technique. A real-time database has been utilized in this work to evaluate the efficiency and effectiveness of the proposed work. It has been found that the proposed technique showed an average of 10.4%, 19.1%, and 6.04% improvement in load balancing rate as compared to DC-LB, MMO-LB, and CRL-LB techniques. It also showed an improvement of 15.7%, 12.18%, and 5.18% in processing delay compared to the same methods, respectively. The standard deviation is also improved by 23.72% compared to the scenario of no-load balancing.