ε-QLMR: ε-greedy based Q-Learning algorithm for Multipath Routing in SDN networks

Abstract

Software Defined Networking (SDN) has leveraged the recent advances in Artificial Intelligence (AI) techniques to meet the main challenges related to network management complexity. For instance, Reinforcement Learning (RL) algorithms, which address decision-making problems, have been extensively used for routing optimization in SDN networks. Based on the interaction between the controller and the network, the RL algorithm can learn optimal routing policies, hence empowering the controller with intelligence capabilities. In this work, we address latency minimization within a multipath SDN network using the Q-Learning algorithm. Unlike most commonly used approaches, which are based on indirect modeling of the flow routing mechanism as an RL problem, we consider directflow modeling to guarantee flow integrity preservation within a multipath configuration. In particular, we provide a performance evaluation of the Q-Learning algorithm using two different exploration strategies, namely, $\varepsilon$-greedy and softmax, in terms of average flow latency and convergence time for different load levels. We show that, for both strategies, the proposed algorithm performs better than the traditional SPF (Short Path First) routing protocol for almost all load levels. As compared to the softmax strategy, we show that $\varepsilon$-greedy achieves lower levels of average latency and comparable convergence time for the considered topology.