Snakes in the Spectrum: Deep and Graph Reinforcement Learning Optimizers for Cognitive Radio Networks

Abstract

Efficient spectrum allocation in Cognitive Radio Networks (CRNs) remains a persistent challenge due to the dynamic activity of Primary Users (PUs) and potential collisions with Secondary Users (SUs). Reinforcement Learning (RL) has been increasingly applied to address this issue; however, conventional deep learning models often struggle to capture the relational structure among users and channels. In this study, we investigate the integration of the Snake Optimizer (SO) with two RL paradigms: Deep Q-Networks (DQN) and Graph-based Reinforcement Learning (DGRL). We implement a CRN environment comprising five PUs, ten SUs, and fifteen channels, optimizing Spectrum Utilization Efficiency (SUE), collision rate, and spectral capacity (SC). Each RL agent proposes channel allocations that are refined using SO to enhance convergence and reduce collisions. Comparative experiments demonstrate that both approaches achieve high performance, with DGRL+SO slightly outperforming DQN+SO across all metrics: average reward (0.4621 vs. 0.4586), SUE (66.27% vs. 65.97%), collision rate (0.0060 vs. 0.0105), and SC (9.940 vs. 9.895). These results indicate that modeling CRNs as graphs provides a measurable advantage in allocation performance, and that combining graph-aware RL with metaheuristic optimization can improve robustness in dynamic spectrum access scenarios.