Deep reinforcement learning for energy maximization in AWS wave energy systems with supercapacitor storage unit

Abstract

A deep deterministic policy gradient (DDPG)-based reinforcement learning (RL) controller is presented in this research to maximize the energy yield of the Archimedes Wave Swing (AWS) wave energy converter (WEC). This is the first application of the DDPG algorithm to the AWS WEC architecture integrated with a supercapacitor storage unit. Unlike discrete deep Q-learning (not suitable for continuous action) or on-policy methods such as soft actor-critic (SAC) and proximal policy optimization (PPO), which require higher computational resources. DDPG was chosen because it handles continuous action spaces, learns model-free optimal policies online, and adapts to irregular waves without requiring accurate excitation force prediction. By using real-time observations of wave excitation force, floater velocity, and position, a DDPG agent provides the value of quadrature axis current ( I q - r e f ) to control the generator damping across irregular wave conditions. The architecture incorporates a supercapacitor storage system coupled via a bidirectional DC-DC converter. This ensures that the AWS generator is supplied with reverse mechanical power whenever needed. The performance of the proposed method is benchmarked against established strategies such as approximate current control (ACC), feedback linearization control (FLC), and model-based control (MBC) under both simulated and real-world wave data. Simulations over 500 s demonstrate that under irregular sea states ( H s = 4 m and T p = 8 s ), the trained DDPG yields energy increases by 20% over MBC, 79% over FLC, 100% over ACC, and 156% over no control. Additional evaluations using real ocean data from Hawaii (NDBC Station 51003) confirm the DDPG agent’s superiority in 8 out of 12 months.