Physics-informed hybrid reinforcement learning for estimating lithium-ion battery state of health

Abstract

Although Data-driven methods are becoming widely applied for estimating the state of health (SOH) of lithium-ion batteries, they often suffer from a lack of interpretability and generalization capabilities. To address these limitations, this research proposes a hybrid methodology that combines Long Short-Term Memory (LSTM) networks and Reinforcement Learning (RL) using Proximal Policy Optimization (PPO) to enhance both SOH prediction accuracy and model interpretability. The proposed methodology begins by training an LSTM model on key battery features for two different operational profiles, thereby capturing the temporal dependencies present in battery degradation. To improve interpretability and adaptive learning, a hybrid model is then introduced, where a PPO agent corrects the LSTM predictions based on a reward function designed to account for physical and monotonic degradation constraints of the SOH. Additionally, two models are trained separately for comparison: a pure RL-based model and an LSTM-based model, which enables comparison between data-driven and control-driven learning. The models are initially trained and validated on the NASA battery dataset and further evaluated on a separate real-world dataset of CALCE CS2 battery cells to assess robustness. Experimental results demonstrated that the hybrid model significantly enhances the robustness and accuracy of SOH prediction, outperforming both standalone LSTM and RL models. Statistical metrics, such as Mean Absolute Error (MAE) and Root Mean Square Error (RMSE), were reduced while [Formula: see text] score has increased significantly. At the same time, interpretable results are improved through an RL feedback mechanism grounded in physical behavior.