Reinforcement Learning for Imbalanced Data in Robotic Anomaly Detection within Autonomous Manufacturing

Abstract

Ensuring reliable anomaly detection in industrial robots is critical for safe and autonomous manufacturing operations. However, it remains challenging due to temporal dependencies and class imbalance in sensor data. This study presents a reinforcement learning approach using Deep Q-Network (DQN) enhanced with Long Short-Term Memory (LSTM) and Gradient Boosting Machine (GBM) for robust anomaly detection in robotic systems. The proposed framework integrates an LSTM into the DQN policy to capture temporal patterns. It also introduces a novel GBM-based reward mechanism that mitigates class imbalance by applying SMOTE (Synthetic Minority Over-sampling Technique) after removing temporal dependencies. Experimental results demonstrate that this hybrid DQN-GBM framework achieves superior performance in precision, recall, and F1-score compared to standalone DQN and DQN-LSTM variants. Beyond technical improvements, this approach enables truly autonomous manufacturing environments by providing adaptive, real-time anomaly detection that reduces human intervention and prevents costly production failures, ultimately contributing to more resilient and self-optimizing industrial systems.