Enhancing Garment Sewing Employee Added Value Monitoring Using a Video-Based Spatio-Temporal Approach Through Computer Vision

Abstract

In the garment sewing industry, tracking worker activity is crucial for measuring production efficiency, particularly differentiating between value-added tasks (sewing, handling garments) and non-value-added activities (equipment maintenance, waiting). While traditional manual tracking methods struggle with scale and accuracy, recent efforts using deep learning, especially single-frame prediction, have aimed to address this challenge. However, these methods struggle to capture the flow of worker activity. By treating each frame independently, they miss the crucial element of time-dependent actions that define real-world work, leading to unreliable results. To overcome these limitations, this paper proposes an approach that considers both spatial (worker pose in a frame) and temporal information (sequence of worker actions) for robust worker activity recognition. The study compares the accuracy of different approaches in classifying tasks as value-added or non-value-added. This includes evaluating keypoint extraction methods (MoveNet, MediaPipe with LSTMs) against video classification using whole frames with convolutional neural networks (CNNs) including 3D convolutional layers (Conv3D), and video transformers (VideoMAE). Resulting in a 30% increase in accuracy when compared to keypoint extraction method.