Physics-Informed Deep Learning for Human Action Recognition: A Biomechanical Approach

Abstract

Human action recognition systems traditionally rely on learning statistical patterns from visual data without explicit modeling of the physical laws governing human motion. This paper presents a physics-informed neural network architecture that integrates biomechanical modeling directly into the learning process. This approach computes kinematic features (joint angles) and kinetic features (torque, energy) from estimated poses and fuses them with visual motion features within a Transformer encoder. A multi-objective loss function encourages physically plausible representations by penalizing biomechanically infeasible poses and energetically unrealistic movements. Testing the proposed method in police traffic gesture recognition achieved 96.11% classification accuracy while maintaining biomechanical feasibility (0.998 average feasibility score). The integration of physics-based features enables the disambiguation of visually similar gestures through their underlying physical signatures. This approach produces interpretable physical measurements that can be validated against biomechanical principles, making it particularly suitable for safety-critical applications where model transparency is essential.