TimeSformer-MIL: A Hybrid Approach for Anomalous Activity Recognition in Real-World Surveillance Videos

Abstract

Anomaly detection in surveillance videos is vital for public safety. This paper introduces TimeSformer-MIL, a hybrid approach combining TimeSformer with Multiple Instance Learning (MIL) to identify anomalous activities. TimeSformer captures spatio-temporal features and highlights significant segments, while MIL uses video-level labels, treating videos as bags of segments. The model learns a deep anomaly ranking with sparsity and temporal smoothness constraints. Evaluated on a subset of the UCF-Crime dataset, TimeSformer-MIL shows significant improvements over three recent deep learning baselines, demonstrating its effectiveness in assessing surveillance footage.