A tiny inertial transformer for human activity recognition via multimodal knowledge distillation and explainable AI

Abstract

Human activity recognition (HAR) is essential for applications such as healthcare monitoring, fitness tracking, and smart environments, yet deploying accurate and interpretable models on resource-constrained devices remains challenging. In this paper, we propose XTinyHAR, a lightweight, transformer-based unimodal framework trained via cross-modal knowledge distillation from a multimodal teacher. Our model incorporates temporal positional embeddings and attention rollout to enhance sequential feature extraction and interpretability. Evaluated on UTD-MHAD and MM-Fit, XTinyHAR achieves test accuracies of 98.71% and 98.55% with F1-scores that match these results and Cohen's Kappa above 0.98. The model remains lightweight (2.45 MB) with fast inference (3.1 ms on CPU, 1.2 ms on GPU) and low computational cost (11.3M FLOPs). Extensive ablation studies confirm the contribution of each component, and subject-wise evaluations demonstrate strong generalization across users. These results highlight XTinyHAR's potential as a high-performance, interpretable, and deployable solution for real-time HAR on edge devices. Our codes are available at: https://github.com/Ism-ail11/XTinyHAR.