3D Landmarks and Dynamic Emotions for Classroom Engagement Analysis: A Unified GNN-Transformer Model with FLAME Blendshapes

Abstract

Student engagement analysis remains pivotal for enhancing educational outcomes, yet existing methods struggle with holistic assessment in classroom environments. This paper introduces a unified GNN-Transformer architecture leveraging FLAME 3D facial landmarks and dynamic emotion modeling for multidimensional engagement analysis. Our approach processes 5,023 vertices from FLAME blendshapes through graph attention networks to capture spatial relationships, while transformer encoders model temporal engagement dynamics. A novel cross-modal attention mechanism fuses spatial-temporal representations to jointly predict behavioral engagement, affective states, and cognitive load. The central idea introduced in this work is the development of a deep learning pipeline that integrates FLAME-based 3D facial dynamics with a joint GNN-Transformer and cross-modal attention framework to enable accurate, robust, and simultaneous, physiologically-grounded prediction of multiple diverse student engagement dimensions. Validated on the DAD-3DHeads dataset, our method demonstrates strong generalization and performance in real-world classroom scenarios, offering significant improvements over conventional 2D approaches through its physiologically-grounded 3D representation.