A neuro-symbolic reasoning for affective-aware personalization in virtual learning environments

Abstract

Purpose This paper aims to introduce a neuro-symbolic affect-aware learning agent designed to optimize learner engagement and knowledge retention in virtual learning environments (VLEs). Design/methodology/approach The proposed system integrates deep neural networks for multimodal emotion recognition (facial, textual and auditory inputs) with a rule-based symbolic reasoning engine that adapts instructional delivery based on detected affective states. Emotion detection was achieved using a hybrid pipeline comprising a ResNet-50 model (trained on AffectNet for facial cues), fine-tuned BERT (on GoEmotions for textual cues) and wav2vec2.0 (on IEMOCAP for speech signals). To evaluate pedagogical effectiveness, a controlled experiment was conducted with 80 participants divided into three groups: a control group, a neural-only agent group and the proposed neuro-symbolic agent group. Learner engagement was quantified using the User Engagement Scale (UES), and learning outcomes were measured using normalized pre-test/post-test gain scores. Findings Results indicate that the neuro-symbolic agent outperformed the baseline by 16.8% in engagement and 21.3% in learning gain, demonstrating the benefits of emotionally adaptive and context-aware instruction. Research limitations/implications The study was conducted with a limited sample size (80 participants) and focused on short-term engagement and learning outcomes. Further research is required to assess long-term effectiveness and generalizability across diverse educational contexts. Social implications The proposed framework highlights the potential of affect-aware, neuro-symbolic systems to enhance learner engagement, promote self-regulated learning and support personalized instruction in VLEs, contributing to more empathetic and human-centered digital education. Originality/value This work presents a novel integration of multimodal emotion recognition with symbolic reasoning for real-time, pedagogically adaptive learning, offering a transparent, interpretable and emotionally responsive approach to VLEs.