Meta-analysis on the influence of AI agents on K-12 student cognitive performance

Abstract

Following the rapid evolution of AI in education, AI agents are being integrated into K-12 classrooms. However, their effectiveness in students’ cognitive enhancement remains inconsistent across studies. This study conducted a meta-analysis to evaluate the effectiveness of AI-agents on students’ cognitive learning outcomes in K-12 education, and their impact on specific cognitive categories, while also highlighting the influence of moderator variables, such as the type of AI agents, learner level, discipline, and experimental duration. A random-effect meta-analytic approach was employed, synthesizing 73 effect sizes from 34 studies published between 2020 and 2025. The main effect size was calculated using Hedge’s g, while the heterogeneity was assessed using the Q, I 2 , and τ 2 statistics. Publication bias was tested using the funnel plot, while the Classic fail-safe N test, Orwin’s fail-safe N test, and trim-and-fill methods were used to assess the robustness of findings. The results demonstrate that: 1) AI agents have a statistically significant, moderate positive effect on students’ cognitive learning outcomes (with an overall effect size of 0.404, p < 0.001); 2) Across cognitive categories, AI agents showed a moderate significant impact on skills-based outcomes (g = 0.391, p < 0.001) and knowledge-based outcomes (g = 0.344, p < 0.05); 3) For higher-order thinking skills, the impact was insignificant (g = 0.540, p = 0.066); 4) There was a significant moderating effect of learner level and discipline under the intervention; 5) The publication bias test showed a slight study effect, whereas sensitivity analysis confirmed the robustness of results. The findings indicate the need to consider both the disciplinary context and students’ characteristics when integrating AI agents in K-12 classrooms. The results provide direction to educators, policymakers, and researchers on the effectiveness of AI-agents on students’ cognitive learning outcomes in K-12 education. • AI agents significantly boost K-12 students' cognitive learning outcomes. • Strongest benefits for skill development and higher-order thinking. • Effectiveness depends on AI type, grade level, and subject area. • Supports tailored implementation strategies for optimal impact.