Personalized Evaluation of Online Predictive Systems through Learner Profiles

Abstract

Distance education institutions continue to face high rates of failure and dropout each year, mainly due to the autonomy of the learner and the absence of continuous monitoring. Numerous predictive systems have been developed to identify students at risk; however, their evaluation often relies on global performance metrics that overlook the heterogeneity of learner profiles. This paper introduces a personalized evaluation methodology for predictive models, applied to real data from Loghate, an online multilingual learning platform implemented at Hassan II University of Casablanca to promote language acquisition among students. After thorough preprocessing, engagement and performance indicators were extracted, and learners were clustered into homogeneous profiles using the K-means algorithm. A stacking ensemble model, combining several supervised learning techniques, was then trained and assessed both globally and within each profile. The model achieved strong global performance (82% accuracy), yet clear disparities appeared across learner profiles, with one group reaching 88% accuracy while another dropped to 75%. These variations show that global metrics can mask subgroup-specific biases and reduce the reliability of early warning systems. This approach reinforces the need for differentiated evaluation to build fairer and more reliable predictive systems that adapt interventions to learners' needs. In the long term, it could support the development of adaptive and explainable dashboards for more personalized and equitable learning environments.