Designing Formative Adaptive Assessment for Engineering Education

Abstract

Assessing competencies in engineering education increasingly requires digital assessment approaches that support learning regulation, instructional decision-making, and educational quality, rather than focusing solely on measurement efficiency. Computerized adaptive testing (CAT), grounded in Item Response Theory (IRT), provides a robust methodological foundation for personalized assessment. However, its pedagogical effectiveness in formative contexts depends critically on curriculum alignment, diagnostic capacity, and adaptive control strategies. This study proposes and evaluates a formative adaptive assessment framework for engineering education that integrates an IRT-based CAT engine with a Bayesian network– based diagnostic component. The framework is designed to support competency-oriented feedback, learning monitoring, and instructional interpretation within a curriculum-aligned assessment structure. Assessment relies on dichotomous multiple-choice items explicitly aligned with engineering learning outcomes, while item selection dynamically adapts to learners’ evolving proficiency estimates. In parallel, probabilistic diagnostic modelling prioritizes under-assessed competencies throughout the adaptive process. Item calibration was conducted using empirical data collected from 612 university students in computer science, and system performance was examined through a simulation-based evaluation involving 500 simulated learners. Results demonstrate high estimation accuracy (r = 0.912) and satisfactory reliability for formative use across most learner profiles. Reduced precision at the extremes of the proficiency continuum and imbalances in item exposure were also observed, highlighting structural limitations primarily related to item bank coverage and curriculum representation rather than to the adaptive algorithms themselves. Overall, the proposed framework positions adaptive assessment as a pedagogically grounded tool for formative learning support, instructional decision-making, and quality assurance in engineering education.