Impact of ChatGPT on learning outcomes and performance of students in computer programming courses: a mixed-method approach

Abstract

The rapid post-2022 integration of generative AI tools, such as ChatGPT, into Nigerian higher education necessitates an empirical evaluation of their impact on programming courses. This study examines the impact of ChatGPT on student learning outcomes and performance, addressing concerns related to critical thinking and academic integrity. We combined student and instructor surveys with course performance data using a mixed-methods approach within a mandatory undergraduate computer programming course in Nigeria (N = 855, an analytic sample derived from 1889 enrolled students).= Statistical analyses, including correlation and regression, examined relationships between ChatGPT usage, student perceptions (enjoyment, confidence, motivation, usefulness), and final scores. Findings reveal a significant divergence: students reported positive perceptions of ChatGPT in terms of affective benefits (e.g., increased confidence and motivation) and task-specific utility (e.g., debugging assistance); however, these perceptions did not correlate with improved academic performance. Regression results indicated that ChatGPT usage and related perceptions had limited predictive power (R2 = 0.006; no predictors were significant). Instructor observations corroborated concerns about over-reliance on AI-generated solutions and potential academic integrity issues. This research underscores the complex, nuanced impact of ChatGPT, highlighting that perceived benefits do not automatically translate into enhanced learning outcomes, and emphasizing the need for careful pedagogical integration strategies. Limitations include the study's single institution design, cross-sectional nature, and reliance on self-report data.