article · Assessment & Evaluation in Higher Education
Large higher education classes face substantial constraints in delivering effective feedback, which is crucial within competency-based learning. An investigation explored integrating artificial intelligence to improve feedback delivery by designing a custom prompt for GPT-4 hosted in a no-code web application. The tool was developed to provide automated feedback to second-year accounting students answering essay or discussion questions in an intermediate accounting course in South Africa. Designed to reflect principles of effective feedback, the system underwent a pilot evaluation. Findings revealed that the generated feedback generally aligned with established feedback principles, although performance varied across different evaluation dimensions. Consistent application of pedagogical best practices remains difficult to achieve, indicating that while large language models can support and supplement traditional evaluation, rigorous oversight remains vital.
Marking written assignments and delivering detailed feedback to large student cohorts requires heavy time and resources. Demonstrating that artificial intelligence can provide pedagogically aligned feedback helps educators manage heavy teaching loads in competency-based programmes. However, the observed variability demonstrates why universities must maintain structured oversight rather than relying entirely on automated evaluation systems.
The work demonstrates an applied, pilot-tested application using a no-code interface integrated with large language models. The primary users are higher education instructors and institutions managing large student cohorts with written assessments. Because the system was tested only in a pilot evaluation and exhibited variability across dimensions, the technology remains at an applied testing stage that still demands direct human supervision before any broad commercial or institutional deployment.
AI-generated from the published abstract. Always read the original work before citing.
Providing effective feedback in large settings presents significant challenges due to time and resource constraints. Given the importance of \nfeedback in competency-based higher education, innovative solutions are \nessential. This study explores the integration of artificial intelligence (AI) \nto enhance feedback delivery. The research focuses on the development \nof a custom prompt for GPT-4 within a no-code web application, designed \nto deliver AI-generated feedback to second-year accounting students on \ndiscussion or essay-style questions in a large competency-based intermediate accounting course in South Africa. The prompt aligns with the principles of effective feedback and was tested in a pilot evaluation. Results \nindicated that the AI-generated feedback generally adhered to these \nprinciples, though some variability was observed across the different \nfeedback dimensions. While challenges remain in consistently guiding AI \nto apply pedagogical best practices, the findings suggest that large language models can complement traditional feedback. Rigorous oversight \nof AI-generated feedback remains critical.
This page summarises published work. The authoritative version sits with the publisher.
DOI: 10.1080/02602938.2024.2415649
Is something wrong with this record? Report it or request removal.
Discussion
Have you built on this work, tried to replicate it, or seen it applied in practice? Share what you know. Verified researchers and MARATTO™ domain experts can open a discussion, and any member can reply. Contributions are reviewed before they appear.
No discussion yet. Open the first thread.
New to MARATTO™? Create a free account.
