Comparative analysis of NLP-driven MCQ generators from text sources

Abstract

The application of learning sciences with technology has been shown to boost learner interactions, yet the potential of advanced tool, particularly those that leverage Natural Language Processing (NLP), still very much untapped in learning contexts. This paper speaks to this age-old problem of generating quality Multiple-Choice questions (MCQs) – a prevalent but time-consuming mode of assessment – via the suggested comprehensive comparison study of template-based AI solutions. The study contrasts general-purpose Large Language Models (LLMs) with specialized MCQ-focused AI programs. The scientific approach employed was quite stringent, where each of the software applications was benchmarked using a common dataset of text across varying levels of complexity and topic. Results indicate that general-purpose LLMs, especially DeepSeek and ChatGPT, consistently present higher performance and reliability, especially when processing complex textual content. Whereas specialized tools offer distinctive formatting options, they exhibit decreasing performance as texts become more complex and signify strong operation constriction at the free versions. Developing solid and effective distractors turned out to be a complicated task for all the tested tools. We conclude the paper by presenting a standardized assessment model, making evidence-based recommendations for developers and teachers, and suggesting ways to incorporate various AI capabilities into modern educational assessment effectively.