MARATTO

article

Harnessing Diversity and Novelty Metrics in Evaluating Course Recommender Systems via a Machine Learning Approach

In plain language

This research evaluates machine learning techniques to improve course recommendation systems within higher education institutions. The investigation examines k-Nearest Neighbours, Matrix Factorisation, and Neural Collaborative Filtering across three primary performance metrics: accuracy, novelty, and diversity. The analysis specifically focuses on understanding the trade-offs that occur when seeking to suggest a varied selection of courses, determining whether pursuing variety reduces overall accuracy. The findings demonstrate that Neural Collaborative Filtering achieves the strongest accuracy while maintaining a balance between novelty and diversity. Although gains in novel and diverse suggestions occur at particular cut-off thresholds, they introduce a measurable trade-off against recommendation accuracy. Overall, Neural Collaborative Filtering presents a viable framework for delivering both accurate and broad educational suggestions.

Key takeaways

  • Neural Collaborative Filtering achieved the highest accuracy among the evaluated models while balancing novelty and diversity.
  • Efforts to increase the novelty and diversity of course suggestions resulted in some reductions in accuracy at specific cut-offs.
  • Comparative analysis included k-Nearest Neighbours, Matrix Factorisation, and Neural Collaborative Filtering models.
  • Future investigations will explore recurrent architectures, such as Gated Recurrent Units, to further develop the recommendation system.

Why it matters

When digital learning platforms recommend courses, focusing purely on familiar choices can narrow a student's educational options. By measuring novelty and diversity alongside accuracy, this work demonstrates how institutions can balance precise recommendations with broader, unexpected subjects, helping learners discover new academic interests without sacrificing relevance.

Commercialisation angle

The research is relevant to educational technology providers and universities seeking to enhance student advising tools and learning management platforms. It remains at an experimental, algorithmic stage, demonstrating model trade-offs rather than a finished tool. Further testing with temporal models is required before real-world deployment.

AI-generated from the published abstract. Always read the original work before citing.

Abstract

This study analyses k-Nearest Neighbors, Matrix Factorisation, and Neural Collaborative Filtering (NCF) for developing a Course Recommendation System in higher education. Key metrics-Accuracy, Novelty, and Diversity-are used to evaluate trade-offs, particularly whether optimising for variety impacts Accuracy. Results indicate that N CF achieves the highest Accuracy while balancing Novelty and Diversity. Improvements in Novelty and Diversity are observed at specific cut-offs, though at some cost to Accuracy. These findings demonstrate the potential of NCF for balancing accuracy and variety in recommendations. Future work will explore algorithms like Gated Recurrent Units and Recurrent Neural Networks to enhance the system further.

Research topics

  • Online Learning and Analytics

Read the original research

This page summarises published work. The authoritative version sits with the publisher.

DOI: 10.1109/raai64504.2024.10949529

Is something wrong with this record? Report it or request removal.

Discussion

Discuss this research

Have you built on this work, tried to replicate it, or seen it applied in practice? Share what you know. Verified researchers and MARATTO™ domain experts can open a discussion, and any member can reply. Contributions are reviewed before they appear.

No discussion yet. Open the first thread.