Reliability of Students’ Teaching Practice Scores Using Generalizability Theory

Abstract

Purpose: The aim was to demonstrate the potency of using Generalisability Theory (GT) over Classical Test Theory (CTT) in reliability estimation. The objectives are to find the major source of error in the Intern Teaching Evaluation Form (ITEF) scores and to identify the optimum number of occasions of rating of teaching practice that would give the most reliable scores. Design/ Methodology/ Approach: A random effects one-facet fully crossed design in which intern (p) was fully crossed with occasion (o) was adopted for the study. In all, 9,082 bachelor’s degree teaching practice triplicate scores for three academic years from 2015/2016 to 2017/2018 were analysed in this study. A univariate generalisability analysis with EduG was performed to analyse data. Findings: The finding for relative interpretation, the scores were strongly reliable. For absolute interpretation, the scores were moderate to strongly dependable. The major source of error in the ITEF scores was the p x o interaction combined with unidentified sources. Research Limitation/Implications: A firm conclusion on one major source of error being a single facet in the ITEF scores could not be made. Practical Implication: The level of reliability and the optimum scoring design of the ITEF established by this study will aid in selecting the ITEF by teacher training institutions in Ghana for evaluation of teaching practice to train pre-service teachers for effective curriculum implementation to achieve national educational aims. Social Implication: Reliable evaluation data can inform policy decisions related to teacher certification, professional development, and educational standards. Policymakers can use this data to develop evidence-based policies that support teacher quality and student achievement. Originality/ Value: It has unearthed the psychometric properties and the ideal scoring design of the ITEF which were hitherto unknown.