Enoch O. Oladunmoye

Reliability in Psychological Measurement

Reliability is among the most frequently reported psychometric properties of psychological and educational measurement instruments, yet it remains one of the most frequently misunderstood. Investigators routinely cite Cronbach's alpha as evidence that a questionnaire is reliable, at times applying a single numerical threshold as though it were a universal decision rule. Such practice can obscure important distinctions among internal consistency, temporal stability, inter-rater agreement, measurement error and other forms of reliability evidence, and it neglects the fact that reliability is conditional on the scores, the population, the measurement conditions and the intended interpretation. This Research Note provides an expanded, structured treatment of reliability in psychological measurement. It sets out the classical test theory model of observed scores, examines internal consistency, test-retest reliability, inter-rater reliability and split-half reliability, and critically appraises the limitations of Cronbach's alpha. It explains why alpha should not be treated as a universal certificate of scale quality and introduces McDonald's omega as a model-based alternative that is often more defensible under congeneric measurement conditions. The Note further discusses the relationship between reliability and validity, the effects of scale length and inter-item covariance on alpha, multidimensionality, generalisability theory, confidence intervals around reliability estimates, and the necessity of estimating reliability in the actual sample under study rather than relying on figures reported in the original validation work. Particular attention is paid to reliability evidence in cross-cultural and African research contexts. A practical, stage-by-stage reliability-analysis workflow using PsychtrixWeb is proposed, alongside guidance on reporting standards and a catalogue of common reliability errors. The overriding principle advanced throughout is that researchers should select the reliability evidence that corresponds to the specific measurement claim being made, rather than defaulting to a single coefficient by convention. Keywords: reliability; Cronbach's alpha; McDonald's omega; internal consistency; measurement error; test-retest reliability; inter-rater reliability; generalisability theory; psychometrics; scale validation; PsychtrixWeb