Designing field tests for multidimensional classification models

Abstract

Diagnostic classification models (DCMs) are multidimensional latent variable models that can provide diagnostic information about the mastery state of examinees’ knowledge components (Rupp, Templin & Henson, 2010). DCMs classify examinees based on specified knowledge components, and provide multidimensional feedback about examinees’ strength and weaknesses. However, recent large-scale assessments have not designed to diagnose, and few practical applications of DCMs exists. Creating multidimensional assessments is needed to meet demands of more detailed feedback. It introduces new challenges to educational assessment research. Field testing is an essential step in creating assessments. Field testing items for unidimensional vs. multidimensional assessments are not the same. Different field test designs result in sparse data, and research has not studied sparse data conditions for DCMs. I investigate the impact of sparse data, due to different field test designs, on the estimation accuracy for DCMs. Results provide needed guidelines for designing DCM-based field tests.