Multilevel detection of possible test tampering through erasure analysis

Abstract

Cheating on high-stakes standardized tests, especially by educators to meet accountability requirement, becomes a widespread serious problem receiving more and more attention. In the limited literature of statistical detection of this test security violation, erasures analysis has the potential to serve as useful data forensic tool. This dissertation developed and compared several methods of erasure analysis for tests with dichotomously scored multiple-choice items in order to identify potential tampering at individual and groups levels. A large-scale grade 8 reading test data set was used to explore characteristics and interrelationships among existing and proposed detection methods. Based this real data set, simulated data sets were generated, containing erasures due to random answer changes, misalignment, speededness, and tampering. Type I error and power rates of different methods were evaluated across simulation settings different in strategies of making illegal wrong-to-right erasures and in numbers of involved examinees and groups.