Issues inherent in detecting attribute-level DIF using item-level methods
Nixon, Casey Boone
MetadataShow full item record
Differential item functioning (DIF) is generally investigated with methods that identify DIF at the item level, though it can also exist at the attribute level. When items with attribute-level DIF are examined with an item-level DIF detection technique, a number of issues may arise. The current research illustrates the utility and advantages of a model that decomposes items into attributes in order to model item difficulty and better examine DIF. To gauge the need for a model such as the Random Effects Linear Logistic Test Model (LLTM-R) in detecting DIF simulated to exist at the attribute level, a simulation study is conducted. Due to its widespread use and popularity, the Mantel-Haenszel DIF-detection procedure is included as a representative item-level method. The sample size, density of the Q-matrix, magnitude of DIF, and magnitude of canceling attribute weights were manipulated. The impact of these variables on power, Type I error rates, and accuracy of the item-level DIF detection technique was investigated. Samples were drawn from a normal distribution and 500 replications of each of the 45 cells were obtained. Findings from the simulations indicated that when DIF was simulated to exist at the attribute level, an item-level DIF detection technique performed inconsistenly in detecting the DIF. Results support that attribute-level DIF must be investigated using an attribute-level DIF detection procedure, using a model such as the LLTM-R. A demonstration presents how to investigate attribute-level DIF using the LLTM-R, and the impact of misspecification of the Q-matrix on the parameter estimates and standard errors. It is concluded that the use of item-level DIF detection techniques is inappropriate for the identification of attribute-level DIF, and the LLTM-R is recommended as a model that provides useful information about the presence and potential causes of such DIF. Suggestions for researchers are presented as a multi-step approach for investigating DIF that may exist at both the level of the item and attribute.