Comparability of covariance structures and accuracy of information criteria in mixed model methods for longitudinal data analysis
Yanosky, Daniel J.
MetadataShow full item record
Modern mixed model methods for analyzing longitudinal data require researchers to select a covariance structure for the data to fully specify the model and obtain statistical tests of the fixed effects. The current study is a Monte Carlo simulation with primary purposes to 1) identify surrogate covariance structures for seven known models and estimate the severity of committing an error in covariance specification in terms of empirical Type I error rates and statistical power, 2) estimate accuracy rates of five information criteria in selecting appropriate covariance structures, and 3) estimate the empirical Type I error rates and power for models chosen by each information criterion. Data were generated corresponding to a single group repeated measures design with N = 10, 30, or 60 subjects and a quantitative response variable measured over t = 3 or 6 occasions. Other salient variables included the magnitude of serial correlation, presence of non-constant variance, and so forth. Data were generated under 72 conditions with 10,000 replications per condition. Statistical Analysis System (SAS) version 9.1 and R version 2.4.0 were used to generate and analyze the data. A preliminary investigation demonstrated that the Kenward-Roger degrees of freedom approximation yields F-tests for the mixed models with superior Type I error control compared to the Between/Within method, Satterthwaite approximation, and the sandwich estimator. Results corresponding to the primary research questions demonstrated 1) seven covariance structures were found to be acceptable approximations of a given true model in 14 instances, 2) rates of selecting appropriate covariance structures for information criteria were found to be substantially influenced by accounting for surrogate structures with a rate of 69% for both AIC and BIC, 3) empirical Type I error rates were found to be slightly conservative and therefore well controlled and power estimates comparable when models were selected by AIC, AICC, HQIC, BIC, and CAIC. Secondary investigations compared the performance of mixed models with classical methods and evaluated the empirical Type I error control of the Group x Time interaction test. The implications of these findings are discussed and heuristics for applied researchers working with this variety of data are suggested.