Show simple item record

dc.contributor.authorRaczynski, Kevin Robert
dc.date.accessioned2015-04-17T04:30:18Z
dc.date.available2015-04-17T04:30:18Z
dc.date.issued2014-08
dc.identifier.otherraczynski_kevin_r_201408_phd
dc.identifier.urihttp://purl.galileo.usg.edu/uga_etd/raczynski_kevin_r_201408_phd
dc.identifier.urihttp://hdl.handle.net/10724/31279
dc.description.abstractThere exists a large body of research on the effectiveness of rater training methods in the Industrial and Organizational Psychology literature. Far less research has been done on the effectiveness of rater training methods in large-scale writing assessments. The purpose of this dissertation is to compare the effectiveness of two widely-used rater training methods—self-paced and collaborative frame-of-reference training—in the context of a large-scale, statewide writing assessment. Sixty-six raters were randomly assigned to the training methods. After training, all raters scored a common set of fifty representative essays. To determine raters’ accuracy on these essays, raters’ scores were compared to resolved expert scores and coded accurate (1) when the scores matched and inaccurate (0) otherwise. This approach was taken because over ninety-nine percent of these comparisons aligned either exactly or within one point. A series of logistic mixed models were then fitted to these binary data. Results suggested that the self-paced method was equivalent in effectiveness to the more time-intensive and costly collaborative method. Implications for large-scale writing assessments and suggestions for further research are discussed.
dc.languageeng
dc.publisheruga
dc.rightspublic
dc.subjectRater training
dc.subjectframe-of-reference training
dc.subjectself-paced
dc.subjectcollaborative
dc.subjectaccuracy
dc.subjectexperts
dc.subjectwriting assessment
dc.subjectlogistic mixed models
dc.subjectvalidity
dc.titleComparing the effectiveness of self-paced and collaborative frame-of-reference training on rater accuracy in a large-scale writing assessment
dc.typeDissertation
dc.description.degreePhD
dc.description.departmentEducational Psychology and Instructional Technology
dc.description.majorEducational Psychology
dc.description.advisorAllan Cohen
dc.description.committeeAllan Cohen
dc.description.committeeZhenqiu Lu
dc.description.committeeSeock-Ho Kim
dc.description.committeeShawn Glynn
dc.description.committeeGeorge Engelhard, Jr.


Files in this item

FilesSizeFormatView

There are no files associated with this item.

This item appears in the following Collection(s)

Show simple item record