A qualitative analysis of interactions between raters and trainers in a large scale writing assessment
Granade, Jeremy Brian
MetadataShow full item record
The purpose of this study was to investigate how interactions between rater-candidates and trainers during scheduled training conferences facilitate raters’ understanding and application of scoring guidelines. Because statewide writing assessments rely on the accuracy of approximately 50-60 novice raters per scoring session, it is essential that new raters receive the most helpful feedback and support from trainers. By focusing on five new raters as they completed the initial training program and begin to score student essays, I have identified common difficulties they encounter, documented strategies trainers utilize to deal with these difficulties, and suggested ways to better prepare trainers to lead effective training conferences. Findings are based on an inductive analysis of interviews with novice raters, and an applied conversation analysis of rater-trainer conference transcripts. The results of this study should be of interest to writing assessment trainers and the larger performance assessment research community.