• Login
    View Item 
    •   Athenaeum Home
    • University of Georgia Theses and Dissertations
    • University of Georgia Theses and Dissertations
    • View Item
    •   Athenaeum Home
    • University of Georgia Theses and Dissertations
    • University of Georgia Theses and Dissertations
    • View Item
    JavaScript is disabled for your browser. Some features of this site may not work without it.

    Ratings of L2 oral performance in English

    Thumbnail
    Date
    2008-05
    Author
    Kang, Okim
    Metadata
    Show full item record
    Abstract
    Judgments of nonnative speaker (NNS) oral language performance are notoriously susceptible to rater biases. While acoustically measurable elements of pronunciation are indeed relevant to comprehensibility, variance in oral proficiency scores due to rater background and linguistic stereotypes constitute trait-irrelevant error. This study tested the supposition that raters’ background characteristics - including attitudes toward World Englishes - influence their rating of oral performances. In addition, a brief psychosocial intervention was undertaken in an attempt to mitigate the impact of potentially biasing rater characteristics. Seventy US undergraduate students rated the speaking and teaching proficiency of eleven international teaching assistants (ITAs). The ITA speech samples were acoustically analyzed on 12 measures of speech rate, pauses, stress, and intonation. Following the intial rating, one group of raters participated in a social-psychological intervention (training), in which undergraduates solved mystery puzzles with ITAs and interacted informally. Finally, 63 raters (including 29 trained) rated the same ITA speech samples a second time. All data collection from participant-raters was conducted online, including brief interviews as a qualitative supplement to the main results. Separate multiple regressions were employed to account for the variance in each dependent variable (rater judgment scores), based on linear combinations of independent variables (rater and speaker characteristics). Rasch modeling also yielded measures of rater stringency. Analysis results revealed that about 20-30 % of variance in proficiency ratings were attributable to rater background characteristics. Rater native English speaker status, amount of contact with NNSs, prior teaching experience, and negative past experience in ITA courses affected student judgments of ITAs’ accented speech. In contrast, 60-70% of the variance in ITAs’ oral performance ratings was attributable collectively to objectively measured prosodic pronunciation factors, especially acoustic fluency. The intercultural sensitization intervention mitigated the impact of rater biases on ratings of ITA instructional competence. Among recommendations for screening and training raters warranted by this study is the notion that students who feel that their class grades have been harmed by NNS instructors should be disqualified as raters in high stakes speech assessment. Conversely, NNSs who wish to improve their perceived oral proficiency should work on avoiding filled and irregularly placed pauses.
    URI
    http://purl.galileo.usg.edu/uga_etd/kang_okim_200805_phd
    http://hdl.handle.net/10724/24652
    Collections
    • University of Georgia Theses and Dissertations

    About Athenaeum | Contact Us | Send Feedback
     

     

    Browse

    All of AthenaeumCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

    My Account

    LoginRegister

    About Athenaeum | Contact Us | Send Feedback