SYCORAX

Abstract

This dissertation describes the development and evaluation of a new analyzer for syntactic complexity known as SYCORAX: the SYntactic COmplexity RAting eXpert.

SYCORAX, like several prior applications for automated analysis of syntactic complexity (e.g., Long et al., 2006; Channell, 2007; MacWhinney, 2011), is based on the Developmental Sentence Scoring (DSS) scale developed by Lee (1974). These existing applications, however, all have one significant limitation in common: they are all strictly based on immediate linear context within a sentence. It is evident from prior work (Channell, 2003; Judson, 2006) that certain syntactic structures involved in DSS are simply not apparent from linear context alone; indeed, many structures incorrectly analyzed by human raters are due to incorrect interpretation of local context (Lively, 1984).

In contrast, SYCORAX incorporates a newly-developed shallow dependency parser known as JED (Just Enough Dependency) optimized for the dependencies which are important in DSS, and uses the resulting parse tree in the calculation of its DSS scores. Even without complete optimization of its DSS rules, the use of shallow parsing in SYCORAX produces a distinct overall boost in the accuracy of syntactic complexity scores on a variety of manually-scored real-world transcripts, as measured using Pearson correlation coefficient and point-by-point accuracy, with no significant increase in execution time.

DSS has proven numerous times to be psycholinguistically valid. It was originally designed to identify language delays in children, and more recent experiments have found it to still be valid in that respect (e.g., Scarborough, 1990); in addition, it has been found to be of use in identifying language decline in adults (Cheung and Kemper, 1992; Kemper et al., 2003, 2004), distinguishing different forms of developmental delay (Finestack and Abbeduto, 2010), and even identifying Alzheimer’s dementia (Kemper et al., 1993). It is believed that the improvement in its automated analysis by SYCORAX will prompt even further research regarding it, much as the prior project CPIDR (Brown et al., 2008) has done for semantic complexity (Covington et al., 2009; Jarrold et al., 2010; Engelman et al., 2010; Tsai, 2010).