Instantiating thematic roles with a recurrent neural network

Abstract

Human languages have both a surface structure and a deep structure. The surface structure of a language can be described by grammatical rules which generate its well-formed sentences. Deep structure can be described by a set of thematic roles which specify the meaning of a sentence independent of any particular language. In this paper, I show that a recurrent neural network can learn to map English sentences generated by a unification-based grammar onto their appropriate thematic roles. Two models are evaluated: (1) a simple recurrent network like those of Jordan (1986) and Elman (1990), and (2) a modified recurrent network which incorporates a time-varying training signal, direct connections between its input and output units, and two separate state layers. It is found that the modified network learns faster and more completely than the simple recurrent network.