Tracking gaze movement while construing and talking about events: a cross-linguistic approach
In this project, we deal with the interrelation between patterns in gaze movement, when watching dynamic video clips, and what is mentioned at what point, when talking about events. Most projects on the interrelation between visual and linguistic processing for production, and the factors that drive it, have centred on the production of single words when relating to objects in still pictures. These studies have shown that there is a tight sequential link between speakers' fixations on a specific entity and their reference to it. There are few studies, however, on gaze movement and patterns of mention in relation to the production of full length event descriptions, especially based on dynamic stimuli. We investigate this interrelation with respect to dynamic, live-recorded video clips, depicting causative events, in which an agent is in the process of making a specific object (e.g. knitting a scarf). We take a cross-linguistic approach and record gaze movement patterns before and during verbalization as the event unfolds over time.
We measure gaze allocation, time-locked to speech onset, in two areas of interest (AoI): the area where the agent is located and the area in which the entity acted upon is located ("action"). The cross-linguistic study of these processes shows how specific linguistic factors and their relevance for event construal, such as grammatical aspectual morphology and degree of specificity of reference to agents, affect the way in which events are visually processed, cognitively construed and talked about.