Language-induced event-representation: competition and multiple object instantiation (Dr Yuki Kamide & Dr Anue Kukona)

When we hear a sentence describing an event in which an object changes its location, we have to keep in memory those changes. On hearing, for example, that a woman got a wine glass down from a cupboard, put it on a table, and then filled it with wine, we need to represent the glass before it was placed on the table (i.e. when it was in the cupboard, empty), and after (when it was on the table, and full) - we need to keep in memory multiple 'instantiations' of the same glass, corresponding to the different 'versions' of the glass before and after. This is a fundamental aspect of our understanding of the events and changes that occur around us in the world within which we live.

Our research asks a question which, surprisingly, has been over-looked in cognitive psychology: What are the implications of having to maintain these distinct instantiations of the same representation? We seem to do so effortlessly, but the reason that this is a pertinent question is that many years' research have shown that maintaining representations in memory that are very similar to one another is not as easy as maintaining representations that are less similar. Similarly, retrieving from memory a representation that is more similar to other representations in memory is ‘harder’ and more ‘costly’. The cost here is in terms of confusability (one might retrieve the wrong representation) and in the speed with which one can retrieve the correct one from memory. Given these facts about human memory, what then are the implications of having to maintain, or retrieve from amongst, multiple instantiations of the same object? Are there similar costs?

The implications of finding such costs (and we have preliminary data suggesting that there are indeed such costs, although these are based on a single pilot study) are profound. For example, there are phenomena observed in childhood, concerned with children’s ability to represent different perspectives on the same objects (e.g. knowing where an object is, and knowing where someone else *thinks* it is) that could be explained in terms of the costs of maintaining these multiple object instantiations in memory.

In our research, we use a range of different techniques that will ask (a) whether we observe costs associated with multiple object instantiation across a range of different tasks; (b) how these costs manifest (accuracy of recall, speed of retrieval from memory (in collaboration with Prof Brian McElree, New York University), or how ‘accessible’ the representations are); (c) whether we can identify specific brain regions which might be implicated in maintaining and keeping distinct these otherwise very similar representations. Thus we use a range of methods, including studying eye movements as participants view scenes and hear descriptions of events pertaining to those scenes; studying decision latencies in a variant of memory tasks which contrast speed and accuracy of retrieval, and functional magnetic resonance imaging (fMRI brain scans – in collaboration with Prof Sharon Thompson-Shill, University of Pennsylvania).

Our exploration of these phenomena have implications for theories of cognitive representation (how we represent the external world in our minds) and cognitive development in childhood (including children’s ability to maintain multiple representations of the same object from different perspectives). It also have implications for how we conceive of the relationship between language (as used to describe events in the external world) and vision (through which we directly perceived those events).

Collaborators::
Prof Gerry Altmann (York), Dr Gitte Joergensen (York)

Funding Source:
Economic and Social Research Council (ESRC), 2011-14
(£730,113 (FEC) jointly with York)