Aditi Kaveti ’23

Communication between conversation partners can be performed in a multitude of ways, including spatial, visual, linguistic, aural, and gestural. These different modalities can affect the way information is transferred and interpreted, as well as the way the information is later recalled and referenced. Above all, communication between partners establishes a common ground between the two as they create a shared experience. Dr. Susan Brennan is a professor of cognitive science at Stony Brook University that studies the psychology of language use, particularly interactive spoken dialogue. 

In a study published in the journal Cognition, Dr. Brennan determined whether the modality with which speakers convey information to their conversational partners leads to the way subsequent references are made towards that particular partner. The research was conducted by examining 32 groups of three participants and using a set of tangram pattern cards. The cards were shared in one of the two modalities, either linguistically or visually, by both modalities, or by none at all. The cards shared linguistically were described, but not shown to the participants, and the cards shared only visually were silently shown to the participant. The team of researchers then rearranged the cards in separate rounds with each participant.

The data collected from this study showed that the modality with which the card was previously presented to them affected the way the participants referred to the specific card. For example, if the card was only presented to the participant in a visual manner, it was then referenced using idea units, words, and reconceptualizations as opposed to cards that were only presented in a linguistic manner. The way the participants referenced the cards depends on what modality they were presented in shows that speakers are able to tailor references to shared experiences with the specific original modality. Dr. Brennan concluded that the memory of the speaker encodes rich-enough representations of common ground between speakers and addressees that effectively cue constraint about the multimodal perceptual condition under which two partners communicate their shared experiences.

References:

1. A. Galati, S. Brennan, What is retained about common ground? distinct effects of linguistic and visual co-presence, Cognition, 215, (2021), doi: 10.1016/j.cognition.2021.104809.
2. Image retrieved from: https://www.maxpixel.net/Head-Talking-Interview-Conversation-People-2004339