What can face and gaze tell us about language use in interaction?

Federico Rossano

Max Planck Institute for Psycholinguistics, The Netherlands


Gaze, questions and normativity: using visible behavior to project, align with and close courses of action Federico Rossano
Exchanging glances while talking and driving Lorenza Mondada
Facial expression as securing shared understanding in sequences involving stance Johanna Ruusuvuori, Anssi Peräkylä
Micro-ecology and macro-functions of gaze in workplace interaction Jürgen Streeck

This panel focuses on the contribution of gaze behavior and facial expressions to the understanding and use of language in face-to-face interaction. It brings together researchers from five different institutions working on conversational data in four different languages (Finnish, American English, French and Italian). All papers rely on video recorded data and use conversation analysis as the primary methodology. By looking at how different visible practices are embedded within courses of action we document some of the resources people rely on to parse actions (whether visible or verbal) into meaningful chunks.

Most of the work on visible behavior in relation to talk has focused on hand gestures and head movements, while we will look at how gaze behavior and facial expressions by the participants are affected and affect the development of a conversation and the competing activities the participants are locally involved in. In particular, Peräkylä and Ruusuvuori will analyze ordinary conversations in Finnish focusing on the use of facial expressions in assessing stories and topics and on the ways in which face is integrated with the lexical content of spoken utterances. Streeck will present a case study of a car-mechanic's gaze behaviors and how they are responsive and relevant to his current involvement in face-to-face interaction and his overall monitoring of the goings-on at his shop. Mondada will focus on the distribution of glances during conversations held in cars between one or more passengers and a driver and will show how their multimodal conducts remains sensitive to the sequential organization of talk in a situation characterized by a multiplicity of foci of attention (on driving and on talking). Rossano will look at the relationship between the deployment of gaze during questions, the specific actions that these questions implement and the likelihood and timing of their responses and will show the impact of the absence of mutual gaze on the natural flowing of the interaction.

All these papers intend to propose the inclusion of the face and the eyes in the analytical picture of sociolinguistics by showing how the multimodal behavior of the participants has an impact on the behavior of the interactants and is locally but also predictably managed during a conversation. The emphasis will therefore be on how the subtleties of gaze and facial expressions are related to what people do with their talk. The panel papers will also show how these subtleties need to be taken into account to comprehend the macro management of perceptually complex activities like driving or fixing a car or eating. This panel will show how micro changes in our gaze directions, for example, influences the responses of another participant in the conversation. Moreover, by presenting papers from different cultures and different languages, this panel shows the pervasive importance of visible behavior in the conduct of any face-to-face interaction, the natural home of language use for human beings.

Among the issues that will be discussed, the following will be particularly relevant:

1) What is the benefit of including visible behavior in the analysis of language use?

2) What are the units of analysis for gaze and facial expressions?

3) Should they be emic or etic?

4) What is the relation between the physical context and in particular the material world and the act of talking to each other? Is it a either/or (either we talk or we pay attention to the rest of the world) or is there a constant negotiation of local priorities?

5) Is visible behavior in interaction an adaptation to what happens in the talk or is the former influencing the very interpretation and understanding of the latter?

6) If people are constantly involved in different concurrent activities, like talking and eating and watching tv, or talking while driving etc., how is their language use affected by them?