Gestures: How to Analyse Forms, Functions, and Meanings

The target audience is MA and PhD students from the fields of linguistics, communication studies, psychology, anthropology, and computer science who are working on, or interested in, the analysis of spoken language in conversation or in monologues, and who wish to expand their research horizons by using video of people talking as data. In addition, professionals from different fields may be interested in how a greater understanding of gestures could broaden their insights. This might include clinicians working in speech pathology and hearing impairment, therapists working in interaction analysis, communication consultants advising public speakers, and engineers and computer scientists working on computer vision analysis of video as big data and on the implementation of gestures in avatars or robots. We welcome participants from all countries and encourage you to apply what you will learn in the course to video recordings of people talking in whatever language you research. The working language for our communication in the course will be English. If you have doubts about your eligibility for the course, please contact us: [email protected]

If you are already using video recordings in your research, you can use these materials for your own project. You should just make sure in advance that you have permission to use the materials in this way (e.g sharing them with the instructors of the course, or potentially the other students, as needed).

A number of researchers have proposed that humans have been communicating multi-modally since prehistoric times, by making vocal sounds combined with hand gestures. Current possibilities for communicating long-distance with digital video have been bringing greater awareness to how powerful the visual side of human communication is. What are we missing if we analyse talk only based on audio recordings or written transcripts? This course will be oriented towards research skills for the analysis of video data of spoken interaction, focusing on how one can study manual gestures with speech. As an outcome, we will see how gesture functions in relation to speech, yet as a different semiotic system from it. The methods to be taught will show what gesture analysis can reveal on several levels, including details about the meanings that speakers are expressing, tacit social aspects of face-to-face interaction that are based on visually perceptible behaviour, and the dynamic nature of human communication and meaning-making.

Topics to be covered include: how to study speakers’ hand movements in terms of units of analysis and in terms of the forms these gestures take, how speakers refer to ideas through pointing and depiction, how manual gestures qualify what speakers say by indicating emphasis and different kinds of stance (perspectives), and how gestures fall along with a range from more to less dependent on speech. 

The course will involve lectures as well as data analysis sessions each day to apply what has been covered in the lectures and in the readings. In addition, some guest lectures by internationally renowned scholars from different sub-fields of gesture studies will provide insights into how the research can be applied in different directions.

By the end of the course, the participants will have:

• An overview of the added value to research that comes from studying manual gestures with speech.
• Insight into ways in which gesture relates to spoken language.
• An understanding of concepts and categories used in gesture studies, including gesture forms and functions.
• Experience with methods of gesture analysis, including gesture annotation using the software ELAN.
• Ideas about how gesture research can be applied in different kinds of research projects.

Examples of possible readings include the following; A digital reader with the texts for the course will be made available. 

1. Introduction to gesture studies; the nature of gesture; overview of gesture types
   - Müller, C., Ladewig, S.H. & Bressem, J. (2013). Gestures and speech from a linguistic perspective: A new field and its history.
   - Kendon, A. (2013). Exploring the utterance roles of visible bodily action.
2. What’s in a gesture? Gesture units and form features
   - Kendon, A. (2004). Gesture units, gesture phrases, and speech.
   - Bressem, J. (2013). A linguistic perspective on the notation of form features in gestures.
3. How do gestures refer to ideas? Deixis, depiction, and semantics
   - Kendon, A. (2004). On pointing.
   - Streeck, J. (2009). Depiction.
4. How do gestures qualify what we say? Stance, emphasis, and pragmatics
   - Cienki, A. (2017). Gesture and pragmatics: From paralinguistic to variably linguistic.
   - Bressem, J. & Müller, C. (2014). A repertoire of German recurrent gestures with pragmatic functions.
5. Putting it all together: Gestures along a continuum, from more to less dependent on speech
   - McNeill, D. (2005). Why gestures?
   - Poggi, I. (2014). Semantics and pragmatics of symbolic gestures.

Alan Cienki is a Professor at the Vrije Universiteit Amsterdam in the areas of language use and cognition and English linguistics. He devised the English-taught MA program at the VU on Multimodal Communication (under Communication and Information Studies). He also founded and directs the Multimodal Communication and Cognition Lab (known as "PoliMod" in Russian) at Moscow State Linguistic University. His research concerns the area where cognitive linguistics overlaps with gesture studies, including the study of metaphor, grammar, semantics, and pragmatics. Further information about his work can be found here  https://research.vu.nl/en/persons/alan-cienki. 

Olga Iriskhanova, is a Professor of general and comparative linguistics at Moscow State Linguistic University in addition to being Director of the Centre for Socio-Cognitive Discourse Studies and Vice Rector for Research at the university. She is also a Senior Researcher at the Russian Academy of Sciences Institute of Linguistics. Her areas of research include theoretical linguistics, multimodal communication, discourse analysis, linguistic creativity, and gesture studies. More information can be found here  http://scodis.com.

In addition, the course will be assisted by some of our PhD candidates who are highly experienced in gesture research, having published articles and given conference talks in this field.