S15: MULTIMODALITY AS TECHNOLOGY: THE CHALLENGE OF CAPTURING AND ANALYSING GESTURE IN A MOBILE LEARNING ENVIRONMENT – HELEN LEE

Studies in CMC which adopt a multimodal approach to data collection and analysis aim to take account of resources such as gesture, gaze, and image (Hampel, 2003, 2019; Hampel & Stickler, 2012; Lee et al., 2019; Satar, 2021; Stickler, 2021) in ways which can provide deep insight into language teaching and learning processes. Multimodality also expands the task of the linguistic researcher and results in a number of challenges. In this talk, I will present a recent PhD study which explored L2 gesture from a theoretical perspective, linking the mode to spoken language: ‘two sides of the same coin’ (McNeill, 2000). The approach was driven by an interest in examining the role of gesture within SLA as dyads completed tasks via videoconferencing, accessed on tablets and smartphones from outside the classroom. The study resulted in a set of methodological procedures and techniques designed to support the fine-grained analysis and interpretation of gestural and spoken meaning, viewed as a holistic unit which is then mediated by technology in complex ways.