John Bateman

John Bateman (PhD in Artificial Intelligence, Edinburgh) has been professor of applied linguistics in the Faculty of Linguistics and Literary Sciences at the University of Bremen since 1999. His main fields of research range over functional linguistics, semiotics, computational linguistics (particularly natural language generation, discourse and dialogue), formal ontology, and the theory and practice of multimodality. He has published extensively in all these areas, with articles appearing in international journals ranging from the Artificial Intelligence Journal to the Journal of Terrorism and Political Violence. Recent books include monographs on multimodality and genre (2008, Palgrave), film (with Karl-Heinrich Schmidt, 2012), text and image (2014, Routledge), and an introductory textbook to the field of multimodality and its study as a whole (with Janina Wildfeuer and Tuomo Hiippala (2017), de Gruyter). His current research focuses on the further development and application of theories of multimodal communication and their empirical evaluation with respect to ever broader ranges of media, artefacts and performances.

Abstract

Developing stronger methodologies for multimodal research into knowledge dissemination: The material challenge and the challenge of materials

As multimodal literacy establishes itself increasingly both as a research orientation and as a practical component of curricula in diverse educational contexts, many challenges are raised concerning just how the necessary competences can be encouraged not only among learners, but also among those tasked with teaching learners just how to effectively engage with multimodal artefacts and performances. Too often educators are asked to address complex multimodal configurations for which they themselves may not have particularly robust conceptual tools and methodological frameworks. Researchers on multimodal literacy need then not only to understand how multimodality operates but also be able to translate that understanding at various levels of technicality, ranging from teacher education to that of the classroom. The difficulties involved in such translation should not be underestimated because meta-language based on theory is by no means automatically a meta-language usable in concrete educational contexts. What is more, the sheer diversity of medial contexts within which communication unfolds is dramatically expanding, which means that skills and meta-languages learnt with respect to specific constellations of modalities may not be immediately applicable to new constellations. Against this background of concerns, the aim of this presentation will be to set out some basic dimensions of organization capable of allowing researchers, practitioners and learners to productively situate and analytically engage with any particular case of multimodal communication that they may encounter. This relies upon a re-orientation to the materiality of communication, be that communication written, spoken, graphic or any combination of expressive forms, in order to help ground any interpretative strategies that may be explored or applied subsequently. The appeal to materiality, and the framework provided for this, has the important function of making analysis, at least in its earliest phases, more responsive to actual perceptible properties of the objects and situations being studied so that analysts, at any level, can become increasingly confident in their discussions and interpretations. Consequences of this approach for achieving broader, more empirical foundations will also be discussed, with examples from film, comics and several other media.

References

Bateman, J.A. (2016). Methodological and theoretical issues for the empirical investigation of multimodality. In N.-M. Klug & H. Stöckl (Ed.), Handbuch Sprache im multimodalen Kontext, (pp. 36-74). Berlin: Mouton de Gruyter.
Bateman, J.A., Wildfeuer, J. & Hiippala, T. (2017). Multimodality – Foundations, Research and Analysis. A Problem-Oriented Introduction. Berlin: Mouton de Gruyter.