Multimodal Discourses Across the Curriculum

Abstract

Multimodality approaches representation and communication as something more than language. It attends to the complex repertoire of semiotic resources and organizational means through which people make meaning – image, speech, gesture, writing, three-dimensional forms, and so on. A social semiotic approach to multimodality sets out to reveal how processes of meaning making (i.e., signification and interpretation or what is called semiosis) shape individuals and societies. In this chapter, we use multimodality to refer to “multimodal social semiotics” (Kress, Multimodality: A social semiotic approach to contemporary communication. London: Routledge, 2010). Its basic assumption is that meanings derive from social action and interaction using semiotic resources as tools. A variety of disciplines and theoretical approaches can be used to explore different aspects of the multimodal landscape (Jewitt, The Routledge handbook of multimodal analysis. London: Routledge, 2014). Psychological theories can be applied to look at how people perceive different modes or to understand the impact of one mode over another on memory, for example. Sociological and anthropological theories and interests can be applied to examine how communities use multimodal conventions to mark and maintain identities. The term multimodality is, however, most strongly linked with theories rooted in linguistics, notably systemic functional linguistics, social semiotic theory, and conversation analysis (Jewitt et al., Introducing multimodality. London: Routledge, 2016a). Examining multimodal discourses across the classroom makes more visible the relationship between the use of semiotic resources by teachers and students and the production of curriculum knowledge, student subjectivity, and pedagogy.