Skip to content
What Is Multimodal Discourse
1.1 What is Discourse? Discourse is an important means for humans to convey information, and it is a linguistic unit with communicative significance or contextual semantics.
1.2 The forms of discourse can be monomodal (单模态) or multimodal (多模态). Here, mode refers to the patterns or methods of conveying information in discourse.
1.3 The spoken or written language we use in our daily conversations or writings is usually a single mode, and what we say or write constitutes a monomodal discourse (单模态语篇).
1.4 “Multimodal” is relative to “monomodal.” If we use images, emojis, sound effects, animations, and other means beyond language, the discourse that contains two or more modes is called multimodal discourse (多模态语篇).
Classification of Modes in Multimodal Discourse
2.1 The “modes” in multimodal discourse can be divided into two main categories, namely, Linguistic Mode and Non-linguistic Mode.
2.2 Linguistic modes can include various languages such as Chinese, English, Spanish, etc., and can be either spoken or written, oral or printed.
2.3 Non-linguistic modes can be further divided into visualVisual (such as non-linguistic symbols, charts, graphics, images, colors, animations, etc.), gesturalGestural (such as non-linguistic expressions, gestures, body language, performance art, etc.), spatialSpatial (using space and position to convey meaning, such as installation art, etc.), and auralAural (such as non-linguistic sounds, music, sound effects, etc.) four subcategories of modes【Note】.
2.4 Multimodal discourse utilizes any two or more of the above five categories of modes to convey information or express meaning.
Why Emphasize Multimodal Discourse Teaching in English Courses
3.1 In the digital information age, multimodal discourse is ubiquitous in our daily work and life. For example, commonly used teaching materials in classrooms such as textbooks, picture books, PPTs, and electronic whiteboards; common multimodal discourse examples in daily life include movies, television, advertisements, WeChat, Weibo, Douyin, Meipian, Toutiao, Xiaohongshu, etc.
3.2 In recent years, the emphasis on multimodal discourse teaching in English courses is essentially an objective reflection of the development of real society.
3.3 The fifth skill mentioned in the primary and secondary school English curriculum standards (2017/2022) is the skill of “seeing,” which mainly aims to cultivate students’ ability to accurately decode information and meaning in English multimodal discourse.
1The linguistic mode in multimodal discourse is specifically the “linguistic” mode; be careful not to confuse it with the “language” in daily discourse, such as body language, symbolic language, color language, etc. Body language, symbolic language, and color language are all considered non-linguistic modes in multimodal discourse.
Key Points and Challenges in Teaching English Multimodal Discourse
4.1 The focus and challenges of multimodal discourse teaching are fundamentally the same as those of traditional discourse teaching, still centered on the aspect of English language culture. From the perspective of curriculum education and English teaching, there is essentially no difference in the focus and challenges of multimodal discourse compared to traditional reading and listening single-mode discourse; both guide students to obtain discourse information and construct meaning through correct decoding of discourse forms.
4.2 Advantages of Students. Compared to the linguistic mode in English, other four modes in multimodal discourse, whether they are charts, icons, images, or sound effects, colors, animations, generally do not pose cognitive challenges for the generation of students who “grew up reading images.” As “digital natives,” primary and secondary school students may be more familiar with various non-linguistic modes in multimodal discourse than their teachers.
4.3 Differences Between Native and Foreign Languages. Chinese students’ understanding of the information and meaning construction process in English multimodal discourse shows significant differences compared to Chinese multimodal discourse, thus leading to different teaching focuses and challenges. When students comprehend multimodal discourse of similar difficulty in English textbooks or extracurricular materials, the difficulties often lie primarily in the English language. In such cases, non-linguistic modes such as images and videos can serve as aids to facilitate academic understanding of the English language background or clues.
4.4 Cultural Aspects. Non-linguistic modes often contain culturally significant elements, such as certain colors, symbols, gestures, etc., which may have different meanings in English culture compared to Chinese culture, potentially affecting the overall meaning conveyed and understood in multimodal discourse. In teaching multimodal discourse, it is also essential to guide students to pay sufficient attention to these non-linguistic modal elements.
4.5 Utilizing Non-linguistic Modes. In foreign language multimodal discourse teaching, the functions of non-linguistic modes usually manifest in two aspects: one is to stimulate students’ reading interest and motivation, and the other is that students activate their existing life experiences and cognition by recognizing these non-linguistic elements, using them as scaffolds to enhance the speed and accuracy of language understanding.
4.6 English teachers need to keep pace with the development of the times and actively understand various non-linguistic modes familiar to students. In multimodal discourse teaching, actively utilizing non-linguistic modes to create rich contexts that align with students’ cognitive experiences, guiding them to fully leverage their existing foundational knowledge and life experiences, and focusing on the synergy of various modes in meaning transmission will enhance language ability, thinking quality, aesthetic ability, and other comprehensive qualities throughout the entire process of multimodal discourse analysis and understanding. Additionally, multimodal discourse teaching typically does not require any special “design.”
(Examples of multimodal discourse analysis: curriculum standard cases, textbook cases, PPT cases, extracurricular discourse cases, etc., will be detailed in another document.)
6. 1 Specific Requirements for “Multimodal Discourse” in Curriculum Standards
The following are the specific statements regarding the teaching of “multimodal discourse” in primary and secondary English education as per the “Compulsory Education English Curriculum Standards (2022 Edition)” (high school level omitted. Please refer to the English curriculum standards (2017/2020)):
1) In the curriculum content section, it mentions the language skill requirement of “seeing.” “Seeing” usually refers to the skill of understanding meanings using graphics, tables, animations, symbols, and videos in multimodal discourse. Understanding multimodal discourse requires not only traditional reading skills but also the ability to observe information in charts and comprehend the meanings of symbols and animations.
2) Content requirements: Level 1 and Level 1+ teaching requirements “infer the meanings conveyed by images, sounds, colors, etc. in multimodal discourse (such as animations, book covers and backs, invitation cards, and greeting cards)”; Level 2 and Level 2+ teaching requirements “understand the meanings conveyed in multimodal discourse (such as animations, posters, book covers and backs, etc.), extracting key information”; Level 3 and Level 3+ teaching requirements “understand the meanings conveyed by non-text resources in multimodal discourse.”
3) The academic quality section requires Level 3 (grades 7-9) to “understand the main content of multimodal discourse (such as radio and television programs, etc.) and obtain key information.”
4) In the curriculum implementation section, it requires English teachers to “enhance the effectiveness of using information technology. Teachers must fully recognize that modern information technology not only provides multimodal means, platforms, and spaces for English teaching but also offers rich resources and opportunities for language learning and use across time and space, playing an important supporting role in creating a good learning situation, promoting educational concept renewal, and transforming teaching methods.”
6. 2 Curriculum Standards Related Requirements for Teachers
The “Compulsory Education English Curriculum Standards” (2022 Edition) states: In the digital information age, English teachers need to keep pace with the times and continuously improve their information literacy, especially focusing on the unique discourse functions of various non-linguistic modes in information transmission and meaning construction. Information technology provides rich modal means, tools, and platforms for English teaching, and English teachers can fully utilize and integrate these resources to create meaningful and engaging situations, enhancing the effectiveness of English learning and promoting the development of students’ core competencies, better achieving the overall goal of curriculum education.
6. 3 Keeping Up with the Times, Staying True to the Original Intention
In the past two to three decades, the attitudes toward multimodal foreign language teaching or second language teaching in the field of international foreign language education have not been consistent. Similar to the long-standing debate among different schools regarding Communicative Language Teaching (CLT) and Task-Based Language Teaching (TBLT), there have been different claims such as strong multimodal (Strong Version) and weak multimodal (Weak Version). Overall, the weak multimodal approach seems to be more suitable for our national conditions, teaching situations, and learning situations. In foreign language teaching, while actively exploring how to fully utilize multimodal resources, we must remain true to the original intention of foreign language curriculum education, especially being vigilant against various forms of formalism and technologicalism that mislead teachers and students in teaching.
6. 4 Two Related Concepts: Multimodal and Multimedia
Another concept closely related to “multimodal” is “multimedia.” The distinction between these two concepts is still under discussion and lacks a consensus. The generally agreed view is that the distinction mainly lies in the different perspectives: “multimodal” emphasizes the way or mode of discourse construction; “multimedia” emphasizes the technical means or tools. When there is no special emphasis on the distinction between the two, the two terms can sometimes be used synonymously.
Welcome to Participate in the Online “11th Ming Shi English Reading Conference for Primary and Secondary Schools” on November 18-19. For documents, please contact Teacher Zhang at 18056570308 (WeChat same number)!
-
Original source: “Primary School Teaching Design” (English) 2022, Issue 11. Content has been modified.
-
Special thanks to Teachers Zhang Yanwen and Jiang Zhenhai for their valuable suggestions on the writing of this article.
