17 Understanding the Five Modes of Multimodality

Overview

As communicators, we make choices. In every situation, we must decide how to best communicate meaning to our intended audiences. It’s a process of deliberation that involves calculated choices, strategies, and moves. And, particularly with today’s modern media, communication isn’t just putting words onto paper.

“Multimodal” assignments have become common in higher education, and it is likely that you will be asked to create multimodal texts during your academic career and beyond – in your professional, personal, and civic life. “Modality” is a rhetorical decision that you need to consider as you explore how to best achieve your intended purposes.

A multimodal text employs more than one “mode” to communicate meaning. The most common modes include the following:

Why would writers want to communicate in a modality other than alphabetic text? We’ve all heard the saying that “a picture is worth 1000 words.” In certain situations, a visual has the potential to convey an idea more effectively and more quickly than written text. Similarly, utilizing other modes (like sound, space, and movement) can help readers to better engage with the topic and experience a moment in a way that could be more difficult to accomplish with words alone.

The Five Modes Explained

Modality is an important rhetorical decision that writers need to consider. Now, we’ll take a more detailed look at each mode of communication. Modes can be used individually and in combination with others to create multimodal texts.

1. Linguistic/Alphabetic Mode

The linguistic/alphabetic mode includes written and spoken words, word choice, vocabulary, grammar, structure, and organization of sentences and paragraphs. This mode is the most widely used, is a form that most people are familiar with, and can be delivered through print and audio.

Shopping lists, emails, text messages, academic essays, and the automated voice you hear when you’re on hold with customer service use the linguistic/alphabetic mode since they rely on words to create meaning.

2. Visual Mode

The visual mode includes images, video, color, visual layout, design, font, size, formatting, symbols, visual data, and even GIFs. It helps us communicate meaning in a way that can be seen by the audience. Sometimes people must see to believe, and visuals can be helpful in persuading viewers. For example, if you want to showcase how climate change has devastated the Arctic ecosystem, you might include a video that shows real-world footage, like the following video by National Geographic. This video is considered a multimodal text since words, visuals, and audio are used together for a stronger effect.

Watch: “Heart-Wrenching Video: Starving Polar Bear on Iceless Land | National Geographic” (~1 min)

The visual mode also includes elements of design and allows writers a more creative way to present text on a page or screen. For example, the image in the photo below makes use of labels, headings, color, and other visual features to help the reader more easily understand the information. Like this image, the textbook you’re reading right now is considered multimodal since it, too, combines multiple modes of communication.

Figure 17.1. Multimodal image of visual pathways in the human brain. [Image Description]

3. Aural Mode

The aural mode includes spoken words, sound, music, volume, rhythm, speed of delivery, pitch, tone, and voice. Sound catches people’s attention, and writers use the aural mode to bring their words to life. For example, have you ever listened to a sports game on the radio? Listen to the way the sportscasters help the audience experience the game through sound. This sportscast is considered a multimodal text since the authors combine words (linguistic/alphabetic mode) with sound (aural mode).

Watch: “Sports Broadcast Audio Entry” (2 mins)

Audiobooks are another example of the aural mode. Listen to the following excerpt from The Carriage, by Jena Baxter. Spoken words can add depth and emotion to a story. As you listen, pay attention to the volume, rhythm, pitch, speed, and tone of the narrator’s voice. Multimodality can enhance the experience of “reading” a text.

Watch: “The Carriage Audible Sample” (5 mins)

4. Gestural Mode

The gestural mode includes movement, speed, expression, body language, facial expression, physical proximity, and interactions between people. It allows writers to communicate meaning through movement. Traditionally, this mode was used primarily in face-to-face interaction; however, modern technology allows writers to show movement virtually in their work, through video. The gestural mode is often used in combination with other modes, such as linguistic/alphabetic (written/spoken), spatial (physical arrangement), and aural (sound) to provide an enhanced sensory experience for the audience.

For example, sign languages use the gestural mode since the position of a sign and its movement are significant factors in generating and distinguishing meaning. In this video, look at how the speakers use movements of the hands, head, face, and body, along with position and speed, to communicate meaning to the audience. Sign languages are considered multimodal communication because they combine linguistic/alphabetic text with movement.

Watch: “Spoken Without Words: Poetry with ASL SLAM” (~3 mins)

5. Spatial Mode

The spatial mode revolves around physical arrangement, including spacing, position, organization, direction, and distance of elements in a text. Writers use the spatial mode of communication in the physical layout and organization of a text. For example, the trifold pamphlet, printed and folded on paper (Fig. 17.2) presents information spatially on six panels. Physical arrangement impacts the way the audience can interact with your work.

Websites also rely heavily on the spatial mode to communicate meaning. Writers make strategic rhetorical decisions about how to arrange digital information in a user-friendly way within a mobile “space.” Features like menus, headers, physical layout, and navigation tools (such as links) help the audience to interact with the site spatially. Websites are considered multimodal texts since multiple modes are used in combination to communicate with the audience.

Multimodal Considerations

Now that you have a further understanding of the five modes of communication and how they work individually and in combination, how can you incorporate them into your writing?

Start by researching the rhetorical situation. It is essential that you have a firm understanding of the purpose, audience, and context surrounding the writing task. Understanding the rhetorical situation will help you to make decisions about which mode or combination of modes might best help you connect with your intended audience.

As the writer, you’ll need to determine which modes could add value to your work. Be careful not to add modes just because you think you should. Each mode you use should add meaning to the text. Consider the opportunities, challenges, and constraints of any writing task and assess and revise your work to meet the needs of the audience.

In conclusion, modality affects how the audience will interact with and generate meaning from your work. Writing in each of the five modes can help you to think “outside the box” as you make rhetorical decisions about the kinds of communication that you could use as you venture beyond the printed word.

Further Reading

• “An Introduction to and Strategies for Multimodal Composing” by Melanie Gagich
• “Thinking Across Modes and Media (and Baking Cake): Two Techniques for Writing with Video, Audio, and Images” by Crystal VanKooten
• “Multimodality in Writing for Digital Media” by Cara Miller

This chapter was remixed by Melisa Garcia.

This chapter was adapted from https://pressbooks.pub/openenglishatslcc/chapter/multi-modal-communication-writing-in-five-modes/
Open English @ SLCC Copyright © 2016 by Ann Fillmore is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, except where otherwise noted.

Lutkewitte, Claire, editor. Multimodal Composition: A Critical Sourcebook. Bedford/St. Martins, 2014.