Chapter 7: Multimodality and Non-Traditional Texts
What Are Multimodal Texts?
Multimodal texts utilize sensory elements to further their rhetorical purpose and persuade an audience. These elements can include audio, visual, and/or physical. You can create a multimodal text using a digital technology tool, but you can also create a multimodal text by hand as well. The following are some examples of digital multimodal composition:
- YouTube video
- Text message
- Word Document
And since we’re looking at types of multimodal texts, let’s also look at some examples of multimodal composition that take place outside of the digital realm:
- Architectural models
The following video, developed for the Arizona State University writing program, discusses how multimodal composition applies to first-year composition courses.
Why Use Digital Composition and Multimodal Texts?
Since our world communicates through mostly digital means, learning how to compose in a digital environment is key to your success not only in school, but also in your current and future jobs, and as a member of society.
In addition to multimodal assignments preparing you for your future writing endeavors, multimodal assignments also allow students to use what they know. Melanie Gagich in her essay “An Introductionto and Strategies for Multimodal Composing” writes that students already have knowledge of multimodal composition. She writes in her essay, “Understanding that you are already composing multimodally in many digital spaces will help you transfer that knowledge and experience into your academic assignments. This understanding might also help alleviate any fears or anxiety you may have when confronted with an assignment that disrupts what you think writing should look like” (Gagich 74). If creating a multimodal text seems new to you, chances are you already have an applied understanding of multimodality, just by virtue of living and socializing in the 21st century.
Gagich continues and writes:
Gagich writes that there are five steps to creating a multimodal text:
- Determine your rhetorical situation.
- Review and analyze other multimodal texts.
- Gather content, media, and tools.
- Cite and attribute information appropriately.
- Begin drafting your text.
Making Visual Choices
According to Foundations of Communication, it may be a cliché to say, “A picture is worth a thousand words,” but visual images have power. Good communication is a multisensory experience. Pre-literate children gravitate toward books with engaging pictures. As adults we graduate to denser books without pictures, yet we still visualize ideas to help us understand the text. That’s because a strong image in a poem or a story appeals both to the readers’ senses and emotions or intellect. Advertisers favor visual media—television, magazines, and billboards—because they are an effective way to hook an audience. Websites rely on color, graphics, icons, and a clear system of visual organization to engage Internet surfers. Visuals bring ideas to life for many readers and audiences in multiple ways:
- As a link between raw data and usable knowledge
- To provide concrete, vivid, and quick representations
- To save space
- To speak in a universal language
- To be persuasive
Types of Visuals
There are many types of visuals you can incorporate in digital and physical multimodal composition to illustrate and emphasize your point. The rest of this section describes how visuals can support and enhance your ideas in a multimodal text.
It’s a visual way to represent love, which is an abstract noun that means different things to a wide range of people.
Maps sometimes include map charts, or statistical maps. In the image to the right, two human palms are displayed, and a map of the world is painted onto them. In the background there is a blue sky with white, fluffy clouds. A map can represent more than geography. This map is a representation of our world, but the image could also speak to lines drawn by humans. One ironic feature of this picture is that the world’s territorial lines are drawn over the lines of two human hands.The image could also speak to matter, how nearly three-fourths of the world is made of water, and how up to 60% of the human body is made of water. Could the image speak to the interconnectedness of all beings on earth? That’s up to the reader to interpret. For more information on how to develop an interpretation you can support in your writing, read over Chapter 22 Using Interpretation to Develop Thesis. Images can also persuade. For more information on how to analyze visual rhetoric, check out Chapter 24 Analyzing Visual Rhetoric.
Graphs and Tables
Graphs can take a variety of forms, the most common being line graphs, bar graphs, and pie charts.
In the graphs above, the creator is using visual representations of numbers to represent growth and decline in their topic. Graphs are a helpful way to visually illustrate change.
This visual illustrates a process. One example of a diagram would be a flow chart. The diagram below illustrates a workflow process.
Photographs (still or moving) depict concrete objects, tell a story, provide a scenario, and persuade an audience.
In the image to the right, a picture of a baby is displayed clearly on a smart phone in the foreground. In the background of the picture, an elderly woman smiles, and her face is blurry. There is contrastbetween the baby whose image is clear, and the woman whose image is blurred. The contrast piques the viewer’s interest. Contrasting images, colors, and subjects can draw a reader into an image force them to ask questions. For more information on using contrast in images, read over Chapter 18.2.
Illustrations can be realistic or abstract. The illustration pictured to the right displays a cartoonish picture of a Polaroid camera, the iconic camera from the eighties. For some readers, the illustration may invoke nostalgia, while for younger viewers, the camera may have a slightly historical feel. The reception of the illustration varies depending on the audience; regardless, the illustration can help persuade a wide range of audience members.
Why Use Visuals?
There are a number of reasons you might consider including visuals in documents, presentations, and other communications. Four reasons are detailed below:
Decorative: Visuals that do not represent objects or actions within the text but are added, instead, for aesthetic effect are considered decorative. Decorative visuals are often added to gain attention or increase the audience’s interest. Visuals can be used this way but can detract from the message you are trying to communicate and, thus, should be used with caution.
Representational: These visuals physically represent or physically resemble objects or actions in the text and are relevant to the content of the text. For example, rather than giving a detailed textual description of a new playground, you might include an image or render of the new playground and use the text to highlight specific features or information.
Analogical: Analogical visuals are used to compare and contrast two things, and explain their likeness or correspondence. For example, a marketing consultant might try to clarify the difference between targeted marketing and mass marketing by including images of a single fisherman with a single fishing rod and line next to an image of a bigger boat with a fishing net. By using the fishing analogy, the marketing consultant is attempting to connect possible prior understanding of the audience, a visual, and the concepts of targeted marketing versus mass marketing.
Organizational: The purpose of organizational images is to provide structure to information, visually define relationships, and illustrate connections. A chart of the hierarchical structure of a company is one example of an organizational image.
|Communication Purpose||Consider These Visuals|
|Depict an object||Photo, 3D Model, Illustration|
|Persuade an audience||Photo, Illustration, Chart (showing statistics)|
|Demonstrate a procedure||Photo, Illustration, Flowchart|
|Explain a process||Diagram, Symbol, Illustration|
|Make comparisons||Bar Graph, Line Graph, Table|
|Demonstrate trends or data||Line Graph|
|Organize information||Map, Table|
Table 18.1 Communication Purposes and Visuals
There are many considerations to keep in mind when choosing visuals. When possible, use a variety of types of visuals, but remember that any visuals you use should enhance the content of the text. For example, only add photos if viewing the photos will clarify the text. Near each visual, explain its purpose concisely. Do not expect your readers to figure out the values of the visuals on their own.
For repositories of openly licensed photos, you can search Wikimedia Commons and Pixabay. Website creation software like WordPress, social media applications like Twitter, and other types of software like graphics makers such as Canva and video creators like Powtoon have their own repositories of free images you can use as well.
This section, “Visual” adapted from “A Picture is Worth 1000 Words: Using Visuals” in Part 1 of Foundations of Professional Communication CC BY 4.0 Originally published at http://www.procomoer.org/foundations/
Making Audio Choices
Including audio in your multimodal project can enhance your text and move a reader both logically and emotionally. Audio enhances your message. You may want to consider the following audio choices:
- Spoken word
- Sound effects
Music: From pop to classical to Bollywood, music can be any use of vocalizations or instrumentals. Music can help convey theme in a video or podcast, and music can help heighten tension and advance plot in a story.
Spoken word: Spoken word audio choices include recording a voice over of the written text or a narration.
Sound effects: Sound effects include any kind of sound, from nature or manmade. A couple of examples include crickets, glass shattering, or applause. Sound effects can help characterize people and convey action. Sound effects can also affect the tone of a text, creating humor or suspense.
National Public Radio develops a wide range of podcasts that integrate music, narration, and sound effects to to tell stories. The sample podcast episode aired on the NPR show Hidden Brain.
Why Use Audio?
There are a number of reasons you might consider including audio in documents, presentations, and other communications. According to the Advisory Group on Computer Graphics (AGOCG), using audio in multimedia has the following advantages:
- It can convey meaning, providing an extra channel of information. It allows redundancy to be incorporated into the presentation of information, so that if the meaning is unclear to a user using visual information alone, the audio may clarify it.
- Different learners use different learning strategies, and audio can provide additional information to support different learning styles, for example some users may learn more by hearing than reading a piece of text.
- Audio can add a sense of realism. Cultural associations with music allow you to convey emotion, time period, geographic location, etc.
- It is useful for directing attention to important events. Non-speech audio may be readily identified by users, for example the sound of breaking glass to signify an error. Since audio can grab the users attention so successfully, it must be used carefully so as not to unduly distract from other media.
- It can add interest to a presentation or program.
- Ease of communication – users may respond better to the spoken word than other media. For example in a company presentation, ‘sound bytes’ from satisfied customers can be used.
The AGOCG also writes that there can be disadvantages to using audio:
- Like most media, files can be large.
- Audio can be easily overused, and when sounds are continually used users tend to tune them out.
- For most people, audio is not as memorable as visual media.
- Good quality audio can be difficult to produce, and like other media most commercial audio, particularly music, is copyright.
- Users must have appropriate hardware and software.
Being a digital writer means that you have to be consciously aware of your audience and their ability or inability to participate in the texts that you create. Not everyone can view a meme or infographic and not everyone can hear the sound on a YouTube video.
Whenever possible, include captions for all videos that you create. This allows those who are hearing-impaired and deaf to access your message. You can edit the videos yourself to add captions, or you can use a platform like YouTube that will auto-generate captions that you can edit.
If you are making a video that contains scenes with any type of action, you will want to create an audio description.
For any type of audio or video, you want to consider including a transcript of the spoken dialogue.
For any photo or graphic you include in your text, make sure to provide alternative text by right clicking on the image in Word, and then select edit Alt-text. using Alt-text is a principle of web accessibility. Users with screen readers will be read an alt attribute to better understand an on-page image.
The purpose of this chapter is to acquaint students with multimodal and digital composition. The following sections of chapter 18 explore the process and tools you can use while digitally composing texts for your academic and workplace audiences.
This chapter is a synthesis of two different creative commons texts:
Sections are written by Anne Turner, published by Central New Mexico Community College, and licensed under CC BY-SA 4.0
Making Visual Choices was adapted from “A Picture is Worth 1000 Words: Using Visuals” in Part 1 of Foundations of Professional Communication CC BY 4.0 Originally published at http://www.procomoer.org/foundations/
Advisory Group on Computer Graphics, “Using Audio in Multimedia,”
Gagich, Melanie. “An Introduction to and and Strategies for Multimodal Composing Melanie Gagich.” Writing Spaces: Readings on Writing, Volume 3, licensed under CC BY-NC-ND 4.0, https://writingspaces.org/sites/default/files/1gagich-introduction-strategies-multimodal-composing.pdf