Visual Rhetoric
Lecture 1: instructional texts
Instructional texts
• How-to texts with procedural information
• Both usable and disposable texts
• Use them and also not use them, but when they are used you
have maximum involvement with the text
Situations of use
Need (procedural)
• Instruction directly affects behaviour (compare to advertising)
• Immediate usage of instructional text
o Example bow tie
o How to wash your hands
Learning (declarative)
• Learning for exam or practice
• For internalization and retention of content
o Example of declarative instruction. Not steps that you have
to follow in order.
Evaluation situation (conditional)
• For deciding if instructions are appropriate for the user’s
circumstances and what they need
o Symptoms of low blood sugar – tells you why you might
need it
o When is chest pain an emergency? Seek emergency care if
the pain is crushing or squeezing and accompanied any of
these symptoms
A characteristic feature of the instructional genre:
• The images show actions or outcomes
• The images are supported by written explanations
Many forms of visual resources: photos, drawings, icons, diagrams, arrows, text, headings, etc.
➔ Thus instructions are multidimensional
Reader converts instructions into actions → how does this happen?
• Visual resources can facilitate conversion by showing the outcome of action
o Build your own leg way – LEGO
• The individual steps
o Build your own leg way - LEGO
• A system of parts, whole, and their relationships
o Bike instruction
• Any unwanted actions and outcomes
o Beer tender system – don’t leave it out in the sun. Don’t pick it up this way, etc.
,Cognitive Theory of Multimedia Learning (CToML)
How do people interpret texts + images?
How do you design a multimedia learning environment that stimulates learning?
• Multimedia → Multimodal content: words (written or spoken) and images (illustrations, photos,
animations, video)
• Multimedia instruction → the use of multimedia to promote learning
• Multimedia learning → learners’ construction of mental representations from the multimedia
instruction
Three perspectives on what multimedia is:
• Delivery media (technology-focused): what devices are used to present information to the
learner (beamer, speaker, screen, interactive article, etc.)
• Presentation modes (learner-focused): focus on what is presented to the learner (words, sounds,
images, video, photographs, animations, etc.). Assumes different channels for processing verbal
and pictorial info.
• Sensory modalities (learner-focused): focus on the learner’s senses of vision and hearing (an
explanation can be written, spoken, illustrated, demonstrated, etc.). Assumes different channels
for processes auditory and visual info.
Assumptions of CtoML:
1. Dual coding/channel
• Verbal and non-verbal information are treated differently
• There are different locations in memory for processing verbal and non-verbal information, and
therefore two different processing paths for them
2. Limited capacity
• In the working memory subsystems, each channel has limited capacity to handle verbal and non-
verbal information
3. Active learning process
• People actively construct knowledge from the two channels
• They select and organize information into coherent mental representations
• They integrate these mental representations with existing knowledge
•
,Mayer’s research focuses on specific learnable content organized into a series of causal events. How
does a bicycle pump work? How does lightning come about?
Mayer tests efficacy of multimedia with experiments that target the learner’s:
• Retention: correct recall of the learned information
• Transfer: correct replication or application of the learned information
How does a bike pump work?
Design principles of CToML
• Multimedia principle: text + images is better than text alone
• Modality principle: spoken text + image is better than written text + image
• Spatial proximity principle: the space between corresponding words and images should be
minimal
Which one is better? → the integrated presentation
is better. Closer to each other. More able to
understand them / better learning.
• Temporal proximity principle: corresponding words and images should occur simultaneously
Mayer’s and others’ research
• Focus on declarative information (know-that)
• Oriented to the retention and transfer of declarative info
• Use of text-only rather than combination of visual + verbal
Michas & Berry (2000)
• Focus on procedural information (know-how)
• Oriented to retention and imitation
• Use of different combinations of visual representations, over three experiments
, Michas & Berry (2000): experiment 1
Five conditions for how to bandage a hand: Text only; Line drawing only; Text + line drawing; Video;
Video stills
Two phase procedure:
- Learning phase: subjects were given step by step instructions (in five conditions), and studied the
materials until they felt they could carry out the task/instruction
- Testing phase: the subjects were tasked with carrying out the instruction (application) and
answering questions about it (retention)
• Results: the best was the video. Easiest to follow those instructions. Second best one was the
text + line drawing.
• Remaining question: why is text + line drawing better than line drawings only?
- Explanation 1: verbal and visual content is processed via two channels, leading to enhanced
learning (dual coding assumption)
- Explanation 2: verbal content gives additional action info that supplements visual info, leading to
enhanced learning
Michas & Berry (2000): experiment 2
- Hypothesis: if action information is important for learning how to do this task, then more action
information will lead to better performance. How you can show action information in an
instruction?
- How can you show action information in an instruction? → No action info vs. action info
displayed (with arrows, text)
- Five conditions: Simple line drawing (no arrows); Simple line drawing + reduced text (no action
info); Enhanced line drawing (with arrows); Simple line drawing + text; Enhanced line drawing +
text
• Best conditions: instructions with action info (arrows or text) → action information helps
learning procedural instructions
• Findings that complicate predictions of dual coding: text that merely disambiguates image < text
that gives action info. Standard text + line drawing about same as enhanced line drawing.
Lecture 1: instructional texts
Instructional texts
• How-to texts with procedural information
• Both usable and disposable texts
• Use them and also not use them, but when they are used you
have maximum involvement with the text
Situations of use
Need (procedural)
• Instruction directly affects behaviour (compare to advertising)
• Immediate usage of instructional text
o Example bow tie
o How to wash your hands
Learning (declarative)
• Learning for exam or practice
• For internalization and retention of content
o Example of declarative instruction. Not steps that you have
to follow in order.
Evaluation situation (conditional)
• For deciding if instructions are appropriate for the user’s
circumstances and what they need
o Symptoms of low blood sugar – tells you why you might
need it
o When is chest pain an emergency? Seek emergency care if
the pain is crushing or squeezing and accompanied any of
these symptoms
A characteristic feature of the instructional genre:
• The images show actions or outcomes
• The images are supported by written explanations
Many forms of visual resources: photos, drawings, icons, diagrams, arrows, text, headings, etc.
➔ Thus instructions are multidimensional
Reader converts instructions into actions → how does this happen?
• Visual resources can facilitate conversion by showing the outcome of action
o Build your own leg way – LEGO
• The individual steps
o Build your own leg way - LEGO
• A system of parts, whole, and their relationships
o Bike instruction
• Any unwanted actions and outcomes
o Beer tender system – don’t leave it out in the sun. Don’t pick it up this way, etc.
,Cognitive Theory of Multimedia Learning (CToML)
How do people interpret texts + images?
How do you design a multimedia learning environment that stimulates learning?
• Multimedia → Multimodal content: words (written or spoken) and images (illustrations, photos,
animations, video)
• Multimedia instruction → the use of multimedia to promote learning
• Multimedia learning → learners’ construction of mental representations from the multimedia
instruction
Three perspectives on what multimedia is:
• Delivery media (technology-focused): what devices are used to present information to the
learner (beamer, speaker, screen, interactive article, etc.)
• Presentation modes (learner-focused): focus on what is presented to the learner (words, sounds,
images, video, photographs, animations, etc.). Assumes different channels for processing verbal
and pictorial info.
• Sensory modalities (learner-focused): focus on the learner’s senses of vision and hearing (an
explanation can be written, spoken, illustrated, demonstrated, etc.). Assumes different channels
for processes auditory and visual info.
Assumptions of CtoML:
1. Dual coding/channel
• Verbal and non-verbal information are treated differently
• There are different locations in memory for processing verbal and non-verbal information, and
therefore two different processing paths for them
2. Limited capacity
• In the working memory subsystems, each channel has limited capacity to handle verbal and non-
verbal information
3. Active learning process
• People actively construct knowledge from the two channels
• They select and organize information into coherent mental representations
• They integrate these mental representations with existing knowledge
•
,Mayer’s research focuses on specific learnable content organized into a series of causal events. How
does a bicycle pump work? How does lightning come about?
Mayer tests efficacy of multimedia with experiments that target the learner’s:
• Retention: correct recall of the learned information
• Transfer: correct replication or application of the learned information
How does a bike pump work?
Design principles of CToML
• Multimedia principle: text + images is better than text alone
• Modality principle: spoken text + image is better than written text + image
• Spatial proximity principle: the space between corresponding words and images should be
minimal
Which one is better? → the integrated presentation
is better. Closer to each other. More able to
understand them / better learning.
• Temporal proximity principle: corresponding words and images should occur simultaneously
Mayer’s and others’ research
• Focus on declarative information (know-that)
• Oriented to the retention and transfer of declarative info
• Use of text-only rather than combination of visual + verbal
Michas & Berry (2000)
• Focus on procedural information (know-how)
• Oriented to retention and imitation
• Use of different combinations of visual representations, over three experiments
, Michas & Berry (2000): experiment 1
Five conditions for how to bandage a hand: Text only; Line drawing only; Text + line drawing; Video;
Video stills
Two phase procedure:
- Learning phase: subjects were given step by step instructions (in five conditions), and studied the
materials until they felt they could carry out the task/instruction
- Testing phase: the subjects were tasked with carrying out the instruction (application) and
answering questions about it (retention)
• Results: the best was the video. Easiest to follow those instructions. Second best one was the
text + line drawing.
• Remaining question: why is text + line drawing better than line drawings only?
- Explanation 1: verbal and visual content is processed via two channels, leading to enhanced
learning (dual coding assumption)
- Explanation 2: verbal content gives additional action info that supplements visual info, leading to
enhanced learning
Michas & Berry (2000): experiment 2
- Hypothesis: if action information is important for learning how to do this task, then more action
information will lead to better performance. How you can show action information in an
instruction?
- How can you show action information in an instruction? → No action info vs. action info
displayed (with arrows, text)
- Five conditions: Simple line drawing (no arrows); Simple line drawing + reduced text (no action
info); Enhanced line drawing (with arrows); Simple line drawing + text; Enhanced line drawing +
text
• Best conditions: instructions with action info (arrows or text) → action information helps
learning procedural instructions
• Findings that complicate predictions of dual coding: text that merely disambiguates image < text
that gives action info. Standard text + line drawing about same as enhanced line drawing.
