Cognition and Emotion
Lecture 1 - Cognition and Emotion in HCI
Research areas in HCI:
Affective computing
o Affective computing = giving computers the ability to recognize, express, and in
some cases ‘have’ emotions.
o Emotions play an essential role in rational decision making, perception, learning,
and other cognitive functions.
o Historically, emotions were considered to be childish bodily reactions separated
from our intellectual faculties. Today, we study emotions by self-reports, by
analyzing behaviour and by psychological responses.
o We find emotions in language (verbal), and facial expressions / body posture /
hand gestures / gait / brain signals / temperature / breathing (non-verbal)
o Challenges of affective computing: affective data collection, evaluation of
emotions, definition of emotions
Persuasive technologies
o Persuasive technologies = using computing power to persuade people.
o In order to persuade someone for a particular behavior change, they should:
o Be sufficiently motivated
o Have the ability to do the behavior
o Be triggered to do that
o Steps: 1) send persuasive message → 2) provide information → 3) provide
motivational feedback → 4) provide social pressure (you walked 100 meters, less
than Erick).
Virtual reality
o Virtual reality = going out of the reality, but still making sense of what we see.
Generating deep fakes is a big part of VR today.
, o Self-embodiment = the perception that the user has a body within the virtual
world. It is effective for teaching empathy by ‘walking in someone else’s shoes’
and can reduce racial bias.
Color and shape are not very important, but motion is. Presence can be broken
when the motion of the visual body doesn’t match with he motion of your actual
body.
Usability engineering
o Usability engineering = accessing information systems and see if users can
interact with the system as intended.
o The steps are: specify usability context → specify requirements → create design
→ evaluate design → (eventually) system meets the requirements.
o We can ask the users/testers questions, but we can also detect their body signals,
for example the finger pulse to measure stress or heart rate (using a pressure-
sensitive mouse is an example), ‘measuring’ their facial expressions, or checking
their brain activity.
Robotics
o We adjust our expectations when engaging with a robot.
Why do information scientists study Cognition & Emotion? →
We can apply the theories in the application areas, and develop them by creating new
tools for measurement and data collection and making meaning out of that data. There
are different theories for certain situations, and by designing and experiencing, we can
determine for ourselves which theory is most accurate.
, Readings lecture 1
Yiend – Cognition and emotion
There are three characteristics that are embodied within an emotional episode:
1. Behaviour. Smiling, turning quiet, voice raising. However, these behaviors can’t be used to
infer emotion directly, since most can be brought under control (suppressing your smile,
falsely acting cheerful). Facial expressions has attracted most research attention.
2. Bodily responses. Sweating, heart raising; all vital for survival. These reactions are controlled
by the autonomic nervous system (ANS). The sympathetic ANS produces responses
associated with arousal. The parasympathetic ANS dominates during periods of rest.
3. Feelings. Anger, fear, love. Researchers are most interested in whether the task elicits
positive or negative feelings. A struggle is to create realistic effects in the lab. Emotions in
real life compared to the lab are usually stronger.
It can be unethical to induce strong emotions in a lab, and strong emotions are hard to elicit
in a predictable fashion (and are hard to die down).
Emotion = a reaction to events relevant to the needs, goals or concerns of an individual. It’s your brain’s creation of what your
bodily sensations mean in relation to what is going on around you in the world. | minutes/hours level
Mood = background affective filter through which internal and external events are appraised. | daily level
Ecological validity = does ...
Emotion spectrum:
- You know about the basic emotions. Different combinations of the basic emotions can produce all
the other emotions (e.g.: joy + acceptance = friendliness).
- The dimensional view is the full range of emotional experience.
- Sets of emotions related to each other form an emotion family.
Challenges are: providing evidence for the existence of a small number of discrete emotion states,
and deciding which and how many emotions should be considered as basic emotions. The current big
5, is: anger, fear, sadness, disgust and happiness.
Differences in language have an effect on the recognition of emotions.
You could classify emotions using and measuring two dimensions: arousal and valence (positive or
negative).
Functions of emotions
- Readjusting and altering (life) goals.
- Increase our chances of survival (nowadays: make us more alert for an exciting moment like an
exam, or while driving).
- Communicating through expression of emotions. Emotional expressions for humans can be honest
signals of emotional status (that’s the case with animals), or can be part of polite interaction and/or
social manipulation.
The psychological changes that are caused by the emotional responses to winning and losing are
called ‘somatic markers’. They guide future decision making.
, Emotion & Cognition
• State emotion = how you feel right now. It’s very variable. Mostly measured by self-reports.
• Trait emotion = more stable personality characteristics. (e.g.: you’re prone to have outbursts
when you’re angry, or always be optimistic, or have doubts).
A trait emotion makes a person more prone to experience the associated state emotion.
The manifestation of emotion (experience/feeling/expression through bodily changes and behaviors)
is the hot component to emotion.
Processing emotional material without actually experiencing it is the cold component to emotion.
This is an example of interaction between cognition and emotion. Researchers are interested in
whether processing emotional material is affected by emotional states and traits (e.g.: does the
processing of sad words change when one actually feels sad?).
Cognitive bias = a person selectively prioritizes information that reinforces a pathological state.
Programs using cognitive bias modification, can modify biases potentially contributing to emotional
disorders.
Cognitive (memory) processes influenced by emotion:
- Mood congruent memory (MCM) is your long-term memory being affected by the emotional state
that you were in at a certain moment.
Even ‘normal’ individuals seem to have a positive and potentially adaptive bias towards memory for
positive information.
- Mood dependent memory (MDM) means that your memory for a particular event will be better if
there is a match between your mood at the time you experienced it and your mood when you try to
recall it.
Anxious patients have an attentional bias for threat, for example they react faster to threat words.
- Semantic interpretation. Depending on one’s emotion of state, he/she can interpret words or
described situations in a certain way.
Attribution theory: we attribute good things as something within our control, and we attribute bad
thing as out of our control / controlled by others or the circumstances.
Does cognition influence emotion?
James-Lange theory: behaviour precedes cognition; the experience of emotion depends on the
behaviour and bodily reaction that follows an event.
Lecture 1 - Cognition and Emotion in HCI
Research areas in HCI:
Affective computing
o Affective computing = giving computers the ability to recognize, express, and in
some cases ‘have’ emotions.
o Emotions play an essential role in rational decision making, perception, learning,
and other cognitive functions.
o Historically, emotions were considered to be childish bodily reactions separated
from our intellectual faculties. Today, we study emotions by self-reports, by
analyzing behaviour and by psychological responses.
o We find emotions in language (verbal), and facial expressions / body posture /
hand gestures / gait / brain signals / temperature / breathing (non-verbal)
o Challenges of affective computing: affective data collection, evaluation of
emotions, definition of emotions
Persuasive technologies
o Persuasive technologies = using computing power to persuade people.
o In order to persuade someone for a particular behavior change, they should:
o Be sufficiently motivated
o Have the ability to do the behavior
o Be triggered to do that
o Steps: 1) send persuasive message → 2) provide information → 3) provide
motivational feedback → 4) provide social pressure (you walked 100 meters, less
than Erick).
Virtual reality
o Virtual reality = going out of the reality, but still making sense of what we see.
Generating deep fakes is a big part of VR today.
, o Self-embodiment = the perception that the user has a body within the virtual
world. It is effective for teaching empathy by ‘walking in someone else’s shoes’
and can reduce racial bias.
Color and shape are not very important, but motion is. Presence can be broken
when the motion of the visual body doesn’t match with he motion of your actual
body.
Usability engineering
o Usability engineering = accessing information systems and see if users can
interact with the system as intended.
o The steps are: specify usability context → specify requirements → create design
→ evaluate design → (eventually) system meets the requirements.
o We can ask the users/testers questions, but we can also detect their body signals,
for example the finger pulse to measure stress or heart rate (using a pressure-
sensitive mouse is an example), ‘measuring’ their facial expressions, or checking
their brain activity.
Robotics
o We adjust our expectations when engaging with a robot.
Why do information scientists study Cognition & Emotion? →
We can apply the theories in the application areas, and develop them by creating new
tools for measurement and data collection and making meaning out of that data. There
are different theories for certain situations, and by designing and experiencing, we can
determine for ourselves which theory is most accurate.
, Readings lecture 1
Yiend – Cognition and emotion
There are three characteristics that are embodied within an emotional episode:
1. Behaviour. Smiling, turning quiet, voice raising. However, these behaviors can’t be used to
infer emotion directly, since most can be brought under control (suppressing your smile,
falsely acting cheerful). Facial expressions has attracted most research attention.
2. Bodily responses. Sweating, heart raising; all vital for survival. These reactions are controlled
by the autonomic nervous system (ANS). The sympathetic ANS produces responses
associated with arousal. The parasympathetic ANS dominates during periods of rest.
3. Feelings. Anger, fear, love. Researchers are most interested in whether the task elicits
positive or negative feelings. A struggle is to create realistic effects in the lab. Emotions in
real life compared to the lab are usually stronger.
It can be unethical to induce strong emotions in a lab, and strong emotions are hard to elicit
in a predictable fashion (and are hard to die down).
Emotion = a reaction to events relevant to the needs, goals or concerns of an individual. It’s your brain’s creation of what your
bodily sensations mean in relation to what is going on around you in the world. | minutes/hours level
Mood = background affective filter through which internal and external events are appraised. | daily level
Ecological validity = does ...
Emotion spectrum:
- You know about the basic emotions. Different combinations of the basic emotions can produce all
the other emotions (e.g.: joy + acceptance = friendliness).
- The dimensional view is the full range of emotional experience.
- Sets of emotions related to each other form an emotion family.
Challenges are: providing evidence for the existence of a small number of discrete emotion states,
and deciding which and how many emotions should be considered as basic emotions. The current big
5, is: anger, fear, sadness, disgust and happiness.
Differences in language have an effect on the recognition of emotions.
You could classify emotions using and measuring two dimensions: arousal and valence (positive or
negative).
Functions of emotions
- Readjusting and altering (life) goals.
- Increase our chances of survival (nowadays: make us more alert for an exciting moment like an
exam, or while driving).
- Communicating through expression of emotions. Emotional expressions for humans can be honest
signals of emotional status (that’s the case with animals), or can be part of polite interaction and/or
social manipulation.
The psychological changes that are caused by the emotional responses to winning and losing are
called ‘somatic markers’. They guide future decision making.
, Emotion & Cognition
• State emotion = how you feel right now. It’s very variable. Mostly measured by self-reports.
• Trait emotion = more stable personality characteristics. (e.g.: you’re prone to have outbursts
when you’re angry, or always be optimistic, or have doubts).
A trait emotion makes a person more prone to experience the associated state emotion.
The manifestation of emotion (experience/feeling/expression through bodily changes and behaviors)
is the hot component to emotion.
Processing emotional material without actually experiencing it is the cold component to emotion.
This is an example of interaction between cognition and emotion. Researchers are interested in
whether processing emotional material is affected by emotional states and traits (e.g.: does the
processing of sad words change when one actually feels sad?).
Cognitive bias = a person selectively prioritizes information that reinforces a pathological state.
Programs using cognitive bias modification, can modify biases potentially contributing to emotional
disorders.
Cognitive (memory) processes influenced by emotion:
- Mood congruent memory (MCM) is your long-term memory being affected by the emotional state
that you were in at a certain moment.
Even ‘normal’ individuals seem to have a positive and potentially adaptive bias towards memory for
positive information.
- Mood dependent memory (MDM) means that your memory for a particular event will be better if
there is a match between your mood at the time you experienced it and your mood when you try to
recall it.
Anxious patients have an attentional bias for threat, for example they react faster to threat words.
- Semantic interpretation. Depending on one’s emotion of state, he/she can interpret words or
described situations in a certain way.
Attribution theory: we attribute good things as something within our control, and we attribute bad
thing as out of our control / controlled by others or the circumstances.
Does cognition influence emotion?
James-Lange theory: behaviour precedes cognition; the experience of emotion depends on the
behaviour and bodily reaction that follows an event.
