Summary Advanced Cognitive Engineering (0HM150)
Lecture 1 Ubiquitous computing and future scenarios
Embodied: virtual reality, tangible computing, affective computing
Embedded: ubiquitous computing, augmented reality, location-based system
MacCready explosion: 10000 years ago humans plus livestock were 0.1% of biomass, now 98%,
shows dominance of humans on this planet. We are powerful since we can built and use tools
Flynn effect: IQ is increasing, we become smarter. Explanation; we have less physical work so more
time to spend on thinking, more intellectually demanding work
Moore’s law: exponential function of computational power, doubling every 18 months
Three main waves in computing:
- Mainframe: one computer, many people
- PC: one person, one computer
- Ubiquitous computing: one person, many computers
Main changes in computing: smaller, faster, cheaper, more of them, networked, location/monitor
sensing, input/output transducers
Ubiquitous computing=Internet of things. The most profound technologies are those that disappear
(Weisner article)they become integrated in everyday object, you don’t see it as a computer (e.g.
camera). They are not cognitively demanding anymore, people use them unconsciously.
“Disappearance is a consequence of human psychology afforded by the technology”. Calm
technology: technology should be integrated and intuitive, moves easily between periphery and
centre
Extended mind: people use (thinking) tools to offload cognitive effort. Thinking=brain+body+tools.
But is machine intelligence making us smarter or more stupid? (Book Smarter than you think)
Einstein “my pencil and I are smarter than I am”. Experiment with stroop task; when harder
questions were asked this interfered with stroop task, so it took longer to name out loud names of
the searching tools. Experiment 2; difference in computer has saved/deleted information, when told
that info was saved people had less correct recalls. So info only stays active when there is no
external saving point. Conclusion: we are smarter since we know where to get the information
Andy Clark’s natural born cyborgs (article): our brains are naturally including tools in our thinking
processes, this is our ‘special character’. Is there a boundary between user and technology and when
do we use tools and when do they become part of who we are Idea of extended mind
Human-technology co-adaptation:
- Our brain is opportunistic: memory tools work like transactive memory, we offload effort to
technology, remember where to find stuff, but not what it was
- It becomes harder to say where the world stops and the person begins
- We are human-technology symbionts (Clark): thinking and reasoning systems whose minds
and selves are spread across biological brain and non-biological circuitry
,Lecture 2 Embodiment and Presence
Steuer: media is developing in 2 axis interactivity and vividness
Immersion: extent to which you are really there (physically)
Presence: psychological consequence of immersion, the feeling that it is really part of your body
(sense of ownership)
Telepresence: is the experience of being there and the illusion of non-mediation. Key factors for
telepresence as these affect the extent in which a user’s actions are responded to by the system in
continuous and natural ways (closed action-perception loops);
- Vividness: the number of sensory-modalities addressed
- Interactivity: extent to which one can influence the stream of digital info
Our possibilities for enacting such action-perception loops are dependent on the way in which we as
human beings are embodied
Pictorial realism is not necessary for presence as low quality graphics can induce presence.
Involvement with content does neither matter.
Metzinger’s orders of embodiment:
- First order, morphology: the parts of the body determine our possibilities for actions, it is
hard-wired, no thinking involved (e.g. number of limbs and muscles). Tool: robotic arm
- Second order, body schema: non-conscious and anonymous (no reference to the self)
performance of the body, aimed at guiding behavior, no thinking involved as the body
schema has learned what to do (e.g. walking). Tools: body schema helps us to use tools,
example near space neglect becomes far when arms is extended with a tool. After
effectbody schema has adapted to the tool
- Third order, body image: consciousness of body, categorization as parts of the body. It is
constructed through sensorimotor integration and the detection of self-specifying
sensorimotor contingencies. Tools: can be experienced as being part of our own body,
rubber hand illusion, helps if the object looks like the real part of body
What is seeing? Is not only dependent on receptors in the eye, it is a registration of sensory motor
contingencies: moving closer makes things larger on retina, when touching an object you see that
your hand is making contact, as a result object are perceived to be located in environment not in the
retina. Media technology allows for similar contingencies to be detected but with the tactile rather
than the visual system
The body itself is not much different from the technologies we use: both mediate the
communication between the mind and the physical world. But we feel as if we can directly perceive
and interact with the worldwe have an illusion of non-mediation with respect to our own body as
well: The (workings of) body and its perceptual and motor systems is transparent to us, such
transparency occurs because perceptual motor-loops involve the detection of invariant (lawful)
contingencies
Presence occurs when the media technology allows us to establish existing contingencies between
action and perception which in turn make the technology itself transparent
, Important points from the lecture:
- Users are embodied agents: we have a brain and body
- Perception is mediatedtools can become part of us due to plasticity of body image and
body schema, yielding the same transparency that we experience when using our own
sensors and body parts (we are cyborgs Clark)
- Transparency: depends on sensorimotor integration and new sensorimotor contingencies
which are to a large extent depended on the way in which we are embodied, so
telepresence is a mental phenomenon
- Nothing is magical about telepresence; it’s a consequence of our ability to adjust to
environment
The two factors vividness and interactivity influence how naturally the system responds to user’s
actions, leading to closed action-perception loops which is a result of how humans are embodied. On
different (Metzinger’s) orders of the body tools can be integrated. Plasticity of the body schema and
body image can lead to telepresence. Due to the detection of sensorimotor contingencies between
action and perception when using tools people experience transparency and people will categorize
the tools as being part one’s own body. This in the end results in the mental phenomenon of
telepresence, the experience of ‘being there’.
Lecture 3 The immersed user & Natural user interfaces
Perceptual criteria of reality:
- Static depth information is provided via several independent mechanisms (texture,
interposition) that are consistent with each other and the observer’s viewpoint. VR: we get
eyestrain from VR because cues are conflicting
- The effective image size fills our entire field of view. VR: there is an end of the sight field
- The resolution and intensity of the image is only limited by the sensitivities of our visual
system. VR: high resolution/intensity is not possible in VR
- Dynamic depth information is coupled to observer motion in real time, VR: not coupled to
observer motion
History:
- Wall paintings and panoramic paintings 360 degrees
- Magic lanterns, phantasmagoria: moving images due to moving light souce
- First pictures
- Chronophotography (Muybridge): quickly showing images gives movement
- Short movies (Lumieres, train movie)
- Cinema’s, television and 3D Stereoscopic films in cinema
- Experiments with smell (smelly Telly)
- Cinerama: immersive vision (rollercoaster)
- Interactivity in teleoperation: challenge achieving sense of ‘being there’
- Virtual reality: for presence you need interactivity, CAVE
- Consumer VR (Facebook)
- Future: Clarke “Any science or technology which is sufficiently advanced is indistinguishable
from magic”
Virtual reality definitions:
Lecture 1 Ubiquitous computing and future scenarios
Embodied: virtual reality, tangible computing, affective computing
Embedded: ubiquitous computing, augmented reality, location-based system
MacCready explosion: 10000 years ago humans plus livestock were 0.1% of biomass, now 98%,
shows dominance of humans on this planet. We are powerful since we can built and use tools
Flynn effect: IQ is increasing, we become smarter. Explanation; we have less physical work so more
time to spend on thinking, more intellectually demanding work
Moore’s law: exponential function of computational power, doubling every 18 months
Three main waves in computing:
- Mainframe: one computer, many people
- PC: one person, one computer
- Ubiquitous computing: one person, many computers
Main changes in computing: smaller, faster, cheaper, more of them, networked, location/monitor
sensing, input/output transducers
Ubiquitous computing=Internet of things. The most profound technologies are those that disappear
(Weisner article)they become integrated in everyday object, you don’t see it as a computer (e.g.
camera). They are not cognitively demanding anymore, people use them unconsciously.
“Disappearance is a consequence of human psychology afforded by the technology”. Calm
technology: technology should be integrated and intuitive, moves easily between periphery and
centre
Extended mind: people use (thinking) tools to offload cognitive effort. Thinking=brain+body+tools.
But is machine intelligence making us smarter or more stupid? (Book Smarter than you think)
Einstein “my pencil and I are smarter than I am”. Experiment with stroop task; when harder
questions were asked this interfered with stroop task, so it took longer to name out loud names of
the searching tools. Experiment 2; difference in computer has saved/deleted information, when told
that info was saved people had less correct recalls. So info only stays active when there is no
external saving point. Conclusion: we are smarter since we know where to get the information
Andy Clark’s natural born cyborgs (article): our brains are naturally including tools in our thinking
processes, this is our ‘special character’. Is there a boundary between user and technology and when
do we use tools and when do they become part of who we are Idea of extended mind
Human-technology co-adaptation:
- Our brain is opportunistic: memory tools work like transactive memory, we offload effort to
technology, remember where to find stuff, but not what it was
- It becomes harder to say where the world stops and the person begins
- We are human-technology symbionts (Clark): thinking and reasoning systems whose minds
and selves are spread across biological brain and non-biological circuitry
,Lecture 2 Embodiment and Presence
Steuer: media is developing in 2 axis interactivity and vividness
Immersion: extent to which you are really there (physically)
Presence: psychological consequence of immersion, the feeling that it is really part of your body
(sense of ownership)
Telepresence: is the experience of being there and the illusion of non-mediation. Key factors for
telepresence as these affect the extent in which a user’s actions are responded to by the system in
continuous and natural ways (closed action-perception loops);
- Vividness: the number of sensory-modalities addressed
- Interactivity: extent to which one can influence the stream of digital info
Our possibilities for enacting such action-perception loops are dependent on the way in which we as
human beings are embodied
Pictorial realism is not necessary for presence as low quality graphics can induce presence.
Involvement with content does neither matter.
Metzinger’s orders of embodiment:
- First order, morphology: the parts of the body determine our possibilities for actions, it is
hard-wired, no thinking involved (e.g. number of limbs and muscles). Tool: robotic arm
- Second order, body schema: non-conscious and anonymous (no reference to the self)
performance of the body, aimed at guiding behavior, no thinking involved as the body
schema has learned what to do (e.g. walking). Tools: body schema helps us to use tools,
example near space neglect becomes far when arms is extended with a tool. After
effectbody schema has adapted to the tool
- Third order, body image: consciousness of body, categorization as parts of the body. It is
constructed through sensorimotor integration and the detection of self-specifying
sensorimotor contingencies. Tools: can be experienced as being part of our own body,
rubber hand illusion, helps if the object looks like the real part of body
What is seeing? Is not only dependent on receptors in the eye, it is a registration of sensory motor
contingencies: moving closer makes things larger on retina, when touching an object you see that
your hand is making contact, as a result object are perceived to be located in environment not in the
retina. Media technology allows for similar contingencies to be detected but with the tactile rather
than the visual system
The body itself is not much different from the technologies we use: both mediate the
communication between the mind and the physical world. But we feel as if we can directly perceive
and interact with the worldwe have an illusion of non-mediation with respect to our own body as
well: The (workings of) body and its perceptual and motor systems is transparent to us, such
transparency occurs because perceptual motor-loops involve the detection of invariant (lawful)
contingencies
Presence occurs when the media technology allows us to establish existing contingencies between
action and perception which in turn make the technology itself transparent
, Important points from the lecture:
- Users are embodied agents: we have a brain and body
- Perception is mediatedtools can become part of us due to plasticity of body image and
body schema, yielding the same transparency that we experience when using our own
sensors and body parts (we are cyborgs Clark)
- Transparency: depends on sensorimotor integration and new sensorimotor contingencies
which are to a large extent depended on the way in which we are embodied, so
telepresence is a mental phenomenon
- Nothing is magical about telepresence; it’s a consequence of our ability to adjust to
environment
The two factors vividness and interactivity influence how naturally the system responds to user’s
actions, leading to closed action-perception loops which is a result of how humans are embodied. On
different (Metzinger’s) orders of the body tools can be integrated. Plasticity of the body schema and
body image can lead to telepresence. Due to the detection of sensorimotor contingencies between
action and perception when using tools people experience transparency and people will categorize
the tools as being part one’s own body. This in the end results in the mental phenomenon of
telepresence, the experience of ‘being there’.
Lecture 3 The immersed user & Natural user interfaces
Perceptual criteria of reality:
- Static depth information is provided via several independent mechanisms (texture,
interposition) that are consistent with each other and the observer’s viewpoint. VR: we get
eyestrain from VR because cues are conflicting
- The effective image size fills our entire field of view. VR: there is an end of the sight field
- The resolution and intensity of the image is only limited by the sensitivities of our visual
system. VR: high resolution/intensity is not possible in VR
- Dynamic depth information is coupled to observer motion in real time, VR: not coupled to
observer motion
History:
- Wall paintings and panoramic paintings 360 degrees
- Magic lanterns, phantasmagoria: moving images due to moving light souce
- First pictures
- Chronophotography (Muybridge): quickly showing images gives movement
- Short movies (Lumieres, train movie)
- Cinema’s, television and 3D Stereoscopic films in cinema
- Experiments with smell (smelly Telly)
- Cinerama: immersive vision (rollercoaster)
- Interactivity in teleoperation: challenge achieving sense of ‘being there’
- Virtual reality: for presence you need interactivity, CAVE
- Consumer VR (Facebook)
- Future: Clarke “Any science or technology which is sufficiently advanced is indistinguishable
from magic”
Virtual reality definitions:
