Lecture 1: Introduction and methods
We have 5 senses: vision (sight), hearing (audition), taste (gustation), smell (olfaction), and
touch (somatosensation). Each of these has a sensory organ associated with it, the eyes
(retina), ears (cochlea), tongue (taste buds), nose (olfactory epithelium), and the skin. But
within those sensory organs there are specific types of cells and specific structures that are
doing the sensing. So, we don’t use our whole eye for capturing light from the outside world,
but we use it for guiding that process. The structure that is interacting with the light to sense
it is the retina.
There is a sixth sense that is often missed but does have a sensory organ interacting with
external physical forces in the world. This is our sense of balance, or vestibular sense. This
relies on interactions of our vestibular labyrinths of the inner ear with gravity and inertia.
There are other feelings and sensations that result from the internal state of the body, such
as thirst, hunger, fullness, heart rate, and blushing.
Within vision all the different aspects begin in the eye and share early visual pathways. But
then are separated into different functions performed by different pathways: form (shape),
motion, color, depth and distance, and cognitive influences: attention and awareness.
Similarly, audition (hearing) all starts with the ear and share early auditory pathways. But
then has separate analyses for the sounds’ location and frequency (pitch).
Vision is the primary sense in humans. Almost every action we make is guided by vision. We
do a lot of analysis of vision. We understand vision very well because it’s easy to study. We
can make visual inputs easily. We can see where the eye is looking, and we can test what a
person can perceive. The visual cortex is also easy to access.
Neural computations
In the earlier stages of vision, it’s even possible to examine the inputs and responses in
neurons in so much detail that we can follow how a pattern of activity in one set of neurons
is analyzed by another set of neurons to give a new type of response.
Dualism is the idea that the mind or spirit is a separate entity from the physical body.
Monism is the idea that the mind is an aspect of the body, held in the brain and nervous
system. Our perception is part of our mind, and in this course, we work from the
fundamental principle that our perception ACTUALLY IS the activity of the neurons in our
brain and nervous system.
Sensation: a translation of the external physical environment into a pattern of neural activity
(by a sensory organ).
Perception: analysis of this neural activity to understand the environment and guide
behavior -> the subjective conscious experience of the outside world.
Sensation and perception reflect interaction between our sensory organs and physical
properties of the world, so they are depended on the physical properties of the world and
limited by the physical properties of our sensors.
,Sensation and perception have evolved to help us survive and reproduce, so they are
optimized for useful (not accurate) representations of the environment, influenced by
interpretation: context and experience, and dependent on limited resources of attention
and awareness. So, all these features of perception suggest that our perception is limited by
the properties of the body and the brain. So, perception is a manifestation of neural activity
in the brain.
So, perception is a translation of the physical environment into a pattern of neural activity
that can be used by our brain to guide behavior. To study perception experimentally we first
change the physical environment of a human or animal (sensory stimulus), and then we can
measure the resulting behavior, or we can measure the resulting change in neural activity.
Psychological approach: quantitative measurements of behavior resulting from perception.
Psychophysics: ‘the scientific study of the relation between stimulus and sensation’.
Psychophysics is best understood as testing whether participants can give an answer that
shows that they perceive the difference between two stimuli -> the ‘Just Noticeable
Difference (JND)’. The variability with which we perceive something depends on the
magnitude of that property or the strength of that stimulus.
Weber- Fechner (psychological) Law describes the relationship between a physical intensity
(I) and its perceived intensity (sensation, klogl). Weber- Fechner determined that this is a
logarithmic relationship: detectable difference increases with average stimulus intensity. We
now know this arises because our sensory receptors tend to follow this logarithmic scaling.
How to determine a perceptual threshold:
The probability that we perceive something isn’t all or nothing. The probability that we
perceive a difference increases as the size of difference increases. The way that
psychophysics experiments work is that they give us two alternatives (yes or no/ brighter or
darker etc.) of the answer that we can give -> 2- alternative forced choice (2AFC). And then
you ask people to decide between them and you force them to make a choice. So, when
they don’t know they have to guess.
Method of constant stimuli: present trials with extent of differences randomized from one
trial to the next. Then you plot detection probability against extent of difference.
è THE PSYCHOMETRIC FUNCTION.
How to determine a perceptual threshold (efficiently)
We estimate parameter of interest (threshold) without determining the entire psychometric
function. This is a widely- used, fast and efficient method. Basically, what you do is you
change the stimulus difference intensity depending on the pattern of people’s previous
responses. This aims to make the next trial maximally informative about the threshold.
Biological approach: what are perception’s neural substrates (i.e., sensory receptors and
brain processing)? Correlate a measure of neural activity with a change in the presented
stimulus or the animal’s behavior (done with lesion studies).
Neural activity is either: spiking activity (action potentials), synaptic activity (synaptic
potentials), or metabolic activity (oxygen & glucose consumption). These are closely related.
,Measuring spiking activity: spikes are the action potentials (firing) of individual neurons.
These are very small changes, so they must be measured directly from the neuron. That
means we need to make invasive recordings inside the brain of an animal or human. Spiking
activity is often seen as the best, clearest measure of neural activity.
Measuring synaptic activity (the communication between cells): several measures at
different scales and resolutions. The smallest of these is the local field potential (LFP).
Spiking, oscillations, and neural interactions
If you have two groups of neurons, some of which are excitatory (so that they will produce
more activity in another group of neurons) and some of them are inhibitory (so that they
will reduce the activity in other neurons). If you have an input in the excitatory pool it will
become active. What it does then is that it is connected to the inhibitory pool, which makes
the inhibitory pool active. What that does is that it inhibits the excitatory pool, so the
excitatory pool of neurons becomes less active and then it stops stimulating the inhibitory
pool of neurons. So that pool of neurons becomes less active too. When the excitatory pool
gets an input, it becomes active again. This goes on and on. This is the wave of the LTP.
EEG measures the LTP from the scalp. It is noninvasive because it’s outside of the head. It
measures them with very poor spatial resolution because you’re measuring from a long way
away. Its advantages are that it’s cheap, it has a high temporal resolution, it moves with the
subject (good for children), and it’s silent (good for auditory perception). Its disadvantages
are that it has a poor spatial resolution, poor signal- to- noise ratio, it only senses activity
near the scalp (cortex), and it’s slow to set up (particularly for large electrode numbers).
Functional MRI advantages are that it has a high spatial resolution, straightforward
analyses/ interpretation, it’s safe and non- invasive, and it has easy access. Its disadvantages
are that it has an indirect measure of neural activity (it measures blood flow), it has low
signal- to- noise ratio, awkward environment, poor temporal resolution, it’s expensive, and
it’s resource- intensive.
MRI scanners: with EEG you know exactly when the change in neural activity happens, and
with fMRI you know where it is. So, it’s a very powerful combination, they cover each other’s
weaknesses when used together.
Signal depends on PD, T1, T2, and T2*
T2* & deoxyhemoglobin
Oxygenated blood? No signal loss
Deoxygenated blood? Signal loss
(1) deoxyhemoglobin affects T2* (MRI signal). (2) blood response follows neural activity. (3)
step 2 overcompensates.
Important: effect 2 doesn’t compensate accurately for effect 1. So, oxyhemoglobin
concentration increases due to increased blood flow. Doesn’t decrease due to oxygen use.
This is what we call the B (blood) O (oxygenation) L (level) D (dependent) signal. This is
what we measure, the increase of blood flow.
, Lecture 2: The eye
Human retina
• Photoreceptor
Visual convergence: horizontal cells, (amacrine cells), bipolar cells, and ganglion cells.
When light reaches our retina
Photoreceptors are on the back of the retina. The photoreceptors are connected to the
middle layer where there’s horizontal cells, bipolar cells, amacrine cells and those connect
to the upper layer where there are ganglion cells, and those make up then the optic neuron.
There is a dip in the fovea. In the fovea we have a high density of cones, the density drops
quickly if you go to the side, or more eccentric, and then the density of rods becomes higher
and takes over. In the fovea we have the sharpest vision
Light is absorbed in the outer segment of the rod (and cone) cells. There is a zillion
membranes in the outer segment and in those membranes are a lot off cross membrane
proteins and those cross-membrane proteins sort of catch the photons. When the light hits
the rod/ photoreceptor (or cone) the rhodopsin is broken down and the cell hyperpolarizes!
(Neurotransmitter release decreases in proportion to amount of light). Light results in
rhodopsin to be broken down and remade -> it’s a loop.
We have 3 types of cones, red cones (long wave), green cones (medium wave), and blue
cones (short wave), and 1 type of rod. We have about 120 million rods and about 6-7 million
cones in our retina. 64% long wave (R), 32% medium wave (G), and 2% short wave (B) rods.
Rods are more sensitive than cones, but insensitive to long wavelengths (anything red).
Appears brighter during daytime (R) than nighttime, dusk (B) -> Purkinje shift.
Rods are also more sensitive cause they have a longer integration ‘widow’.
When light doesn’t reach our retina anymore: the effect of dark adaption.
If one of the cone types (pigments) is missing it means that you can differentiate between
fewer colors. If your L- cone is missing it causes protanopia, if your M-cone is missing it
causes deuteranopia, and if your S-cone is missing it causes tritanopia.
The more light you have the sharper you see. Also, the more photoreceptors, or more
photoreceptors in smaller receptive fields the sharper you see. Acuity (sharpness) and
amount of light help us determine what an image is.
Summary: the light on the retina is ‘noisy’ because of the photon noise. The density of
receptors determines spatial acuity (how sharp you can see). The type of receptor
determines temporal acuity/ sensitivity. The stimulus- point on the retina has a spread with
gradual slope towards un- stimulated area. Therefore, we need more processing.
Adaption and micro- saccades
We don’t perceive light. We perceive changes in light (intensity) across space and or time.
We have 5 senses: vision (sight), hearing (audition), taste (gustation), smell (olfaction), and
touch (somatosensation). Each of these has a sensory organ associated with it, the eyes
(retina), ears (cochlea), tongue (taste buds), nose (olfactory epithelium), and the skin. But
within those sensory organs there are specific types of cells and specific structures that are
doing the sensing. So, we don’t use our whole eye for capturing light from the outside world,
but we use it for guiding that process. The structure that is interacting with the light to sense
it is the retina.
There is a sixth sense that is often missed but does have a sensory organ interacting with
external physical forces in the world. This is our sense of balance, or vestibular sense. This
relies on interactions of our vestibular labyrinths of the inner ear with gravity and inertia.
There are other feelings and sensations that result from the internal state of the body, such
as thirst, hunger, fullness, heart rate, and blushing.
Within vision all the different aspects begin in the eye and share early visual pathways. But
then are separated into different functions performed by different pathways: form (shape),
motion, color, depth and distance, and cognitive influences: attention and awareness.
Similarly, audition (hearing) all starts with the ear and share early auditory pathways. But
then has separate analyses for the sounds’ location and frequency (pitch).
Vision is the primary sense in humans. Almost every action we make is guided by vision. We
do a lot of analysis of vision. We understand vision very well because it’s easy to study. We
can make visual inputs easily. We can see where the eye is looking, and we can test what a
person can perceive. The visual cortex is also easy to access.
Neural computations
In the earlier stages of vision, it’s even possible to examine the inputs and responses in
neurons in so much detail that we can follow how a pattern of activity in one set of neurons
is analyzed by another set of neurons to give a new type of response.
Dualism is the idea that the mind or spirit is a separate entity from the physical body.
Monism is the idea that the mind is an aspect of the body, held in the brain and nervous
system. Our perception is part of our mind, and in this course, we work from the
fundamental principle that our perception ACTUALLY IS the activity of the neurons in our
brain and nervous system.
Sensation: a translation of the external physical environment into a pattern of neural activity
(by a sensory organ).
Perception: analysis of this neural activity to understand the environment and guide
behavior -> the subjective conscious experience of the outside world.
Sensation and perception reflect interaction between our sensory organs and physical
properties of the world, so they are depended on the physical properties of the world and
limited by the physical properties of our sensors.
,Sensation and perception have evolved to help us survive and reproduce, so they are
optimized for useful (not accurate) representations of the environment, influenced by
interpretation: context and experience, and dependent on limited resources of attention
and awareness. So, all these features of perception suggest that our perception is limited by
the properties of the body and the brain. So, perception is a manifestation of neural activity
in the brain.
So, perception is a translation of the physical environment into a pattern of neural activity
that can be used by our brain to guide behavior. To study perception experimentally we first
change the physical environment of a human or animal (sensory stimulus), and then we can
measure the resulting behavior, or we can measure the resulting change in neural activity.
Psychological approach: quantitative measurements of behavior resulting from perception.
Psychophysics: ‘the scientific study of the relation between stimulus and sensation’.
Psychophysics is best understood as testing whether participants can give an answer that
shows that they perceive the difference between two stimuli -> the ‘Just Noticeable
Difference (JND)’. The variability with which we perceive something depends on the
magnitude of that property or the strength of that stimulus.
Weber- Fechner (psychological) Law describes the relationship between a physical intensity
(I) and its perceived intensity (sensation, klogl). Weber- Fechner determined that this is a
logarithmic relationship: detectable difference increases with average stimulus intensity. We
now know this arises because our sensory receptors tend to follow this logarithmic scaling.
How to determine a perceptual threshold:
The probability that we perceive something isn’t all or nothing. The probability that we
perceive a difference increases as the size of difference increases. The way that
psychophysics experiments work is that they give us two alternatives (yes or no/ brighter or
darker etc.) of the answer that we can give -> 2- alternative forced choice (2AFC). And then
you ask people to decide between them and you force them to make a choice. So, when
they don’t know they have to guess.
Method of constant stimuli: present trials with extent of differences randomized from one
trial to the next. Then you plot detection probability against extent of difference.
è THE PSYCHOMETRIC FUNCTION.
How to determine a perceptual threshold (efficiently)
We estimate parameter of interest (threshold) without determining the entire psychometric
function. This is a widely- used, fast and efficient method. Basically, what you do is you
change the stimulus difference intensity depending on the pattern of people’s previous
responses. This aims to make the next trial maximally informative about the threshold.
Biological approach: what are perception’s neural substrates (i.e., sensory receptors and
brain processing)? Correlate a measure of neural activity with a change in the presented
stimulus or the animal’s behavior (done with lesion studies).
Neural activity is either: spiking activity (action potentials), synaptic activity (synaptic
potentials), or metabolic activity (oxygen & glucose consumption). These are closely related.
,Measuring spiking activity: spikes are the action potentials (firing) of individual neurons.
These are very small changes, so they must be measured directly from the neuron. That
means we need to make invasive recordings inside the brain of an animal or human. Spiking
activity is often seen as the best, clearest measure of neural activity.
Measuring synaptic activity (the communication between cells): several measures at
different scales and resolutions. The smallest of these is the local field potential (LFP).
Spiking, oscillations, and neural interactions
If you have two groups of neurons, some of which are excitatory (so that they will produce
more activity in another group of neurons) and some of them are inhibitory (so that they
will reduce the activity in other neurons). If you have an input in the excitatory pool it will
become active. What it does then is that it is connected to the inhibitory pool, which makes
the inhibitory pool active. What that does is that it inhibits the excitatory pool, so the
excitatory pool of neurons becomes less active and then it stops stimulating the inhibitory
pool of neurons. So that pool of neurons becomes less active too. When the excitatory pool
gets an input, it becomes active again. This goes on and on. This is the wave of the LTP.
EEG measures the LTP from the scalp. It is noninvasive because it’s outside of the head. It
measures them with very poor spatial resolution because you’re measuring from a long way
away. Its advantages are that it’s cheap, it has a high temporal resolution, it moves with the
subject (good for children), and it’s silent (good for auditory perception). Its disadvantages
are that it has a poor spatial resolution, poor signal- to- noise ratio, it only senses activity
near the scalp (cortex), and it’s slow to set up (particularly for large electrode numbers).
Functional MRI advantages are that it has a high spatial resolution, straightforward
analyses/ interpretation, it’s safe and non- invasive, and it has easy access. Its disadvantages
are that it has an indirect measure of neural activity (it measures blood flow), it has low
signal- to- noise ratio, awkward environment, poor temporal resolution, it’s expensive, and
it’s resource- intensive.
MRI scanners: with EEG you know exactly when the change in neural activity happens, and
with fMRI you know where it is. So, it’s a very powerful combination, they cover each other’s
weaknesses when used together.
Signal depends on PD, T1, T2, and T2*
T2* & deoxyhemoglobin
Oxygenated blood? No signal loss
Deoxygenated blood? Signal loss
(1) deoxyhemoglobin affects T2* (MRI signal). (2) blood response follows neural activity. (3)
step 2 overcompensates.
Important: effect 2 doesn’t compensate accurately for effect 1. So, oxyhemoglobin
concentration increases due to increased blood flow. Doesn’t decrease due to oxygen use.
This is what we call the B (blood) O (oxygenation) L (level) D (dependent) signal. This is
what we measure, the increase of blood flow.
, Lecture 2: The eye
Human retina
• Photoreceptor
Visual convergence: horizontal cells, (amacrine cells), bipolar cells, and ganglion cells.
When light reaches our retina
Photoreceptors are on the back of the retina. The photoreceptors are connected to the
middle layer where there’s horizontal cells, bipolar cells, amacrine cells and those connect
to the upper layer where there are ganglion cells, and those make up then the optic neuron.
There is a dip in the fovea. In the fovea we have a high density of cones, the density drops
quickly if you go to the side, or more eccentric, and then the density of rods becomes higher
and takes over. In the fovea we have the sharpest vision
Light is absorbed in the outer segment of the rod (and cone) cells. There is a zillion
membranes in the outer segment and in those membranes are a lot off cross membrane
proteins and those cross-membrane proteins sort of catch the photons. When the light hits
the rod/ photoreceptor (or cone) the rhodopsin is broken down and the cell hyperpolarizes!
(Neurotransmitter release decreases in proportion to amount of light). Light results in
rhodopsin to be broken down and remade -> it’s a loop.
We have 3 types of cones, red cones (long wave), green cones (medium wave), and blue
cones (short wave), and 1 type of rod. We have about 120 million rods and about 6-7 million
cones in our retina. 64% long wave (R), 32% medium wave (G), and 2% short wave (B) rods.
Rods are more sensitive than cones, but insensitive to long wavelengths (anything red).
Appears brighter during daytime (R) than nighttime, dusk (B) -> Purkinje shift.
Rods are also more sensitive cause they have a longer integration ‘widow’.
When light doesn’t reach our retina anymore: the effect of dark adaption.
If one of the cone types (pigments) is missing it means that you can differentiate between
fewer colors. If your L- cone is missing it causes protanopia, if your M-cone is missing it
causes deuteranopia, and if your S-cone is missing it causes tritanopia.
The more light you have the sharper you see. Also, the more photoreceptors, or more
photoreceptors in smaller receptive fields the sharper you see. Acuity (sharpness) and
amount of light help us determine what an image is.
Summary: the light on the retina is ‘noisy’ because of the photon noise. The density of
receptors determines spatial acuity (how sharp you can see). The type of receptor
determines temporal acuity/ sensitivity. The stimulus- point on the retina has a spread with
gradual slope towards un- stimulated area. Therefore, we need more processing.
Adaption and micro- saccades
We don’t perceive light. We perceive changes in light (intensity) across space and or time.
