Language
Lecture 1 - Speech Perception
Psycholinguistics: the branch of psychology that is involved in understanding and producing language,
and the link with the brain.
- Broca’s Aphasia
- Problems producing language
• language is telegram style
• low on grammar, function words
• but meaningful
- Language comprehension is pretty much spared
- Inferior frontal gyrus (IFG)
- Wernicke’s Aphasia
- Language production is fluent, and often abundant
• BUT is often lacks meaning
- Language comprehension is also affected
- Posterior superior temporal gyrus (pSTG)
- Wernicke-Lichtheim-Geschwind model
1) Wernicke’s area (auditory input) → phonological lexicon with sounds of words, speech
perception
2) Conceptual representation → meaning
3) Broca’s area (speech output) → speech planning
- Hickok and Poeppel’s Dual Stream Model
It suggests a network of different brain regions, all involved in the process of taking auditory
input and conveying it to linguistic meaning; dorsal and ventral stream.
,Speech
a) Propositions
- Meaningful grammatical units within a sentence: “once you have examined the city”
b) Words (are made up of morphemes)
c) Morphemes
- The smallest meaningful units of language that cannot be further divided or analyzed
as it would convey a different meaning. E.g., “surrounding” -> morphemes would be
“surround” and “ing”.
d) Phonemes
- The sound that makes up a morpheme. Sometimes a phoneme is a morpheme (-s/-y)
- The smallest linguistic units in speech that change the meaning of an expression.
* The basic difference between morphemes and phonemes is that morphemes have
meaning in themselves, while phonemes do not necessarily have meaning, but changing
their sound would change also the meaning of the morpheme they are part of.
- The basic sounds of a language: words are made up of phonemes.
- Phoneme perception
• Minimal Pairs: you make a minimal change in the phoneme to see if the
meaning has changed
Phonology
• Manner of articulation
e.g. plosives (stops; “d”, “t”) and fricatives (“f”, “sh”)
• Place of articulation determines the sound
• these are all ways of describing consonants
- voicing
- voiced vs. voiceless (depending if the vocal cords are activated or not)
e.g. /t/= voiceless; /d/ = voiced, BUT identical in all other phonological regards; place of
articulation (tongue on upper teeth) and type of articulation (plosives)
• Describing vowels:
- possible by position of the tongue
● Variation in speech sounds
- age
- sex
- dialect
- speaking rate
- speech context; “coarticulation” = the acoustic realization of phonemes is influenced by the
surrounding phonemes because we speak in a continuous fashion, already anticipating the sound
of the rest of the word + the shape of the mouth before the actual phoneme can affect the
following one.
,● Phoneme perception → physical properties of a phoneme varies quite dramatically depending
on context, but the perceivers usually have a constant experience.
In fact, the different stimuli represent a continuum in which there is a gradual change, we hear a
sudden transition from one vowel to the other one. This phenomenon is called categorical
perception
*VOT: voice onset times → the time before the vocal cords start vibrating. Very short for voiced
consonants (“d”) and longer for unvoiced consonants (“t”). It is possible to manipulate the VOT
to see categorical perception in which a clear phonetic boundary is present, even though the
physical properties of the phoneme vary gradually.
Dual Stream Model
This model states that Superior Temporal Gyrus (STG) and Superior Temporal Sulcus (STS) are
areas specifically involved in phoneme projection.
Evidence:
1) Chang, Rieger, Johnson et al. 2010
• intracranial high-density cortical surface arrays in posterior (dorsal) STG
• stimuli: phonemes varying gradually from /ba/ to /da/ to /ga/: physical change is continuous
but the perception is categorical
How?
• they used an unsupervised multidimensional scaling (multidimensional pattern analysis) to
, recognize the pattern of brain activation. In 110-150ms the algorithm is very good at
distinguishing the pattern that goes along with /ba/, /da/ and /ga/. it actually categorizes the
sounds categorically as much as a brain would categorize it. Electrical activation around 110ms is
categorical: the auditory input changes gradually, but the brain patterns shift abruptly, similar
to perception. Shows that the brain thus transforms the sound into categories with specific
regions responding to each.
→ STG does show categorical perception
2) Binder et al, 2000
• used Speech and Tones stimuli
- words (sleep)
- nonwords (kleep)
- reversed words
These three stimuli (speech stimuli) have in common that they all are represented by phoneme
or phonemes-like sounds in there.
- tones (nothing speech-like about it)
• measured lateral STG and middle STS
→ those are the areas that are activated by phonemes-like sounds (NOT the tones!)
3) Odaka & Hickok, 2006
• another approach to test where phonological processing takes place:
- use of phonologically complex stimuli vs simple
- neighborhood density (neighbor words can be derived from another word if you change
one phoneme e.g. slip-slap); * it’s known that people are faster to identify words with a smaller
neighborhood because of less competition.
→ STS is specifically extra active with high neighborhood density (smaller neighborhood) words:
phonologically complex words.
4) Formisano et al., 2008
• listeners inside an fMRI scanner hear different vowels
• classification algorithms can successfully categorize vowels based on brain activation patterns
in fMRI even when different speakers were pronouncing the vowels.
• they looked inside the algorithm to be able to find the specific voxels that contributed the most
to successful categorization.
→ STG and STS contribute the most in categorizing vowels and phonemes more in general
Lecture 1 - Speech Perception
Psycholinguistics: the branch of psychology that is involved in understanding and producing language,
and the link with the brain.
- Broca’s Aphasia
- Problems producing language
• language is telegram style
• low on grammar, function words
• but meaningful
- Language comprehension is pretty much spared
- Inferior frontal gyrus (IFG)
- Wernicke’s Aphasia
- Language production is fluent, and often abundant
• BUT is often lacks meaning
- Language comprehension is also affected
- Posterior superior temporal gyrus (pSTG)
- Wernicke-Lichtheim-Geschwind model
1) Wernicke’s area (auditory input) → phonological lexicon with sounds of words, speech
perception
2) Conceptual representation → meaning
3) Broca’s area (speech output) → speech planning
- Hickok and Poeppel’s Dual Stream Model
It suggests a network of different brain regions, all involved in the process of taking auditory
input and conveying it to linguistic meaning; dorsal and ventral stream.
,Speech
a) Propositions
- Meaningful grammatical units within a sentence: “once you have examined the city”
b) Words (are made up of morphemes)
c) Morphemes
- The smallest meaningful units of language that cannot be further divided or analyzed
as it would convey a different meaning. E.g., “surrounding” -> morphemes would be
“surround” and “ing”.
d) Phonemes
- The sound that makes up a morpheme. Sometimes a phoneme is a morpheme (-s/-y)
- The smallest linguistic units in speech that change the meaning of an expression.
* The basic difference between morphemes and phonemes is that morphemes have
meaning in themselves, while phonemes do not necessarily have meaning, but changing
their sound would change also the meaning of the morpheme they are part of.
- The basic sounds of a language: words are made up of phonemes.
- Phoneme perception
• Minimal Pairs: you make a minimal change in the phoneme to see if the
meaning has changed
Phonology
• Manner of articulation
e.g. plosives (stops; “d”, “t”) and fricatives (“f”, “sh”)
• Place of articulation determines the sound
• these are all ways of describing consonants
- voicing
- voiced vs. voiceless (depending if the vocal cords are activated or not)
e.g. /t/= voiceless; /d/ = voiced, BUT identical in all other phonological regards; place of
articulation (tongue on upper teeth) and type of articulation (plosives)
• Describing vowels:
- possible by position of the tongue
● Variation in speech sounds
- age
- sex
- dialect
- speaking rate
- speech context; “coarticulation” = the acoustic realization of phonemes is influenced by the
surrounding phonemes because we speak in a continuous fashion, already anticipating the sound
of the rest of the word + the shape of the mouth before the actual phoneme can affect the
following one.
,● Phoneme perception → physical properties of a phoneme varies quite dramatically depending
on context, but the perceivers usually have a constant experience.
In fact, the different stimuli represent a continuum in which there is a gradual change, we hear a
sudden transition from one vowel to the other one. This phenomenon is called categorical
perception
*VOT: voice onset times → the time before the vocal cords start vibrating. Very short for voiced
consonants (“d”) and longer for unvoiced consonants (“t”). It is possible to manipulate the VOT
to see categorical perception in which a clear phonetic boundary is present, even though the
physical properties of the phoneme vary gradually.
Dual Stream Model
This model states that Superior Temporal Gyrus (STG) and Superior Temporal Sulcus (STS) are
areas specifically involved in phoneme projection.
Evidence:
1) Chang, Rieger, Johnson et al. 2010
• intracranial high-density cortical surface arrays in posterior (dorsal) STG
• stimuli: phonemes varying gradually from /ba/ to /da/ to /ga/: physical change is continuous
but the perception is categorical
How?
• they used an unsupervised multidimensional scaling (multidimensional pattern analysis) to
, recognize the pattern of brain activation. In 110-150ms the algorithm is very good at
distinguishing the pattern that goes along with /ba/, /da/ and /ga/. it actually categorizes the
sounds categorically as much as a brain would categorize it. Electrical activation around 110ms is
categorical: the auditory input changes gradually, but the brain patterns shift abruptly, similar
to perception. Shows that the brain thus transforms the sound into categories with specific
regions responding to each.
→ STG does show categorical perception
2) Binder et al, 2000
• used Speech and Tones stimuli
- words (sleep)
- nonwords (kleep)
- reversed words
These three stimuli (speech stimuli) have in common that they all are represented by phoneme
or phonemes-like sounds in there.
- tones (nothing speech-like about it)
• measured lateral STG and middle STS
→ those are the areas that are activated by phonemes-like sounds (NOT the tones!)
3) Odaka & Hickok, 2006
• another approach to test where phonological processing takes place:
- use of phonologically complex stimuli vs simple
- neighborhood density (neighbor words can be derived from another word if you change
one phoneme e.g. slip-slap); * it’s known that people are faster to identify words with a smaller
neighborhood because of less competition.
→ STS is specifically extra active with high neighborhood density (smaller neighborhood) words:
phonologically complex words.
4) Formisano et al., 2008
• listeners inside an fMRI scanner hear different vowels
• classification algorithms can successfully categorize vowels based on brain activation patterns
in fMRI even when different speakers were pronouncing the vowels.
• they looked inside the algorithm to be able to find the specific voxels that contributed the most
to successful categorization.
→ STG and STS contribute the most in categorizing vowels and phonemes more in general
