Lieve Göbbels
Cognitive Science II (JBC090)
Semester 1, 2021-2022
Cognitive Science II
Introduction to Natural Language Processing (NLP) 2
Introduction 2
Why is language dif cult? 2
How do we learn language? 3
A brief history of NLP 4
Text mining preliminaries 4
Collecting Data 6
Noisy text 6
Regular Expressions 6
System evaluation 8
Normalization 8
Classi cation 10
Probability-based representations 10
Smoothing 12
Naive Bayes 14
ID3 Decision Trees 15
Linear and logistic regression 16
SVMs 18
k-Nearest Neighbors 18
Generalization 19
Representation 20
Meaning as counts 20
Predicting word meaning 22
Neural models of language 24
Information extraction 27
NLP for Data Science 27
Sequence classi cation 27
Information extraction 32
Deep Learning 34
Recurrent models 34
Transformers 38
Lab: evaluation 41
Ethics 42
Ethics in computational research 42
Research examples 42
Explainability and digital privacy 43
, Introduction to Natural Language Processing (NLP)
In short:
• Introduction
• Why is language dif cult?
• How do we learn language?
• A brief history of NLP
• Text mining preliminaries
Introduction
This course covers a part of the NLP eld and focuses more on applications than the linguistic part
of this eld. It is discussed how NLP and AI are linked and the role of cognitive science within this.
Several NLP techniques are explained and used for extracting information from language data.
Why is language dif cult?
For processing purposes language is complex, among other things because it is inherently
ambiguous. This is a challenge for text processing practices like sentiment analysis, for one needs to
implement for example sarcasm detection and some extent of world knowledge (or common
knowledge).
CONTRAST
In language objects are described with nouns, actions with verbs, and properties with adjectives.
These are so-called ‘part(s) of speech’ (word categories) and are usually denoted with part of speech
tags (see image). The collection of part(s) of speech from a sentence forms the syntax, or phrase
structure. Next to this, morphology explains the word form and its relations. Semantics (and
pragmatics) explain the meaning of the words, and discourse elaborates on the contextual meaning.
However, most sentences are ambiguous in syntax and semantics (and discourse). For example the
following sentences allow for multiple interpretations that lead to di erent part of speech tags and
overall meaning:
• “They saw a kid with a telescope.”
• “Flying planes can be dangerous.”
• “Time ies like an arrow.”
Cognitive Science II (JBC090)
Semester 1, 2021-2022
Cognitive Science II
Introduction to Natural Language Processing (NLP) 2
Introduction 2
Why is language dif cult? 2
How do we learn language? 3
A brief history of NLP 4
Text mining preliminaries 4
Collecting Data 6
Noisy text 6
Regular Expressions 6
System evaluation 8
Normalization 8
Classi cation 10
Probability-based representations 10
Smoothing 12
Naive Bayes 14
ID3 Decision Trees 15
Linear and logistic regression 16
SVMs 18
k-Nearest Neighbors 18
Generalization 19
Representation 20
Meaning as counts 20
Predicting word meaning 22
Neural models of language 24
Information extraction 27
NLP for Data Science 27
Sequence classi cation 27
Information extraction 32
Deep Learning 34
Recurrent models 34
Transformers 38
Lab: evaluation 41
Ethics 42
Ethics in computational research 42
Research examples 42
Explainability and digital privacy 43
, Introduction to Natural Language Processing (NLP)
In short:
• Introduction
• Why is language dif cult?
• How do we learn language?
• A brief history of NLP
• Text mining preliminaries
Introduction
This course covers a part of the NLP eld and focuses more on applications than the linguistic part
of this eld. It is discussed how NLP and AI are linked and the role of cognitive science within this.
Several NLP techniques are explained and used for extracting information from language data.
Why is language dif cult?
For processing purposes language is complex, among other things because it is inherently
ambiguous. This is a challenge for text processing practices like sentiment analysis, for one needs to
implement for example sarcasm detection and some extent of world knowledge (or common
knowledge).
CONTRAST
In language objects are described with nouns, actions with verbs, and properties with adjectives.
These are so-called ‘part(s) of speech’ (word categories) and are usually denoted with part of speech
tags (see image). The collection of part(s) of speech from a sentence forms the syntax, or phrase
structure. Next to this, morphology explains the word form and its relations. Semantics (and
pragmatics) explain the meaning of the words, and discourse elaborates on the contextual meaning.
However, most sentences are ambiguous in syntax and semantics (and discourse). For example the
following sentences allow for multiple interpretations that lead to di erent part of speech tags and
overall meaning:
• “They saw a kid with a telescope.”
• “Flying planes can be dangerous.”
• “Time ies like an arrow.”
