Les 1: What is AI?
Artificial= something we made
Intelligence= having the ability to understand certain things. Being
skilled.
It all started with the invention of a mechanical calculator and the
automaton in the 1800’s. His work focused on how behavior is influenced
by its consequences, emphasizing reinforcement and punishment as
central to the learning process.
Culloch and Pitss made the first mathematical neuron model. A model
that shows how neurons can learn. Hebb’s most famous contribution is
the concept that "cells that fire together, wire together." This idea is
foundational to our understanding of how learning and memory occur in
the brain through the strengthening of synaptic connections.
Key contributions of Donald Hebb:
1. Hebbian Learning: Hebb proposed that when one neuron
repeatedly activates another, the connection between them
strengthens, making future activations easier.
After the war we can talk about ‘intelligence’ . War always stimulates
technological development. Turing proposed the idea of the Turing Test
to address the question of whether machines can think. The test involves
a machine trying to imitate human conversation well enough that a
human evaluator cannot reliably tell the difference between the machine
and a person. Minsky contributed to the understanding of neural
networks and artificial neurons. He developed one of the first learning
machines, called the SNARC.
Newell and Simon's approach to AI was based on the idea that human
intelligence could be modeled using symbols and rules. This became
known as symbolic AI or the physical symbol system hypothesis. They
argued that intelligence results from the manipulation of symbols
according to rules, and that any system capable of such manipulation
could, in theory, exhibit intelligent behavior.
The physical symbol system hypothesis is considered a cornerstone of
classical AI. It states that symbols (representations of objects, actions,
etc.) and the rules for manipulating them are sufficient for general
intelligent action.
,The birth of AI starts with John Mccarthy. Joseph Weizenbaum creating
ELIZA, an early natural language processing program that simulated
conversation with a human. A programm based on rules. In the 1970’s the
funding for AI dried up! AI Winter
Second AI winter was 1990’s. Keep in mind that the internet and
computer world was emerging in the background which is necessary to
provide a good platform for AI. Imagenet= huge image classification data
set. Gartner’s hype cycle is often used as a sign that a new winter may be
coming.
AGI (Artificial General Intelligence): This refers to a type of AI that
possesses the ability to understand, learn, and apply intelligence across a
wide range of tasks at a level comparable to that of a human being. ASI
(Artificial Superintelligence): This is a theoretical concept that refers
to an AI that surpasses human intelligence and capability in virtually
every aspect, including creativity, problem-solving, and social skills.
Turings view vs Searle’s view. Understanding vs. Behavior: Turing
focuses on behavior as a measure of intelligence, while Searle
argues that understanding and meaning are essential.
Nature of Intelligence: Turing's perspective suggests that
machines could achieve human-like intelligence, whereas Searle
contends that they cannot possess true understanding or
consciousness.
These debates continue to shape discussions on the nature of intelligence,
consciousness, and the capabilities of artificial intelligence.
Focus on ‘rational’ view:
evaluate through well-defined and quantitative criteria
, Les 2: Learning from data
Components of machine learning
The transformation based on data, this process is called ‘learning’.
Looking at machine learning from outside(acting)=> example data and
evalution critreia. Looking at it from inside ( thinking)=> models
and learning. Normally you have the mathematical model that gives you
the result. Training data are the model inputs. Training approach together
with loss function goes back in the mathematical model.
Model= mathematical model + training apporach.Model features=
numerical inputs to the model; “”behaviour= numerical outputs of the
model. Models can overfit on training data: Need separate subsets of data
for training and model evaluation!
Learning=> offline and online. Offline: learning only used collected
training data, no more adaptation. Online is a system with pre-used
data with adaptation. Machine learning is a model for data by tuning
the model parameters, to minimize numerical loss function for a gives set
of data. At condition that the data is collected and stored. Machine
learning occurred early 80’s till 2000’s.
Machine learning: pixels are features. Machine learning models need
numbers as inputs and get numbers as outputs. Class labels are not
numbers.
Supervised learning: In supervised learning, the algorithm learns from
labeled data, where both the input data and the corresponding correct
outputs (labels) are provided. The output is a real number. The model is
trained on a dataset that includes input-output pairs. The goal is to learn a
mapping from inputs to the correct output (label). During training, the
model makes predictions, and based on the errors (difference between
predicted and actual labels), it adjusts to improve. Examples: object
, detection, image classfication. In a supervised learning task, each
training sample consists of the inputs to the model for one observation
(the features) and the desired output of the model for that observation
(the label). The learning algorithm minimises the difference between the
model outputs (predictions) and the labels for the training samples. The
loss is a function of this difference. A regression task is a supervised
learning task for which the labels are continuous (real) numbers. A
classification task is a supervised learning task for which the labels
represent discrete categories (single or multilabel), without making any
assumptions about possible relations between these categories. An
ordinal regression task is a supervised learning task for which the labels
are discrete categories that are strictly ordered.
In unsupervised learning, the algorithm is trained on data without
labeled responses, meaning the model must learn patterns, structures, or
relationships within the data without explicit instructions. The algorithm
tries to find hidden patterns or intrinsic structures in the data. Since no
labels are provided, the model identifies clusters, groupings, or
associations based on similarities or differences in the data. Manifold
learning; where the data lives. In an usupervised learning task, a training
sample consists only of the inputs to the model for one observation (the
features). The learning model puts a hypothesis on the structure of the
data. The learning algorithm optimally matches this structure to the data.
The loss expresses how well the hypothesis matches the data. In
dimensionality reduction, the aim is to transform the features such that a
subset contains as much of the original information as possible. In
clustering, a clustering model and a number of clusters is put forward. The
aim is to tune the clustering model such that the data are optimally
clustered according to some criterion (e.g. optimal separation of the
clusters) In density learning, a model for the joint probability density of
the features is put forward. The parameters of that model are tuned to
optimally match the data. Examples: generating artificial customer data,
there are no labels. It’s a generative model. Recommended music is a mix
of supervised and unsupervised.
Artificial= something we made
Intelligence= having the ability to understand certain things. Being
skilled.
It all started with the invention of a mechanical calculator and the
automaton in the 1800’s. His work focused on how behavior is influenced
by its consequences, emphasizing reinforcement and punishment as
central to the learning process.
Culloch and Pitss made the first mathematical neuron model. A model
that shows how neurons can learn. Hebb’s most famous contribution is
the concept that "cells that fire together, wire together." This idea is
foundational to our understanding of how learning and memory occur in
the brain through the strengthening of synaptic connections.
Key contributions of Donald Hebb:
1. Hebbian Learning: Hebb proposed that when one neuron
repeatedly activates another, the connection between them
strengthens, making future activations easier.
After the war we can talk about ‘intelligence’ . War always stimulates
technological development. Turing proposed the idea of the Turing Test
to address the question of whether machines can think. The test involves
a machine trying to imitate human conversation well enough that a
human evaluator cannot reliably tell the difference between the machine
and a person. Minsky contributed to the understanding of neural
networks and artificial neurons. He developed one of the first learning
machines, called the SNARC.
Newell and Simon's approach to AI was based on the idea that human
intelligence could be modeled using symbols and rules. This became
known as symbolic AI or the physical symbol system hypothesis. They
argued that intelligence results from the manipulation of symbols
according to rules, and that any system capable of such manipulation
could, in theory, exhibit intelligent behavior.
The physical symbol system hypothesis is considered a cornerstone of
classical AI. It states that symbols (representations of objects, actions,
etc.) and the rules for manipulating them are sufficient for general
intelligent action.
,The birth of AI starts with John Mccarthy. Joseph Weizenbaum creating
ELIZA, an early natural language processing program that simulated
conversation with a human. A programm based on rules. In the 1970’s the
funding for AI dried up! AI Winter
Second AI winter was 1990’s. Keep in mind that the internet and
computer world was emerging in the background which is necessary to
provide a good platform for AI. Imagenet= huge image classification data
set. Gartner’s hype cycle is often used as a sign that a new winter may be
coming.
AGI (Artificial General Intelligence): This refers to a type of AI that
possesses the ability to understand, learn, and apply intelligence across a
wide range of tasks at a level comparable to that of a human being. ASI
(Artificial Superintelligence): This is a theoretical concept that refers
to an AI that surpasses human intelligence and capability in virtually
every aspect, including creativity, problem-solving, and social skills.
Turings view vs Searle’s view. Understanding vs. Behavior: Turing
focuses on behavior as a measure of intelligence, while Searle
argues that understanding and meaning are essential.
Nature of Intelligence: Turing's perspective suggests that
machines could achieve human-like intelligence, whereas Searle
contends that they cannot possess true understanding or
consciousness.
These debates continue to shape discussions on the nature of intelligence,
consciousness, and the capabilities of artificial intelligence.
Focus on ‘rational’ view:
evaluate through well-defined and quantitative criteria
, Les 2: Learning from data
Components of machine learning
The transformation based on data, this process is called ‘learning’.
Looking at machine learning from outside(acting)=> example data and
evalution critreia. Looking at it from inside ( thinking)=> models
and learning. Normally you have the mathematical model that gives you
the result. Training data are the model inputs. Training approach together
with loss function goes back in the mathematical model.
Model= mathematical model + training apporach.Model features=
numerical inputs to the model; “”behaviour= numerical outputs of the
model. Models can overfit on training data: Need separate subsets of data
for training and model evaluation!
Learning=> offline and online. Offline: learning only used collected
training data, no more adaptation. Online is a system with pre-used
data with adaptation. Machine learning is a model for data by tuning
the model parameters, to minimize numerical loss function for a gives set
of data. At condition that the data is collected and stored. Machine
learning occurred early 80’s till 2000’s.
Machine learning: pixels are features. Machine learning models need
numbers as inputs and get numbers as outputs. Class labels are not
numbers.
Supervised learning: In supervised learning, the algorithm learns from
labeled data, where both the input data and the corresponding correct
outputs (labels) are provided. The output is a real number. The model is
trained on a dataset that includes input-output pairs. The goal is to learn a
mapping from inputs to the correct output (label). During training, the
model makes predictions, and based on the errors (difference between
predicted and actual labels), it adjusts to improve. Examples: object
, detection, image classfication. In a supervised learning task, each
training sample consists of the inputs to the model for one observation
(the features) and the desired output of the model for that observation
(the label). The learning algorithm minimises the difference between the
model outputs (predictions) and the labels for the training samples. The
loss is a function of this difference. A regression task is a supervised
learning task for which the labels are continuous (real) numbers. A
classification task is a supervised learning task for which the labels
represent discrete categories (single or multilabel), without making any
assumptions about possible relations between these categories. An
ordinal regression task is a supervised learning task for which the labels
are discrete categories that are strictly ordered.
In unsupervised learning, the algorithm is trained on data without
labeled responses, meaning the model must learn patterns, structures, or
relationships within the data without explicit instructions. The algorithm
tries to find hidden patterns or intrinsic structures in the data. Since no
labels are provided, the model identifies clusters, groupings, or
associations based on similarities or differences in the data. Manifold
learning; where the data lives. In an usupervised learning task, a training
sample consists only of the inputs to the model for one observation (the
features). The learning model puts a hypothesis on the structure of the
data. The learning algorithm optimally matches this structure to the data.
The loss expresses how well the hypothesis matches the data. In
dimensionality reduction, the aim is to transform the features such that a
subset contains as much of the original information as possible. In
clustering, a clustering model and a number of clusters is put forward. The
aim is to tune the clustering model such that the data are optimally
clustered according to some criterion (e.g. optimal separation of the
clusters) In density learning, a model for the joint probability density of
the features is put forward. The parameters of that model are tuned to
optimally match the data. Examples: generating artificial customer data,
there are no labels. It’s a generative model. Recommended music is a mix
of supervised and unsupervised.
