Deep learning Summary
Lesson 1
Artificial intelligence is the ability of a computer to perform tasks commonly
associated with intelligent beings.
Machine learning is the study of algorithms that learn from examples and experience
instead of relying on hard-coded rules and make predictions on new data.
Deep learning is a subfield of machine learning focusing on learning data
representations as successive layers of increasingly meaningful representations.
Deep learning
• Evolution of Neural Networks, with more layers/modules
o Multi-layers process for learning rich hierarchical features.
• Create non-linear, hierarchical representations, transforming data into
increasingly abstract forms at each layer.
• DL models adapt to various input/output modalities and sizes, from images to
complex languages structures.
• Viewed as differentiable functional programming, it applies
optimization techniques like gradient descent for efficient model
training:
o The models are constructed using functions that are
differentiable.
Why deep learning
• Better algorithms & understanding
• Increased computing power
• Augmented availability of labelled and unlabelled data
• Development and availability of open-source tools and models
Supervised learning
A supervised learning model:
• Define a mapping from input to output (family of possible
equations)
• Learn the mapping from paired input/output data examples
(find an equation that fits training data well.
Univariate: one output
Multivariate: more than one output
Regression: continuous numbers as output
Classification: discrete classes as output
Binary: two class classification
Multiclass: three or more distinct classes Figure 1: Univariate regression problem
,Examples
Image super resolution: the goal is to improve the clarity and fidelity of an image by
enhancing quality, resolution, or level of detail in a low-resolution image.
Unsupervised learning
Operates on datasets without labelled data. Used for:
• Clustering
• Finding outliers
• Generating new examples
• Filling in missing data.
Clustering
• Involves grouping or partitioning a set of data points into subgroups based on
their similarities.
• Objective is identifying patterns or structures within the data, with the
assumption that data points within the same cluster are more similar to each
other than to those in other clusters.
,Lesson 2
Supervised learning overview
Model:
• The family of mathematical equations defining the mapping
relations between inputs and outputs
• The model consists of a structure and a set of parameters
that are adjusted during training to optimize the mapping.
Training:
• The process of feeding the model with training data, allowing it to learn and
adjust its parameters.
• The goal of training is to identify the set of parameters
that maximize model capability to map inputs to outputs,
ideally generalizing well to new, unseen data.
Inference:
• The process of using a trained model to make predictions
or decisions, typically on new, unseen data.
Supervised regression
• Goal: Predict a continuous outcome variable (dependent variable) based on one
or more predictor variables (independent variables)
• Objective: construct a model that can establish a relationship between the
independent variables and the dependent variable.
• The model is constructed by minimizing a loss function that measures the
discrepancy between the predicted values and the actual values in the training
data.
Notation
𝑎𝑔𝑒
• Input (independent variables): 𝑥 = [𝑚𝑖𝑙𝑒𝑎𝑔𝑒 ]
• Output (dependent variable): 𝑦 = 𝑓 [𝑝𝑟𝑖𝑐𝑒]
• Model 𝑦 = 𝑓[𝑥, ∅]
, Loss function
• Training dataset of I pairs of input/output examples: {𝑥𝑖 , 𝑦𝑖 }𝐼𝑖=1
• The loss function measures the difference between the model’s predictions and
actual outcomes (how bad the model is). Its value depends on:
o The set of parameters controlling the structure of the equation
o The chosen family of mathematical equations
o The training dataset
• 𝐿[∅] returns a scalar that is smaller when the model maps inputs to outputs
better.
Training
Training consists in finding the model’s parameters to minimize the lost function, i.e.:
̂ = argmin[𝐿[∅]]
∅
∅
Testing and Inference
• The trained model will be evaluated on a separate dataset (test dataset) to
assess its performance and generalization capabilities.
• The model will be used in inference to make predictions and comparing these
predictions against the actual values to measure all the relevant metrics.
Example: 1D Linear regression model
Lesson 1
Artificial intelligence is the ability of a computer to perform tasks commonly
associated with intelligent beings.
Machine learning is the study of algorithms that learn from examples and experience
instead of relying on hard-coded rules and make predictions on new data.
Deep learning is a subfield of machine learning focusing on learning data
representations as successive layers of increasingly meaningful representations.
Deep learning
• Evolution of Neural Networks, with more layers/modules
o Multi-layers process for learning rich hierarchical features.
• Create non-linear, hierarchical representations, transforming data into
increasingly abstract forms at each layer.
• DL models adapt to various input/output modalities and sizes, from images to
complex languages structures.
• Viewed as differentiable functional programming, it applies
optimization techniques like gradient descent for efficient model
training:
o The models are constructed using functions that are
differentiable.
Why deep learning
• Better algorithms & understanding
• Increased computing power
• Augmented availability of labelled and unlabelled data
• Development and availability of open-source tools and models
Supervised learning
A supervised learning model:
• Define a mapping from input to output (family of possible
equations)
• Learn the mapping from paired input/output data examples
(find an equation that fits training data well.
Univariate: one output
Multivariate: more than one output
Regression: continuous numbers as output
Classification: discrete classes as output
Binary: two class classification
Multiclass: three or more distinct classes Figure 1: Univariate regression problem
,Examples
Image super resolution: the goal is to improve the clarity and fidelity of an image by
enhancing quality, resolution, or level of detail in a low-resolution image.
Unsupervised learning
Operates on datasets without labelled data. Used for:
• Clustering
• Finding outliers
• Generating new examples
• Filling in missing data.
Clustering
• Involves grouping or partitioning a set of data points into subgroups based on
their similarities.
• Objective is identifying patterns or structures within the data, with the
assumption that data points within the same cluster are more similar to each
other than to those in other clusters.
,Lesson 2
Supervised learning overview
Model:
• The family of mathematical equations defining the mapping
relations between inputs and outputs
• The model consists of a structure and a set of parameters
that are adjusted during training to optimize the mapping.
Training:
• The process of feeding the model with training data, allowing it to learn and
adjust its parameters.
• The goal of training is to identify the set of parameters
that maximize model capability to map inputs to outputs,
ideally generalizing well to new, unseen data.
Inference:
• The process of using a trained model to make predictions
or decisions, typically on new, unseen data.
Supervised regression
• Goal: Predict a continuous outcome variable (dependent variable) based on one
or more predictor variables (independent variables)
• Objective: construct a model that can establish a relationship between the
independent variables and the dependent variable.
• The model is constructed by minimizing a loss function that measures the
discrepancy between the predicted values and the actual values in the training
data.
Notation
𝑎𝑔𝑒
• Input (independent variables): 𝑥 = [𝑚𝑖𝑙𝑒𝑎𝑔𝑒 ]
• Output (dependent variable): 𝑦 = 𝑓 [𝑝𝑟𝑖𝑐𝑒]
• Model 𝑦 = 𝑓[𝑥, ∅]
, Loss function
• Training dataset of I pairs of input/output examples: {𝑥𝑖 , 𝑦𝑖 }𝐼𝑖=1
• The loss function measures the difference between the model’s predictions and
actual outcomes (how bad the model is). Its value depends on:
o The set of parameters controlling the structure of the equation
o The chosen family of mathematical equations
o The training dataset
• 𝐿[∅] returns a scalar that is smaller when the model maps inputs to outputs
better.
Training
Training consists in finding the model’s parameters to minimize the lost function, i.e.:
̂ = argmin[𝐿[∅]]
∅
∅
Testing and Inference
• The trained model will be evaluated on a separate dataset (test dataset) to
assess its performance and generalization capabilities.
• The model will be used in inference to make predictions and comparing these
predictions against the actual values to measure all the relevant metrics.
Example: 1D Linear regression model
