Contents
Week 1.........................................................................................................2
Week 2.......................................................................................................10
Week 3.......................................................................................................16
Week 4.......................................................................................................25
Week 5.......................................................................................................32
Week 6.......................................................................................................43
Week 7.......................................................................................................51
,Week 1
Goal lecture 1: We will discuss how to deal with missing values, how to
compute the correlation/association between two features, methods to
encode categorical features and handle class imbalance.
Feature = numerical variable (column)
Instances = rows
There are 3 ways to handle missing values:
1. Remove the problem feature containing missing values.
Recommended when there are many missing values for that feature
(not advised)
2. Remove the instances containing missing values. Recommended
when there are many missing values for that feature (not advised)
3. The most popular: replacing the missing values for a given feature
with a representative value such as the mean, the median or the
mode of that feature
But there are also machine learning models that are trained on the non-
missing information!
Autoencoders are deep neural networks that involve two neural blocks
named encoder and decoder.
- The encoder reduces the problem dimensionality
- The decoder completes the pattern.
Feature scaling (so that each feature is In the same
scale)
Normalization
,It allows encoding all numeric features in the [0,1] scale.
Standardization
Similar to the normalization, but the transformed
values might not be in the [0,1] interval.
Correlation between two numerical values
Pearson’s correlation is used when we want to determine the
correlation between two numerical variables given k observations. Only
when the value lies between [-1,1]
Example:
Mean x: 20.67
Mean y: 234,44
Do for each x – xmean and for each y –
ymean. sum all x and y differences and
multiply.
, Association between two categorical (ordinal or nominal) variables
X2 assocation measure is used when we want to measure the
association between two categorical variables given k observations.
Step 1 to make a contingency table:
Step 2:
The expected value is the multiplication of the
individual frequencies divided by the number of
observations.
Example:
Week 1.........................................................................................................2
Week 2.......................................................................................................10
Week 3.......................................................................................................16
Week 4.......................................................................................................25
Week 5.......................................................................................................32
Week 6.......................................................................................................43
Week 7.......................................................................................................51
,Week 1
Goal lecture 1: We will discuss how to deal with missing values, how to
compute the correlation/association between two features, methods to
encode categorical features and handle class imbalance.
Feature = numerical variable (column)
Instances = rows
There are 3 ways to handle missing values:
1. Remove the problem feature containing missing values.
Recommended when there are many missing values for that feature
(not advised)
2. Remove the instances containing missing values. Recommended
when there are many missing values for that feature (not advised)
3. The most popular: replacing the missing values for a given feature
with a representative value such as the mean, the median or the
mode of that feature
But there are also machine learning models that are trained on the non-
missing information!
Autoencoders are deep neural networks that involve two neural blocks
named encoder and decoder.
- The encoder reduces the problem dimensionality
- The decoder completes the pattern.
Feature scaling (so that each feature is In the same
scale)
Normalization
,It allows encoding all numeric features in the [0,1] scale.
Standardization
Similar to the normalization, but the transformed
values might not be in the [0,1] interval.
Correlation between two numerical values
Pearson’s correlation is used when we want to determine the
correlation between two numerical variables given k observations. Only
when the value lies between [-1,1]
Example:
Mean x: 20.67
Mean y: 234,44
Do for each x – xmean and for each y –
ymean. sum all x and y differences and
multiply.
, Association between two categorical (ordinal or nominal) variables
X2 assocation measure is used when we want to measure the
association between two categorical variables given k observations.
Step 1 to make a contingency table:
Step 2:
The expected value is the multiplication of the
individual frequencies divided by the number of
observations.
Example:
