AI Exam 2 Review
Classification
When the output of the hypothesis function is a finite set of values
Regression
When the output of the hypothesis function is a continous value
Skip Ad
Accuracy
(number of correct predictions) / (number of predicitions)
Mean Squared Error
Hypothesis
A function, learned from the training data and a member of the hypothesis space,
that maps inputs to outputs
Bias
The amount by which the output of a hypothesis consistently varies from the true
answer in a particular direction, regardless of the extract training data
Bias refers to a model that is...
underfit
For Linear Regression, what is likely to cause high bias?
Too few training iterations and too few coefficients
Variance
The amount by which the output of a hypothesis randomly varies from the true
answer, when trained on slightly different data sets
Variance refers to a model that is...
overfit
Training set
A set of input-output pair examples, used as input to a machine learning program to
create a hypothesis
Test set
A set of input-output pair examples, used to evaluate the performance of a trained
model
What are indications of overfitting?
Low test accuracy and high train accuracy
What are indications of underfitting?
Low test accuracy and low train accuracy
How do you fight overfitting in kNN models?
Increase the value of k and scale features
How do you fight underfitting in kNN models?
, Decrease the value of k
How do you fight overfitting in linear regression models?
Put bounds on degrees of freedom (regulation)
How do you fight underfitting in linear regression models?
Add more features or polynomial terms to capture nonlinear relationships and
decrease regulation
How do you fight overfitting in decision tree models?
Limit the max depth and increase minimum sample split
How do you fight underfitting in decision tree models?
Increase Model Complexity (depth) and feature engineering
How do you fight overfitting in neural network models?
Regulation and early stopping
How do you fight underfitting in neural network models?
Increase model complexity (more layers, neurons, width of layers) and adjust
learning rate, batch size, and optimizer parameters
Decision Tree Classification
Predicts categorical outcomes, where the target variable is discrete and belongs to a
specific class
Decision Tree Regression
Predicts continous outcome, where the target variable is numeric and can take any
real value within a range
Decision Tree Hyperparameters
max_depth
min_samples_split
min_samples_leaf
max_leaf_nodes
max_features
min_samples_split
min samples a node must have before it can split
min_samples_leaf
min samples a leaf must have
max_features
max number of features evaluated
Classification
When the output of the hypothesis function is a finite set of values
Regression
When the output of the hypothesis function is a continous value
Skip Ad
Accuracy
(number of correct predictions) / (number of predicitions)
Mean Squared Error
Hypothesis
A function, learned from the training data and a member of the hypothesis space,
that maps inputs to outputs
Bias
The amount by which the output of a hypothesis consistently varies from the true
answer in a particular direction, regardless of the extract training data
Bias refers to a model that is...
underfit
For Linear Regression, what is likely to cause high bias?
Too few training iterations and too few coefficients
Variance
The amount by which the output of a hypothesis randomly varies from the true
answer, when trained on slightly different data sets
Variance refers to a model that is...
overfit
Training set
A set of input-output pair examples, used as input to a machine learning program to
create a hypothesis
Test set
A set of input-output pair examples, used to evaluate the performance of a trained
model
What are indications of overfitting?
Low test accuracy and high train accuracy
What are indications of underfitting?
Low test accuracy and low train accuracy
How do you fight overfitting in kNN models?
Increase the value of k and scale features
How do you fight underfitting in kNN models?
, Decrease the value of k
How do you fight overfitting in linear regression models?
Put bounds on degrees of freedom (regulation)
How do you fight underfitting in linear regression models?
Add more features or polynomial terms to capture nonlinear relationships and
decrease regulation
How do you fight overfitting in decision tree models?
Limit the max depth and increase minimum sample split
How do you fight underfitting in decision tree models?
Increase Model Complexity (depth) and feature engineering
How do you fight overfitting in neural network models?
Regulation and early stopping
How do you fight underfitting in neural network models?
Increase model complexity (more layers, neurons, width of layers) and adjust
learning rate, batch size, and optimizer parameters
Decision Tree Classification
Predicts categorical outcomes, where the target variable is discrete and belongs to a
specific class
Decision Tree Regression
Predicts continous outcome, where the target variable is numeric and can take any
real value within a range
Decision Tree Hyperparameters
max_depth
min_samples_split
min_samples_leaf
max_leaf_nodes
max_features
min_samples_split
min samples a node must have before it can split
min_samples_leaf
min samples a leaf must have
max_features
max number of features evaluated
