COMP1008 EXAM QUESTIONS AND
ANSWERS
Differentiate between regression and linear regression - ANSWER-- Regression: the
prediction of a continuous label in supervised learning
- Linear regression: technique to estimate a relationship between an output variable
Y (label) and input feature(s) X (i.e. find a line of best fit)
What is the goal of linear regression? - ANSWER-Minimize the difference between
the predicted label and the actual label (called the mean square error)
What are the pros and cons of linear regression? - ANSWER-- Pros: easy to
interpret, fast to train, can work well even with limited training data
- Cons: applicable only if relationship is linear, sensitive to outliers, relies on
assumptions about the data
Define model tuning - ANSWER-Adjusting model hyperparameters (e.g. depth of
tree)
What is overfitting? - ANSWER-When model fits too well to the training data →
negatively impacts performance on unseen data (poor generalisability!)
Why does overfitting happen? - ANSWER-- Training data is too limited
- Training data is too noisy (irrelevant features)
- Model is too complex
What are the first artificial neurons? - ANSWER-- Most basic units of neural networks
- Combine binary inputs with some weights
- Compare with a fixed threshold to decide if neuron fires or not
How can we calculate whether a (first) artificial neuron will fire or not? - ANSWER--
Binary classification
- let x = input, w = weight, t = threshold
- weighted sum = x1(w1) + x2(w2) + ... + xn(wn)
- weighted sum ≥ t → neuron fires
How can we implement logic functions using the first artificial neurons? - ANSWER-
Using a truth table, implement threshold and weights by hand
What is a Perceptron? (important) - ANSWER-- A learning algorithm that learns the
weights of the binary inputs
- Can converge to implement a function
- Generally, set t = 1, and randomly initialise weights
- As more instances are checked, if they are misclassified then the weights are
adjusted
- Changes the slope of the decision boundary
- Repeated until epoch produces no errors
, Define epoch - ANSWER-The number of passes a training dataset takes around an
algorithm
What is the LR? (in relation to the Perceptron) - ANSWER-Learning rate:
Hyperparameter of the model that controls how much we adjust the weights by each
time an error is produced
What is a limitation of the Perceptron? (important) - ANSWER-Cannot learn non-
linearly seperable functions (meaning the two classes cannot be separated by a
single straight line) (e.g. XOR)
How can perceptrons learn non-linearly seperable functions? - ANSWER-By
connecting them together to form a multilayer perceptron (MLP) which is an artifical
neural network (ANN)
What are some pros and cons of neural networks? - ANSWER-- Pros: can learn
more complicated decision boundaries, can handle a large number of features
- Cons: difficult to interpret ('black box'), difficult to design NN architectures that work
well for a problem (lots of hyperparamters to tune), slow training time
Define search space - ANSWER-Set of all possible solutions to a problem
How can we define a search problem using a search tree? - ANSWER-- Nodes:
possible states
- Branches: operators
- Goal test: test if goal state was reached
- Path cost: how much it costs to take a particular path
Define neighbourhood (in search trees) - ANSWER-All possible states reachable
from a given state
Differentiate between techniques used in supervised learning vs. unsupervised
learning - ANSWER-- Supervised: linear regression, decision trees, ANNs
- Unsupervised: clustering, association rules
What is the aim of data exploration? - ANSWER-Truly understand the data and the
problem; informs what approach(es) we take
What is the is the aim of data preprocessing? - ANSWER-Get the data into a suitable
format for machine learning
Define data mining - ANSWER-Exploration and analysis of large quanities of data to
discover valid, new, useful patterns in data
What is the aim of supervised learning? - ANSWER-Predict the label of new
instances
What is the aim of unsupervised learning? - ANSWER-Uncover hidden patterns
(which are called dependency rules)
ANSWERS
Differentiate between regression and linear regression - ANSWER-- Regression: the
prediction of a continuous label in supervised learning
- Linear regression: technique to estimate a relationship between an output variable
Y (label) and input feature(s) X (i.e. find a line of best fit)
What is the goal of linear regression? - ANSWER-Minimize the difference between
the predicted label and the actual label (called the mean square error)
What are the pros and cons of linear regression? - ANSWER-- Pros: easy to
interpret, fast to train, can work well even with limited training data
- Cons: applicable only if relationship is linear, sensitive to outliers, relies on
assumptions about the data
Define model tuning - ANSWER-Adjusting model hyperparameters (e.g. depth of
tree)
What is overfitting? - ANSWER-When model fits too well to the training data →
negatively impacts performance on unseen data (poor generalisability!)
Why does overfitting happen? - ANSWER-- Training data is too limited
- Training data is too noisy (irrelevant features)
- Model is too complex
What are the first artificial neurons? - ANSWER-- Most basic units of neural networks
- Combine binary inputs with some weights
- Compare with a fixed threshold to decide if neuron fires or not
How can we calculate whether a (first) artificial neuron will fire or not? - ANSWER--
Binary classification
- let x = input, w = weight, t = threshold
- weighted sum = x1(w1) + x2(w2) + ... + xn(wn)
- weighted sum ≥ t → neuron fires
How can we implement logic functions using the first artificial neurons? - ANSWER-
Using a truth table, implement threshold and weights by hand
What is a Perceptron? (important) - ANSWER-- A learning algorithm that learns the
weights of the binary inputs
- Can converge to implement a function
- Generally, set t = 1, and randomly initialise weights
- As more instances are checked, if they are misclassified then the weights are
adjusted
- Changes the slope of the decision boundary
- Repeated until epoch produces no errors
, Define epoch - ANSWER-The number of passes a training dataset takes around an
algorithm
What is the LR? (in relation to the Perceptron) - ANSWER-Learning rate:
Hyperparameter of the model that controls how much we adjust the weights by each
time an error is produced
What is a limitation of the Perceptron? (important) - ANSWER-Cannot learn non-
linearly seperable functions (meaning the two classes cannot be separated by a
single straight line) (e.g. XOR)
How can perceptrons learn non-linearly seperable functions? - ANSWER-By
connecting them together to form a multilayer perceptron (MLP) which is an artifical
neural network (ANN)
What are some pros and cons of neural networks? - ANSWER-- Pros: can learn
more complicated decision boundaries, can handle a large number of features
- Cons: difficult to interpret ('black box'), difficult to design NN architectures that work
well for a problem (lots of hyperparamters to tune), slow training time
Define search space - ANSWER-Set of all possible solutions to a problem
How can we define a search problem using a search tree? - ANSWER-- Nodes:
possible states
- Branches: operators
- Goal test: test if goal state was reached
- Path cost: how much it costs to take a particular path
Define neighbourhood (in search trees) - ANSWER-All possible states reachable
from a given state
Differentiate between techniques used in supervised learning vs. unsupervised
learning - ANSWER-- Supervised: linear regression, decision trees, ANNs
- Unsupervised: clustering, association rules
What is the aim of data exploration? - ANSWER-Truly understand the data and the
problem; informs what approach(es) we take
What is the is the aim of data preprocessing? - ANSWER-Get the data into a suitable
format for machine learning
Define data mining - ANSWER-Exploration and analysis of large quanities of data to
discover valid, new, useful patterns in data
What is the aim of supervised learning? - ANSWER-Predict the label of new
instances
What is the aim of unsupervised learning? - ANSWER-Uncover hidden patterns
(which are called dependency rules)
