Study Notes – Key
Algorithms, Math
Foundations & Exam Review
Question 1: In the context of supervised learning, what is the
primary goal of the learning algorithm?
A) To discover hidden patterns and structures in unlabeled data.
B) To learn a mapping function from input features to a target
output variable.
C) To find the optimal policy for an agent interacting with an
environment.
D) To reduce the dimensionality of the input data for visualization.
Answer: B) To learn a mapping function from input features
to a target output variable.
Rationale: Supervised learning is defined by the presence of labeled
training data, where each input has a corresponding correct output.
The primary goal is to learn a function that can accurately predict
the output for new, unseen inputs. Option A describes unsupervised
learning, while Option C describes reinforcement learning. Option
D is a technique often used in unsupervised learning, not the
primary goal of supervised learning.
,Question 2: Which of the following is a key difference between
linear regression and logistic regression?
A) Linear regression predicts continuous values, while logistic
regression predicts discrete values.
B) Linear regression uses a squared error cost function, while
logistic regression uses a cross-entropy cost function.
C) Linear regression is a classification algorithm, while logistic
regression is a regression algorithm.
D) Linear regression has a closed-form solution, while logistic
regression does not.
Answer: B) Linear regression uses a squared error cost
function, while logistic regression uses a cross-entropy cost
function.
Rationale: This is the most fundamental difference. Linear
regression minimizes the sum of squared errors (e.g., Mean Squared
Error), assuming a Gaussian distribution of errors. Logistic
regression, used for binary classification, minimizes the cross-
entropy (or log-loss) cost function, which is derived from the
principle of maximum likelihood. Option A is partially true but
imprecise (logistic regression predicts probabilities, which are
continuous). Option C is false. Option D is generally true (linear
regression has a closed-form Normal Equation) but the cost
function difference is the more defining characteristic of their
functional forms.
,Question 3: What is the primary purpose of regularization in
machine learning models?
A) To increase the training accuracy of the model.
B) To speed up the training process.
C) To prevent overfitting by penalizing complex models.
D) To increase the number of features used by the model.
Answer: C) To prevent overfitting by penalizing complex
models.
Rationale: Regularization (like L1 or L2) adds a penalty term to the
cost function that discourages large weights. This constrains the
model's complexity, forcing it to learn simpler, more generalizable
patterns from the data. This reduces variance and helps prevent
overfitting to the training data. Its purpose is not to increase
training accuracy (which it often reduces) or speed up training (it
often adds a small overhead).
Question 4: In the context of Support Vector Machines (SVMs),
what is the "kernel trick"?
A) A method for selecting the most important support vectors.
B) A technique to train SVMs more efficiently on large datasets.
C) A way to implicitly map data to a higher-dimensional space
without explicitly computing the coordinates.
D) A method for converting a multi-class classification problem
into multiple binary classification problems.
, Answer: C) A way to implicitly map data to a higher-
dimensional space without explicitly computing the
coordinates.
Rationale: The "kernel trick" is a powerful technique used in SVMs
to solve non-linear classification problems. It involves replacing the
dot product in the SVM's objective function with a kernel function.
This kernel function allows the algorithm to operate in a high-
dimensional feature space without ever calculating the coordinates
of the data in that space, which can be computationally infeasible.
This enables SVMs to find non-linear decision boundaries efficiently.
Question 5: Which of the following statements is TRUE about
Principal Component Analysis (PCA)?
A) It is a supervised learning technique for feature selection.
B) It seeks to find the directions (principal components) that
maximize the variance of the data.
C) It is primarily used to improve the performance of a
classification model.
D) It requires the data to be from a non-linear manifold.
Answer: B) It seeks to find the directions (principal
components) that maximize the variance of the data.
Rationale: PCA is a dimensionality reduction technique. Its goal is
to find a new set of orthogonal axes (principal components) in the
data space. The first principal component is the direction of
maximum variance in the data; the second is the direction of