CS 7641 MACHINE LEARNING FINAL
EXAM STUDY GUIDE 2026
COMPREHENSIVE REVIEW PRACTICE
QUESTIONS AND ANSWER KEY
Wrapping Methods of Feature Selection - CORRECT
ANSWER-- Hill Climbing
- Randomized Optimization
- Forward/Backward Selection
Strongly relevant - CORRECT ANSWER-Removing xi degrades
the Bayes Optimal Classifier (BOC)
{Recall BOC is the classifier that takes the weighted average of all
hypotheses based on their probability of being the correct
hypothesis}
Usefulness - CORRECT ANSWER-Measures effect on
particular predictor
(MINIMIZING ERROR|LEARNER)
Feature Transformation - CORRECT ANSWER-Pre-processing
a set of features to create a new feature set, while retaining as
much (relevant/useful) information as possible
,Principal Component Analysis - CORRECT ANSWER-Finds
directions that maximize variance and that are mutually
orthogonal
Independent Components Analysis - CORRECT ANSWER-
Finds linear transformation of feature space into new feature
space such that new features are mutually independent (mutual
info I between new pairs is 0, I between old and new is high as
possible)
Random Components Analysis - CORRECT ANSWER-Projects
into random directions
Linear Discriminant Analysis - CORRECT ANSWER-Finds a
projection that discriminates based on the label
Reinforcement Learning - CORRECT ANSWER-Agent must
learn behavior through trial-and-error interactions in a dynamic
environment.
Markov Decision Process (MDP) - CORRECT ANSWER-
Variable Length Encoding - CORRECT ANSWER-Code which
maps source symbols to a variable number of bits (smaller bits for
more frequent symbols)
, kMeans properties - CORRECT ANSWER-- Each iteration
polynomial
- Finite (exponential) iterations
- Error usually decreases monotonically
- Can get stuck
EM Properties - CORRECT ANSWER-- Monotonically non-
decreasing likelihood
- Does not converge (but does practically) b/c of infinite
configurations of probabilities
- Will not diverge
- Can get stuck
- Will work with any distribution
Richness - CORRECT ANSWER-For any assignment of objects
to clusters, there is some distance matrix D such that your
clustering scheme P_D returns the clustering
Scale invariance - CORRECT ANSWER-Scaling distances by a
positive value doesn't change the clustering.
Consistency - CORRECT ANSWER-Shrinking intracluster
distances and expanding intercluster distances does not change
the clustering
EXAM STUDY GUIDE 2026
COMPREHENSIVE REVIEW PRACTICE
QUESTIONS AND ANSWER KEY
Wrapping Methods of Feature Selection - CORRECT
ANSWER-- Hill Climbing
- Randomized Optimization
- Forward/Backward Selection
Strongly relevant - CORRECT ANSWER-Removing xi degrades
the Bayes Optimal Classifier (BOC)
{Recall BOC is the classifier that takes the weighted average of all
hypotheses based on their probability of being the correct
hypothesis}
Usefulness - CORRECT ANSWER-Measures effect on
particular predictor
(MINIMIZING ERROR|LEARNER)
Feature Transformation - CORRECT ANSWER-Pre-processing
a set of features to create a new feature set, while retaining as
much (relevant/useful) information as possible
,Principal Component Analysis - CORRECT ANSWER-Finds
directions that maximize variance and that are mutually
orthogonal
Independent Components Analysis - CORRECT ANSWER-
Finds linear transformation of feature space into new feature
space such that new features are mutually independent (mutual
info I between new pairs is 0, I between old and new is high as
possible)
Random Components Analysis - CORRECT ANSWER-Projects
into random directions
Linear Discriminant Analysis - CORRECT ANSWER-Finds a
projection that discriminates based on the label
Reinforcement Learning - CORRECT ANSWER-Agent must
learn behavior through trial-and-error interactions in a dynamic
environment.
Markov Decision Process (MDP) - CORRECT ANSWER-
Variable Length Encoding - CORRECT ANSWER-Code which
maps source symbols to a variable number of bits (smaller bits for
more frequent symbols)
, kMeans properties - CORRECT ANSWER-- Each iteration
polynomial
- Finite (exponential) iterations
- Error usually decreases monotonically
- Can get stuck
EM Properties - CORRECT ANSWER-- Monotonically non-
decreasing likelihood
- Does not converge (but does practically) b/c of infinite
configurations of probabilities
- Will not diverge
- Can get stuck
- Will work with any distribution
Richness - CORRECT ANSWER-For any assignment of objects
to clusters, there is some distance matrix D such that your
clustering scheme P_D returns the clustering
Scale invariance - CORRECT ANSWER-Scaling distances by a
positive value doesn't change the clustering.
Consistency - CORRECT ANSWER-Shrinking intracluster
distances and expanding intercluster distances does not change
the clustering
