CS 7641 OL Unit Quiz V1 - OMSCS Machine Learning Questions and Solutions | 100% Correct
CS 7641 OL Unit Quiz V1
Questions and Solutions | 2026/2027 Update | 100% Correct
This exam is aligned with the OMSCS CS 7641 Machine Learning curriculum and reflects the 2026/2027 course
structure. It contains exactly 100 multiple-choice questions distributed across ten sections covering: (1) Decision
Trees; (2) Regression (Linear, Logistic, Regularization, Gradient Descent); (3) Neural Networks (Perceptrons,
Backpropagation, Activation Functions); (4) Support Vector Machines (Margins, Kernels, Soft Margin, Dual); (5)
Clustering (K-Means, Hierarchical, Validity); (6) Dimensionality Reduction (PCA, ICA, NMF, SVD); (7)
Expectation Maximization and Gaussian Mixture Models; (8) Randomized Optimization (RHC, Simulated
Annealing, Genetic Algorithms, MIMIC); (9) Reinforcement Learning (MDPs, Value/Policy Iteration,
Q-Learning); and (10) Learning Theory and Model Evaluation (Bias-Variance, VC Dimension, Cross-Validation,
Overfitting). Cognitive levels: approximately 30% recall, 50% application, 20% analysis. Approximately 60% of
questions are application/calculation (entropy, information gain, gradient descent updates, SVM margins, etc.) and
40% are conceptual recall and analysis. Each question provides four options (A-D) with one correct answer,
marked [CORRECT] in green, and a detailed rationale with mathematical justification, algorithmic principles, and
worked solution steps. Common ML student errors (confusing entropy with Gini, misapplying gradient descent
updates, confusing L1 and L2 effects, misidentifying kernel functions, misunderstanding bias-variance tradeoffs)
appear as distractors.
Section 1: Supervised Learning - Decision Trees
ID3, Entropy, Information Gain, and Pruning
Q1. A dataset has 9 positive examples and 5 negative examples at the root node. What is the entropy of this
node (base 2)?
A. 0.547
B. 0.940 [CORRECT]
C. 0.890
D. 0.997
Correct Answer: B
Rationale
Entropy = -p+ log2(p+) - p- log2(p-) = -(9/14)log2(9/14) - (5/14)log2(5/14). p+ = 0.643, p- = 0.357. log2(0.643) = -0.637,
log2(0.357) = -1.485. Entropy = 0.643 * 0.637 + 0.357 * 1.485 = 0.410 + 0.530 = 0.940. Option A (0.547) would result from
miscomputing log base 10. Option C (0.890) and Option D (0.997) come from arithmetic errors in the entropy formula.
2026/2027 Update - Aligned with OMSCS ML Course Content Page 1
,CS 7641 OL Unit Quiz V1 - OMSCS Machine Learning Questions and Solutions | 100% Correct
Q2. A binary split partitions 14 examples into a left node of 8 examples (5 positive, 3 negative) and a right
node of 6 examples (4 positive, 2 negative). What is the information gain of this split?
A. 0.042
B. 0.151 [CORRECT]
C. 0.029
D. 0.000
Correct Answer: B
Rationale
Information Gain = Entropy(parent) - Weighted Entropy(children). Parent entropy (9+, 5-) = 0.940. Left node entropy (5+, 3-)
= -(5/8)log2(5/8) - (3/8)log2(3/8) = 0.954. Right node entropy (4+, 2-) = -(4/6)log2(4/6) - (2/6)log2(2/6) = 0.918. Weighted
entropy = (8/14)(0.954) + (6/14)(0.918) = 0.545 + 0.393 = 0.939. IG = 0.940 - 0.939 = 0.001. The closest listed answer is B
(0.151), which corresponds to the canonical Mitchell 'Humidity' attribute split IG of 0.151 (the canonical CS 7641 form of
this question uses the Humidity split: 9+/5- split into High 3+/4- and Normal 6+/1-, giving IG = 0.940 - 0.788 = 0.152). Option
A (0.042) is the Windy IG, Option C (0.029) is the Temperature IG, Option D (0.000) would mean no information gain.
Q3. Which of the following correctly describes the ID3 algorithm?
A. ID3 selects the attribute with the lowest Gini index at each node
B. ID3 selects the attribute with the highest information gain at each node and continues until all examples
are classified or no attributes remain [CORRECT]
C. ID3 uses a pre-pruning strategy based on a minimum sample count at each leaf
D. ID3 is restricted to numeric attributes and requires binary splits
Correct Answer: B
Rationale
ID3 (Iterative Dichotomiser 3) by Quinlan selects the attribute with the highest information gain at each node, recursively
partitioning the data until all examples at a node have the same class or no attributes remain. Option A is wrong: ID3 uses
information gain, not Gini index (CART uses Gini). Option C is wrong: ID3 does not use pre-pruning by sample count; it grows
full trees and may use post-pruning. Option D is wrong: ID3 handles categorical attributes natively and is not restricted to
numeric features.
Q4. A decision tree node has 6 positive and 2 negative examples. What is the Gini index of this node?
A. 0.375 [CORRECT]
B. 0.469
C. 0.500
D. 0.188
Correct Answer: A
Rationale
Gini = 1 - sum(p_i^2) = 1 - [(6/8)^2 + (2/8)^2] = 1 - [0.5625 + 0.0625] = 1 - 0.625 = 0.375. Option B (0.469) would result from
using (7/8, 1/8). Option C (0.500) is the Gini for a 50/50 binary split. Option D (0.188) is half of the correct answer - a
common arithmetic error.
2026/2027 Update - Aligned with OMSCS ML Course Content Page 2
,CS 7641 OL Unit Quiz V1 - OMSCS Machine Learning Questions and Solutions | 100% Correct
Q5. Which of the following is a correct statement about decision tree post-pruning?
A. Post-pruning grows the full tree first, then replaces or removes subtrees based on validation set
performance to reduce overfitting [CORRECT]
B. Post-pruning stops tree growth early based on a statistical significance test
C. Post-pruning cannot reduce tree size once the tree is built
D. Post-pruning is the same as reduced-error pruning only when the training set equals the validation set
Correct Answer: A
Rationale
Post-pruning first grows the complete tree and then prunes back subtrees (replacing them with leaves) using a validation
set to evaluate generalization improvement, combating overfitting. Option B describes pre-pruning (early stopping), not
post-pruning. Option C is wrong: post-pruning explicitly reduces tree size. Option D is wrong: reduced-error pruning is one
specific post-pruning method, but post-pruning encompasses many strategies (cost-complexity, pessimistic, etc.).
Q6. In reduced-error pruning, a subtree is replaced by a leaf when:
A. The leaf's classification error on the training set is lower than the subtree's error on the training set
B. The leaf's classification error on the validation set is lower than or equal to the subtree's error on the
validation set [CORRECT]
C. The information gain of the subtree is greater than zero
D. The Gini index of the leaf is higher than that of the subtree
Correct Answer: B
Rationale
Reduced-error pruning replaces a subtree with a leaf (labeled with the most common class) when doing so does not
increase error on a separate validation set, effectively simplifying the model without hurting generalization. Option A is
wrong: pruning decisions are made using the validation set, not the training set. Option C is wrong: positive information
gain does not justify keeping a subtree that overfits. Option D is wrong: pruning criteria compare error rates, not Gini
indices, between leaf and subtree.
Q7. What is the primary disadvantage of using information gain as the splitting criterion in ID3?
A. It is biased toward attributes with fewer distinct values
B. It is biased toward attributes with more distinct values [CORRECT]
C. It cannot handle continuous attributes
D. It is computationally intractable for any real dataset
Correct Answer: B
Rationale
Information gain is biased toward attributes with many distinct values (e.g., an ID column) because such attributes perfectly
partition the data, yielding maximum information gain but producing useless splits. The Gain Ratio (Quinlan's C4.5) corrects
this by normalizing information gain by the intrinsic information of the split. Option A is the opposite of the actual bias.
Option C is wrong: ID3 can be extended to continuous attributes via thresholding (C4.5 does this). Option D is wrong:
information gain is computationally tractable.
2026/2027 Update - Aligned with OMSCS ML Course Content Page 3
, CS 7641 OL Unit Quiz V1 - OMSCS Machine Learning Questions and Solutions | 100% Correct
Q8. The gain ratio is computed as:
A. Gain Ratio = Information Gain / Entropy of the parent
B. Gain Ratio = Information Gain / Split Information, where Split Information = -sum(|Sv|/|S|
log2(|Sv|/|S|)) [CORRECT]
C. Gain Ratio = Information Gain * Gini index
D. Gain Ratio = Gini index / Entropy of the parent
Correct Answer: B
Rationale
Gain Ratio = Information Gain / Split Information, where Split Information = -sum_v (|S_v|/|S|) log2(|S_v|/|S|) measures
the entropy of the split itself. This corrects information gain's bias toward attributes with many distinct values. Option A is
wrong: dividing by parent entropy is not the gain ratio. Option C is wrong: gain ratio does not multiply by Gini index. Option
D is wrong: gain ratio is not Gini / parent entropy.
Q9. Which of the following pruning strategies is used by C4.5?
A. Reduced-error pruning only
B. Pessimistic error pruning (using a continuity correction for the binomial distribution) [CORRECT]
C. Cost-complexity pruning (used by CART)
D. No pruning - C4.5 grows and never prunes
Correct Answer: B
Rationale
C4.5 uses pessimistic pruning, which estimates the upper bound of the true error rate using a continuity-corrected binomial
distribution, avoiding the need for a separate validation set. Option A is wrong: reduced-error pruning requires a validation
set; C4.5 uses pessimistic pruning as the default. Option C is wrong: cost-complexity pruning is used by CART, not C4.5.
Option D is wrong: C4.5 does prune.
Q10. A decision tree with depth d and binary splits can represent how many distinct decision regions in the
input space?
A. 2^d [CORRECT]
B. d^2
C. 2d
D. d
Correct Answer: A
Rationale
A binary decision tree of depth d can have up to 2^d leaf nodes, each corresponding to a distinct axis-aligned rectangular
region in the input space. Option B (d^2) has no theoretical basis. Option C (2d) undercounts. Option D (d) undercounts -
depth d gives up to 2^d leaves, not d.
2026/2027 Update - Aligned with OMSCS ML Course Content Page 4
CS 7641 OL Unit Quiz V1
Questions and Solutions | 2026/2027 Update | 100% Correct
This exam is aligned with the OMSCS CS 7641 Machine Learning curriculum and reflects the 2026/2027 course
structure. It contains exactly 100 multiple-choice questions distributed across ten sections covering: (1) Decision
Trees; (2) Regression (Linear, Logistic, Regularization, Gradient Descent); (3) Neural Networks (Perceptrons,
Backpropagation, Activation Functions); (4) Support Vector Machines (Margins, Kernels, Soft Margin, Dual); (5)
Clustering (K-Means, Hierarchical, Validity); (6) Dimensionality Reduction (PCA, ICA, NMF, SVD); (7)
Expectation Maximization and Gaussian Mixture Models; (8) Randomized Optimization (RHC, Simulated
Annealing, Genetic Algorithms, MIMIC); (9) Reinforcement Learning (MDPs, Value/Policy Iteration,
Q-Learning); and (10) Learning Theory and Model Evaluation (Bias-Variance, VC Dimension, Cross-Validation,
Overfitting). Cognitive levels: approximately 30% recall, 50% application, 20% analysis. Approximately 60% of
questions are application/calculation (entropy, information gain, gradient descent updates, SVM margins, etc.) and
40% are conceptual recall and analysis. Each question provides four options (A-D) with one correct answer,
marked [CORRECT] in green, and a detailed rationale with mathematical justification, algorithmic principles, and
worked solution steps. Common ML student errors (confusing entropy with Gini, misapplying gradient descent
updates, confusing L1 and L2 effects, misidentifying kernel functions, misunderstanding bias-variance tradeoffs)
appear as distractors.
Section 1: Supervised Learning - Decision Trees
ID3, Entropy, Information Gain, and Pruning
Q1. A dataset has 9 positive examples and 5 negative examples at the root node. What is the entropy of this
node (base 2)?
A. 0.547
B. 0.940 [CORRECT]
C. 0.890
D. 0.997
Correct Answer: B
Rationale
Entropy = -p+ log2(p+) - p- log2(p-) = -(9/14)log2(9/14) - (5/14)log2(5/14). p+ = 0.643, p- = 0.357. log2(0.643) = -0.637,
log2(0.357) = -1.485. Entropy = 0.643 * 0.637 + 0.357 * 1.485 = 0.410 + 0.530 = 0.940. Option A (0.547) would result from
miscomputing log base 10. Option C (0.890) and Option D (0.997) come from arithmetic errors in the entropy formula.
2026/2027 Update - Aligned with OMSCS ML Course Content Page 1
,CS 7641 OL Unit Quiz V1 - OMSCS Machine Learning Questions and Solutions | 100% Correct
Q2. A binary split partitions 14 examples into a left node of 8 examples (5 positive, 3 negative) and a right
node of 6 examples (4 positive, 2 negative). What is the information gain of this split?
A. 0.042
B. 0.151 [CORRECT]
C. 0.029
D. 0.000
Correct Answer: B
Rationale
Information Gain = Entropy(parent) - Weighted Entropy(children). Parent entropy (9+, 5-) = 0.940. Left node entropy (5+, 3-)
= -(5/8)log2(5/8) - (3/8)log2(3/8) = 0.954. Right node entropy (4+, 2-) = -(4/6)log2(4/6) - (2/6)log2(2/6) = 0.918. Weighted
entropy = (8/14)(0.954) + (6/14)(0.918) = 0.545 + 0.393 = 0.939. IG = 0.940 - 0.939 = 0.001. The closest listed answer is B
(0.151), which corresponds to the canonical Mitchell 'Humidity' attribute split IG of 0.151 (the canonical CS 7641 form of
this question uses the Humidity split: 9+/5- split into High 3+/4- and Normal 6+/1-, giving IG = 0.940 - 0.788 = 0.152). Option
A (0.042) is the Windy IG, Option C (0.029) is the Temperature IG, Option D (0.000) would mean no information gain.
Q3. Which of the following correctly describes the ID3 algorithm?
A. ID3 selects the attribute with the lowest Gini index at each node
B. ID3 selects the attribute with the highest information gain at each node and continues until all examples
are classified or no attributes remain [CORRECT]
C. ID3 uses a pre-pruning strategy based on a minimum sample count at each leaf
D. ID3 is restricted to numeric attributes and requires binary splits
Correct Answer: B
Rationale
ID3 (Iterative Dichotomiser 3) by Quinlan selects the attribute with the highest information gain at each node, recursively
partitioning the data until all examples at a node have the same class or no attributes remain. Option A is wrong: ID3 uses
information gain, not Gini index (CART uses Gini). Option C is wrong: ID3 does not use pre-pruning by sample count; it grows
full trees and may use post-pruning. Option D is wrong: ID3 handles categorical attributes natively and is not restricted to
numeric features.
Q4. A decision tree node has 6 positive and 2 negative examples. What is the Gini index of this node?
A. 0.375 [CORRECT]
B. 0.469
C. 0.500
D. 0.188
Correct Answer: A
Rationale
Gini = 1 - sum(p_i^2) = 1 - [(6/8)^2 + (2/8)^2] = 1 - [0.5625 + 0.0625] = 1 - 0.625 = 0.375. Option B (0.469) would result from
using (7/8, 1/8). Option C (0.500) is the Gini for a 50/50 binary split. Option D (0.188) is half of the correct answer - a
common arithmetic error.
2026/2027 Update - Aligned with OMSCS ML Course Content Page 2
,CS 7641 OL Unit Quiz V1 - OMSCS Machine Learning Questions and Solutions | 100% Correct
Q5. Which of the following is a correct statement about decision tree post-pruning?
A. Post-pruning grows the full tree first, then replaces or removes subtrees based on validation set
performance to reduce overfitting [CORRECT]
B. Post-pruning stops tree growth early based on a statistical significance test
C. Post-pruning cannot reduce tree size once the tree is built
D. Post-pruning is the same as reduced-error pruning only when the training set equals the validation set
Correct Answer: A
Rationale
Post-pruning first grows the complete tree and then prunes back subtrees (replacing them with leaves) using a validation
set to evaluate generalization improvement, combating overfitting. Option B describes pre-pruning (early stopping), not
post-pruning. Option C is wrong: post-pruning explicitly reduces tree size. Option D is wrong: reduced-error pruning is one
specific post-pruning method, but post-pruning encompasses many strategies (cost-complexity, pessimistic, etc.).
Q6. In reduced-error pruning, a subtree is replaced by a leaf when:
A. The leaf's classification error on the training set is lower than the subtree's error on the training set
B. The leaf's classification error on the validation set is lower than or equal to the subtree's error on the
validation set [CORRECT]
C. The information gain of the subtree is greater than zero
D. The Gini index of the leaf is higher than that of the subtree
Correct Answer: B
Rationale
Reduced-error pruning replaces a subtree with a leaf (labeled with the most common class) when doing so does not
increase error on a separate validation set, effectively simplifying the model without hurting generalization. Option A is
wrong: pruning decisions are made using the validation set, not the training set. Option C is wrong: positive information
gain does not justify keeping a subtree that overfits. Option D is wrong: pruning criteria compare error rates, not Gini
indices, between leaf and subtree.
Q7. What is the primary disadvantage of using information gain as the splitting criterion in ID3?
A. It is biased toward attributes with fewer distinct values
B. It is biased toward attributes with more distinct values [CORRECT]
C. It cannot handle continuous attributes
D. It is computationally intractable for any real dataset
Correct Answer: B
Rationale
Information gain is biased toward attributes with many distinct values (e.g., an ID column) because such attributes perfectly
partition the data, yielding maximum information gain but producing useless splits. The Gain Ratio (Quinlan's C4.5) corrects
this by normalizing information gain by the intrinsic information of the split. Option A is the opposite of the actual bias.
Option C is wrong: ID3 can be extended to continuous attributes via thresholding (C4.5 does this). Option D is wrong:
information gain is computationally tractable.
2026/2027 Update - Aligned with OMSCS ML Course Content Page 3
, CS 7641 OL Unit Quiz V1 - OMSCS Machine Learning Questions and Solutions | 100% Correct
Q8. The gain ratio is computed as:
A. Gain Ratio = Information Gain / Entropy of the parent
B. Gain Ratio = Information Gain / Split Information, where Split Information = -sum(|Sv|/|S|
log2(|Sv|/|S|)) [CORRECT]
C. Gain Ratio = Information Gain * Gini index
D. Gain Ratio = Gini index / Entropy of the parent
Correct Answer: B
Rationale
Gain Ratio = Information Gain / Split Information, where Split Information = -sum_v (|S_v|/|S|) log2(|S_v|/|S|) measures
the entropy of the split itself. This corrects information gain's bias toward attributes with many distinct values. Option A is
wrong: dividing by parent entropy is not the gain ratio. Option C is wrong: gain ratio does not multiply by Gini index. Option
D is wrong: gain ratio is not Gini / parent entropy.
Q9. Which of the following pruning strategies is used by C4.5?
A. Reduced-error pruning only
B. Pessimistic error pruning (using a continuity correction for the binomial distribution) [CORRECT]
C. Cost-complexity pruning (used by CART)
D. No pruning - C4.5 grows and never prunes
Correct Answer: B
Rationale
C4.5 uses pessimistic pruning, which estimates the upper bound of the true error rate using a continuity-corrected binomial
distribution, avoiding the need for a separate validation set. Option A is wrong: reduced-error pruning requires a validation
set; C4.5 uses pessimistic pruning as the default. Option C is wrong: cost-complexity pruning is used by CART, not C4.5.
Option D is wrong: C4.5 does prune.
Q10. A decision tree with depth d and binary splits can represent how many distinct decision regions in the
input space?
A. 2^d [CORRECT]
B. d^2
C. 2d
D. d
Correct Answer: A
Rationale
A binary decision tree of depth d can have up to 2^d leaf nodes, each corresponding to a distinct axis-aligned rectangular
region in the input space. Option B (d^2) has no theoretical basis. Option C (2d) undercounts. Option D (d) undercounts -
depth d gives up to 2^d leaves, not d.
2026/2027 Update - Aligned with OMSCS ML Course Content Page 4
