SL Unit Quiz Solutions
CS 7641: Machine Learning
1 Question 1 - Decision Trees
Dataset
Example Weather SoilMoisture Temperature TimeAvailable Garden
1 Sunny Dry Warm Yes Yes
2 Rainy Wet Cool No No
3 Overcast Moist Warm Yes Yes
4 Sunny Dry Hot No No
5 Rainy Wet Mild Yes No
6 Overcast Moist Cool Yes Yes
7 Sunny Moist Mild Yes Yes
Part 2. Decision Tree Applications (MCMA)
Answer the following questions based on the gardening dataset. Select all that apply.
Question 1: Interpreting Leaf Nodes You end up with a leaf node where all examples had
the same label. What can you conclude?
A. This node will not be split further.
B. This node has zero entropy.
C. All features contributed equally to this node.
D. The path to this node may include multiple splits.
E. There must be at least two examples at this node.
F. ID3 assigns the most common label at this node.
Answer: A, B, D
1
, Question 2: Handling Ties in Information Gain Suppose two attributes yield the same
highest information gain. What options does ID3 have?
A. It will choose randomly between them.
B. It will prefer the one with fewer unique values.
C. It will halt tree construction due to ambiguity.
D. It may follow a fixed attribute order as a tiebreaker.
E. It will select the one that appears first in the dataset.
F. It splits on both attributes in parallel.
Answer: A, D, E
Question 3: Decision Trees and Generalization After training the tree on your gardening
data, you test on new data. Which practices help generalization?
A. Prune branches that overfit training data.
B. Avoid splitting if the information gain is very low.
C. Add more attributes to better capture patterns.
D. Limit tree depth or minimum examples per leaf.
E. Include the label as an input to prevent underfitting.
F. Perform cross-validation to tune stopping criteria.
Answer: A, B, D, F
Part 3. ID3 Calculation (with All Attributes)
a) Entropy of the target variable Garden:
- Yes: 4 examples (1, 3, 6, 7) - No: 3 examples (2, 4, 5)
4 4 3 3
H(Garden) = − log2 + log2 ≈ −(0.571 · −0.807 + 0.429 · −1.222) = 0.985
7 7 7 7
b) Information Gain for Weather
- Sunny (1, 4, 7): 2 Yes, 1 No → Entropy = 0.918 - Rainy (2, 5): 0 Yes, 2 No → Entropy =
0 - Overcast (3, 6): 2 Yes, 0 No → Entropy = 0
3 2 2
H(Garden | W eather) = (0.918) + (0) + (0) = 0.393
7 7 7
IG(Garden, W eather) = 0.985 − 0.393 = 0.592
2
CS 7641: Machine Learning
1 Question 1 - Decision Trees
Dataset
Example Weather SoilMoisture Temperature TimeAvailable Garden
1 Sunny Dry Warm Yes Yes
2 Rainy Wet Cool No No
3 Overcast Moist Warm Yes Yes
4 Sunny Dry Hot No No
5 Rainy Wet Mild Yes No
6 Overcast Moist Cool Yes Yes
7 Sunny Moist Mild Yes Yes
Part 2. Decision Tree Applications (MCMA)
Answer the following questions based on the gardening dataset. Select all that apply.
Question 1: Interpreting Leaf Nodes You end up with a leaf node where all examples had
the same label. What can you conclude?
A. This node will not be split further.
B. This node has zero entropy.
C. All features contributed equally to this node.
D. The path to this node may include multiple splits.
E. There must be at least two examples at this node.
F. ID3 assigns the most common label at this node.
Answer: A, B, D
1
, Question 2: Handling Ties in Information Gain Suppose two attributes yield the same
highest information gain. What options does ID3 have?
A. It will choose randomly between them.
B. It will prefer the one with fewer unique values.
C. It will halt tree construction due to ambiguity.
D. It may follow a fixed attribute order as a tiebreaker.
E. It will select the one that appears first in the dataset.
F. It splits on both attributes in parallel.
Answer: A, D, E
Question 3: Decision Trees and Generalization After training the tree on your gardening
data, you test on new data. Which practices help generalization?
A. Prune branches that overfit training data.
B. Avoid splitting if the information gain is very low.
C. Add more attributes to better capture patterns.
D. Limit tree depth or minimum examples per leaf.
E. Include the label as an input to prevent underfitting.
F. Perform cross-validation to tune stopping criteria.
Answer: A, B, D, F
Part 3. ID3 Calculation (with All Attributes)
a) Entropy of the target variable Garden:
- Yes: 4 examples (1, 3, 6, 7) - No: 3 examples (2, 4, 5)
4 4 3 3
H(Garden) = − log2 + log2 ≈ −(0.571 · −0.807 + 0.429 · −1.222) = 0.985
7 7 7 7
b) Information Gain for Weather
- Sunny (1, 4, 7): 2 Yes, 1 No → Entropy = 0.918 - Rainy (2, 5): 0 Yes, 2 No → Entropy =
0 - Overcast (3, 6): 2 Yes, 0 No → Entropy = 0
3 2 2
H(Garden | W eather) = (0.918) + (0) + (0) = 0.393
7 7 7
IG(Garden, W eather) = 0.985 − 0.393 = 0.592
2
