CS 7641 FINAL - SUPERVISED LEARNING STUDYGUIDE QUESTIONS AND
CORRECT ANSWERS
Question:
1. What is classification?
Answer:
- Mapping complex inputs to a label
Question:
2. What is regression?
Answer:
- Mapping complex inputs to a numeric value, usually continuous
Question:
3. What is overfitting?
Answer:
`- Model is too closely fit to the training data and no longer works on real-world data
Question:
4. What is cross-validation?
Answer:
- Split the training data into "fake" train and test sets
Question:
5. What are the parts of a classification problem?
Answer:
- Instances, the input data
- The concept, the function the data represents
- The target concept, the answer we want
- The hypotheses -- all possible target concept functions
- Samples paired to the correct output
- A candidate, a potential target concept
- A testing set, to evaluate how close the candidate is to the target concept
Question:
6. How does the ID3 decision tree algorithm work?
Answer:
- Find the best attribute, the one with the most information gain
- Create a branch for each possible value of that attribute
- Lump the training data into its respective branch
- Repeat if there is still more than one example per leaf
Question:
7. How is information gain in decision trees measured?
, Answer:
- Entropy reduction: the greatest decrease in probability of seeing multiple different values
Question:
8. What are the two kinds of bias to worry about when designing a classifier?
Answer:
- Restriction bias: we only consider candidate functions that can be represented by our classifier
- Preference bias: we prefer certain kinds of hypotheses over others
Question:
9. bias of the ID3 decision tree algorithm?
Answer:
- Prefers correct decision trees over incorrect ones
- Prefers shallower trees
Question:
10. How does a (boolean) decision tree handle continuous values?
Answer:
- Binning: ask if the value is greater than or less than a particular value
Question:
11. What is decision tree pruning?
Answer:
- Remove the branches with the least effect on training error
Question:
12. Why is decision tree pruning used?
Answer:
- To avoid overfitting on training data
Question:
13. How do you adapt a decision tree to regression problems?
Answer:
- Information gain is no longer useful, so we can split on the values where the categories have
the least variance or most correlation
Question:
14. When is ensemble learning useful?
Answer:
- When there are features that are inconclusive on their own, but combining them is much more
conclusive
Question:
15. What's the general approach to ensemble learning algorithms?
CORRECT ANSWERS
Question:
1. What is classification?
Answer:
- Mapping complex inputs to a label
Question:
2. What is regression?
Answer:
- Mapping complex inputs to a numeric value, usually continuous
Question:
3. What is overfitting?
Answer:
`- Model is too closely fit to the training data and no longer works on real-world data
Question:
4. What is cross-validation?
Answer:
- Split the training data into "fake" train and test sets
Question:
5. What are the parts of a classification problem?
Answer:
- Instances, the input data
- The concept, the function the data represents
- The target concept, the answer we want
- The hypotheses -- all possible target concept functions
- Samples paired to the correct output
- A candidate, a potential target concept
- A testing set, to evaluate how close the candidate is to the target concept
Question:
6. How does the ID3 decision tree algorithm work?
Answer:
- Find the best attribute, the one with the most information gain
- Create a branch for each possible value of that attribute
- Lump the training data into its respective branch
- Repeat if there is still more than one example per leaf
Question:
7. How is information gain in decision trees measured?
, Answer:
- Entropy reduction: the greatest decrease in probability of seeing multiple different values
Question:
8. What are the two kinds of bias to worry about when designing a classifier?
Answer:
- Restriction bias: we only consider candidate functions that can be represented by our classifier
- Preference bias: we prefer certain kinds of hypotheses over others
Question:
9. bias of the ID3 decision tree algorithm?
Answer:
- Prefers correct decision trees over incorrect ones
- Prefers shallower trees
Question:
10. How does a (boolean) decision tree handle continuous values?
Answer:
- Binning: ask if the value is greater than or less than a particular value
Question:
11. What is decision tree pruning?
Answer:
- Remove the branches with the least effect on training error
Question:
12. Why is decision tree pruning used?
Answer:
- To avoid overfitting on training data
Question:
13. How do you adapt a decision tree to regression problems?
Answer:
- Information gain is no longer useful, so we can split on the values where the categories have
the least variance or most correlation
Question:
14. When is ensemble learning useful?
Answer:
- When there are features that are inconclusive on their own, but combining them is much more
conclusive
Question:
15. What's the general approach to ensemble learning algorithms?
