ISYE 6501 FINAL UPDATED ACTUAL EXAM QUESTIONS
AND CORRECT ANSWERS
Question 1.
Factor Based Models
ANSWER
classification, clustering, regression. Implicitly assumed that we have a lot of factors in the final
model
Question 2.
Why limit number of factors in a model? 2 reasons
ANSWER
overfitting: when # of factors is close to or larger than # of data points. Model may fit too
closely to random effects simplicity: simple models are usually better
Question 3.
Classical variable selection ap proaches
ANSWER
1. Forward selection
-
2. Backwards elimination
3. Stepwise regression greedy algorithms
Question 4.
Backward elimination
ANSWER
variable selection; classical Opposite of forward selection. Start with model with all factors, at
each step find worst factor and remove from model. Continue until no more to add, # of factor
threshold is satisfied. Remove factors at the end that were not good enough
,Question 5.
Forward selection
ANSWER
variable selection; classical Start with model with no factors, at each step find best new factor
to add. Continue until none bad enough to remove, # of factor threshold is satisfied. Remove
factors at the end that were not good enough
Question 6.
Stepwise regression
ANSWER
variable selection; classical Combination of forward selection and backwards elimination. Start
with all or no factors. Each step remove/add a factor. As it continues, after adding in new factor
we eliminate right away any factors that may be good. Helps model adjust when new factors
are added, goodness values change
Question 7.
Ways of determining if factors are good enough in variable selection
ANSWER
p-value, Rsquared, AIC, BIC
Question 8.
Greedy algorithm
ANSWER
At each step, it does the one thing that looks best without taking future options into
consideration. Good for initial analysis
1. Forward selection
2. Backwards elimination
3. Stepwise regression
Question 9.
Global variable selection approaches
ANSWER
1. LASSO
2. Elastic Net Slower, but tend to give better predictive models
, Question 10.
LASSO
ANSWER
variable selection; global
- SCALE the date (as with any constrained sum of coefficients)
- add a constraint to the standard regression equation
- minimize sum of squared errors
- T = limit or "budget" on how large the sum of squared errors can get. Budget will be used on
most important coefficients
- Method for limiting the number of variables in a model by limiting the sum of all coefficients'
absolute values. Can be very helpful when number of data points is less than number of
factors.
Question 11.
Elastic Net
ANSWER
variable selection; global
- SCALE the date (as with any constrained sum of coefficients)
- T = limit or "budget" on how large the sum of squared errors can get. Budget will be used on
most important coefficients
- Combination of lasso and ridge regression.
- Variable selection benefits of LASSO
- Predictive benefits of ridge regression
Question 12.
Ridge Regression
ANSWER
AND CORRECT ANSWERS
Question 1.
Factor Based Models
ANSWER
classification, clustering, regression. Implicitly assumed that we have a lot of factors in the final
model
Question 2.
Why limit number of factors in a model? 2 reasons
ANSWER
overfitting: when # of factors is close to or larger than # of data points. Model may fit too
closely to random effects simplicity: simple models are usually better
Question 3.
Classical variable selection ap proaches
ANSWER
1. Forward selection
-
2. Backwards elimination
3. Stepwise regression greedy algorithms
Question 4.
Backward elimination
ANSWER
variable selection; classical Opposite of forward selection. Start with model with all factors, at
each step find worst factor and remove from model. Continue until no more to add, # of factor
threshold is satisfied. Remove factors at the end that were not good enough
,Question 5.
Forward selection
ANSWER
variable selection; classical Start with model with no factors, at each step find best new factor
to add. Continue until none bad enough to remove, # of factor threshold is satisfied. Remove
factors at the end that were not good enough
Question 6.
Stepwise regression
ANSWER
variable selection; classical Combination of forward selection and backwards elimination. Start
with all or no factors. Each step remove/add a factor. As it continues, after adding in new factor
we eliminate right away any factors that may be good. Helps model adjust when new factors
are added, goodness values change
Question 7.
Ways of determining if factors are good enough in variable selection
ANSWER
p-value, Rsquared, AIC, BIC
Question 8.
Greedy algorithm
ANSWER
At each step, it does the one thing that looks best without taking future options into
consideration. Good for initial analysis
1. Forward selection
2. Backwards elimination
3. Stepwise regression
Question 9.
Global variable selection approaches
ANSWER
1. LASSO
2. Elastic Net Slower, but tend to give better predictive models
, Question 10.
LASSO
ANSWER
variable selection; global
- SCALE the date (as with any constrained sum of coefficients)
- add a constraint to the standard regression equation
- minimize sum of squared errors
- T = limit or "budget" on how large the sum of squared errors can get. Budget will be used on
most important coefficients
- Method for limiting the number of variables in a model by limiting the sum of all coefficients'
absolute values. Can be very helpful when number of data points is less than number of
factors.
Question 11.
Elastic Net
ANSWER
variable selection; global
- SCALE the date (as with any constrained sum of coefficients)
- T = limit or "budget" on how large the sum of squared errors can get. Budget will be used on
most important coefficients
- Combination of lasso and ridge regression.
- Variable selection benefits of LASSO
- Predictive benefits of ridge regression
Question 12.
Ridge Regression
ANSWER
