ISYE 6501 FINAL LATEST ACTUAL EXAM WITH
VERIFIED QUESTIONS & CORRECT ANSWERS
Question 1.
Factor Based Models
ANSWER
classification, clustering, regression. Implicitly assumed that we have a lot of factors in the
final model
Question 2.
Why limit number of factors in a model? 2 reasons
ANSWER
overfitting: when # of factors is close to or larger than # of data points. Model may fit too
closely to random effects
simplicity: simple models are usually better
Question 3.
Classical variable selection approaches
ANSWER
1. Forward selection
2. Backwards elimination
3. Stepwise regression
greedy algorithms
Question 4.
Backward elimination
ANSWER
variable selection; classical
Opposite of forward selection. Start with model with all factors, at each step find worst factor
and remove from model. Continue until no more to add, # of factor threshold is satisfied.
Remove factors at the end that were not good enough
,Question 5.
Forward selection
ANSWER
variable selection; classical
Start with model with no factors, at each step find best new factor to add. Continue until none
bad enough to remove, # of factor threshold is satisfied. Remove factors at the end that were
not good enough
Question 6.
Stepwise regression
ANSWER
variable selection; classical
Combination of forward selection and backwards elimination. Start with all or no factors. Each
step remove/add a factor. As it continues, after adding in new factor we eliminate right away
any factors that may be good. Helps model adjust when new factors are added, goodness
values change
Question 7.
Ways of determining if factors are good enough in variable selection
ANSWER
p-value, Rsquared, AIC, BIC
Question 8.
Greedy algorithm
ANSWER
At each step, it does the one thing that looks best
without taking future options into consideration. Good for initial analysis
1. Forward selection
2. Backwards elimination
3. Stepwise regression
, Question 9.
Global variable selection approaches
ANSWER
1. LASSO
2. Elastic Net
Slower, but tend to give better predictive models
Question 10.
LASSO
ANSWER
variable selection; global
- SCALE the date (as with any constrained sum of coefficients)
- add a constraint to the standard regression equation
- minimize sum of squared errors
- T = limit or "budget" on how large the sum of squared errors can get. Budget will be used on
most important coefficients
- Method for limiting the number of variables in a model by limiting the sum of all coefficients'
absolute values. Can be very helpful when number of data points is less than number of
factors.
Question 11.
Elastic Net
ANSWER
variable selection; global
- SCALE the date (as with any constrained sum of coefficients)
- T = limit or "budget" on how large the sum of squared errors can get. Budget will be used on
most important coefficients
- Combination of lasso and ridge regression.
- Variable selection benefits of LASSO
- Predictive benefits of ridge regression
VERIFIED QUESTIONS & CORRECT ANSWERS
Question 1.
Factor Based Models
ANSWER
classification, clustering, regression. Implicitly assumed that we have a lot of factors in the
final model
Question 2.
Why limit number of factors in a model? 2 reasons
ANSWER
overfitting: when # of factors is close to or larger than # of data points. Model may fit too
closely to random effects
simplicity: simple models are usually better
Question 3.
Classical variable selection approaches
ANSWER
1. Forward selection
2. Backwards elimination
3. Stepwise regression
greedy algorithms
Question 4.
Backward elimination
ANSWER
variable selection; classical
Opposite of forward selection. Start with model with all factors, at each step find worst factor
and remove from model. Continue until no more to add, # of factor threshold is satisfied.
Remove factors at the end that were not good enough
,Question 5.
Forward selection
ANSWER
variable selection; classical
Start with model with no factors, at each step find best new factor to add. Continue until none
bad enough to remove, # of factor threshold is satisfied. Remove factors at the end that were
not good enough
Question 6.
Stepwise regression
ANSWER
variable selection; classical
Combination of forward selection and backwards elimination. Start with all or no factors. Each
step remove/add a factor. As it continues, after adding in new factor we eliminate right away
any factors that may be good. Helps model adjust when new factors are added, goodness
values change
Question 7.
Ways of determining if factors are good enough in variable selection
ANSWER
p-value, Rsquared, AIC, BIC
Question 8.
Greedy algorithm
ANSWER
At each step, it does the one thing that looks best
without taking future options into consideration. Good for initial analysis
1. Forward selection
2. Backwards elimination
3. Stepwise regression
, Question 9.
Global variable selection approaches
ANSWER
1. LASSO
2. Elastic Net
Slower, but tend to give better predictive models
Question 10.
LASSO
ANSWER
variable selection; global
- SCALE the date (as with any constrained sum of coefficients)
- add a constraint to the standard regression equation
- minimize sum of squared errors
- T = limit or "budget" on how large the sum of squared errors can get. Budget will be used on
most important coefficients
- Method for limiting the number of variables in a model by limiting the sum of all coefficients'
absolute values. Can be very helpful when number of data points is less than number of
factors.
Question 11.
Elastic Net
ANSWER
variable selection; global
- SCALE the date (as with any constrained sum of coefficients)
- T = limit or "budget" on how large the sum of squared errors can get. Budget will be used on
most important coefficients
- Combination of lasso and ridge regression.
- Variable selection benefits of LASSO
- Predictive benefits of ridge regression
