ISYE 6501 Final with correct questions
and answers.
,Factor Based Models - correct ans:classification, clustering, regression. Factor
Factor Based Models - correct ans:classification, clustering, regression. Based
Models - correct ans:classification, clustering, regression. Implicitly assumed
that we have a lot of factors in the final model
Why limit number of factors in a model? 2 reasons - correct ans:overfitting:
when # of factors is close to or larger than # of data points. Model may fit
too closely to random effects
simplicity: simple models are usually better
Classical variable selection approaches - correct ans:1. Forward selection
2. Backwards elimination
3. Stepwise regression
greedy algorithms
Backward elimination - correct ans:variable selection; classical
Opposite of forward selection. Start with model with all factors, at each step
find worst factor and remove from model. Continue until no more to add, #
of factor threshold is satisfied. Remove factors at the end that were not good
enough
Forward selection - correct ans:variable selection; classical
Start with model with no factors, at each step find best new factor to add.
Continue until none bad enough to remove, # of factor threshold is satisfied.
Remove factors at the end that were not good enough
Stepwise regression - correct ans:variable selection; classical
Combination of forward selection and backwards elimination. Start with all or
no factors. Each step remove/add a factor. As it continues, after adding in
new factor we eliminate right away any factors that may be good. Helps
model adjust when new factors are added, goodness values change
, Ways of determining if factors are good enough in variable selection - correct
ans:p-value, Rsquared, AIC, BIC
Greedy algorithm - correct ans:At each step, it does the one thing that looks
best
without taking future options into consideration. Good for initial analysis
1. Forward selection
2. Backwards elimination
3. Stepwise regression
Global variable selection approaches - correct ans:1. LASSO
2. Elastic Net
Slower, but tend to give better predictive models
LASSO - correct ans:variable selection; global
- SCALE the date (as with any constrained sum of coefficients)
- add a constraint to the standard regression equation
- minimize sum of squared errors
- T = limit or "budget" on how large the sum of squared errors can get.
Budget will be used on most important coefficients
- Method for limiting the number of variables in a model by limiting the sum
of all coefficients' absolute values. Can be very helpful when number of data
points is less than number of factors.
Elastic Net - correct ans:variable selection; global
- SCALE the date (as with any constrained sum of coefficients)
- T = limit or "budget" on how large the sum of squared errors can get.
Budget will be used on most important coefficients
and answers.
,Factor Based Models - correct ans:classification, clustering, regression. Factor
Factor Based Models - correct ans:classification, clustering, regression. Based
Models - correct ans:classification, clustering, regression. Implicitly assumed
that we have a lot of factors in the final model
Why limit number of factors in a model? 2 reasons - correct ans:overfitting:
when # of factors is close to or larger than # of data points. Model may fit
too closely to random effects
simplicity: simple models are usually better
Classical variable selection approaches - correct ans:1. Forward selection
2. Backwards elimination
3. Stepwise regression
greedy algorithms
Backward elimination - correct ans:variable selection; classical
Opposite of forward selection. Start with model with all factors, at each step
find worst factor and remove from model. Continue until no more to add, #
of factor threshold is satisfied. Remove factors at the end that were not good
enough
Forward selection - correct ans:variable selection; classical
Start with model with no factors, at each step find best new factor to add.
Continue until none bad enough to remove, # of factor threshold is satisfied.
Remove factors at the end that were not good enough
Stepwise regression - correct ans:variable selection; classical
Combination of forward selection and backwards elimination. Start with all or
no factors. Each step remove/add a factor. As it continues, after adding in
new factor we eliminate right away any factors that may be good. Helps
model adjust when new factors are added, goodness values change
, Ways of determining if factors are good enough in variable selection - correct
ans:p-value, Rsquared, AIC, BIC
Greedy algorithm - correct ans:At each step, it does the one thing that looks
best
without taking future options into consideration. Good for initial analysis
1. Forward selection
2. Backwards elimination
3. Stepwise regression
Global variable selection approaches - correct ans:1. LASSO
2. Elastic Net
Slower, but tend to give better predictive models
LASSO - correct ans:variable selection; global
- SCALE the date (as with any constrained sum of coefficients)
- add a constraint to the standard regression equation
- minimize sum of squared errors
- T = limit or "budget" on how large the sum of squared errors can get.
Budget will be used on most important coefficients
- Method for limiting the number of variables in a model by limiting the sum
of all coefficients' absolute values. Can be very helpful when number of data
points is less than number of factors.
Elastic Net - correct ans:variable selection; global
- SCALE the date (as with any constrained sum of coefficients)
- T = limit or "budget" on how large the sum of squared errors can get.
Budget will be used on most important coefficients
