Social Science Economics Econometrics
ISYE 6501 Final
Save
Students also studied
Flashcard sets Study guides
Isye 6501 Final exam MGT 8803 Metrics True/False
323 terms 110 terms 10 terms
Kayla_Patten2 Preview baytoes Preview lukemrudolph
classification, clustering, regression. Implicitly assumed that we
Factor Based Models
have a lot of factors in the final model
overfitting: when # of factors is close to or larger than # of data
Why limit number of factors in a
points. Model may fit too closely to random effects
model? 2 reasons
simplicity: simple models are usually better
1. Forward selection
Classical variable selection 2. Backwards elimination
approaches 3. Stepwise regression
greedy algorithms
variable selection; classical
Opposite of forward selection. Start with model with all factors,
Backward elimination at each step find worst factor and remove from model. Continue
until no more to add, # of factor threshold is satisfied. Remove
factors at the end that were not good enough
variable selection; classical
Start with model with no factors, at each step find best new
Forward selection factor to add. Continue until none bad enough to remove, # of
factor threshold is satisfied. Remove factors at the end that were
not good enough
variable selection; classical
Combination of forward selection and backwards elimination.
Start with all or no factors. Each step remove/add a factor. As it
Stepwise regression
continues, after adding in new factor we eliminate right away any
factors that may be good. Helps model adjust when new factors
are added, goodness values change
Ways of determining if factors p-value, Rsquared, AIC, BIC
are good enough in variable
selection
, At each step, it does the one thing that looks best
without taking future options into consideration. Good for initial
analysis
Greedy algorithm
1. Forward selection
2. Backwards elimination
3. Stepwise regression
1. LASSO
Global variable selection 2. Elastic Net
approaches
Slower, but tend to give better predictive models
variable selection; global
- SCALE the date (as with any
constrained sum of coefficients)
- add a constraint to the standard
regression equation
- minimize sum of squared errors
- T = limit or "budget" on how large the
sum of squared errors can get. Budget
LASSO
will be used on most important
coefficients
- Method for limiting the number of
variables in a model by limiting the
sum of all coefficients’ absolute values.
Can be very helpful when number of
data points is less than number of
factors.
variable selection; global
- SCALE the date (as with any
constrained sum of coefficients)
- T = limit or "budget" on how large the
sum of squared errors can get. Budget
will be used on most important
Elastic Net
coefficients
- Combination of lasso and ridge
regression.
- Variable selection benefits of LASSO
- Predictive benefits of ridge
regression
- Method of regularization by limiting
the sum of the squares of the
coefficients. Will reduce the
magnitude of coefficients, not the
number of variables chosen.
- The quadratic term in ridge
Ridge Regression regression
tends to shrink the coefficient values
i.e Whatever the basic regression
model coefficients would be,
the quadratic constraint pushes them
toward zero
or regularizes them.
ISYE 6501 Final
Save
Students also studied
Flashcard sets Study guides
Isye 6501 Final exam MGT 8803 Metrics True/False
323 terms 110 terms 10 terms
Kayla_Patten2 Preview baytoes Preview lukemrudolph
classification, clustering, regression. Implicitly assumed that we
Factor Based Models
have a lot of factors in the final model
overfitting: when # of factors is close to or larger than # of data
Why limit number of factors in a
points. Model may fit too closely to random effects
model? 2 reasons
simplicity: simple models are usually better
1. Forward selection
Classical variable selection 2. Backwards elimination
approaches 3. Stepwise regression
greedy algorithms
variable selection; classical
Opposite of forward selection. Start with model with all factors,
Backward elimination at each step find worst factor and remove from model. Continue
until no more to add, # of factor threshold is satisfied. Remove
factors at the end that were not good enough
variable selection; classical
Start with model with no factors, at each step find best new
Forward selection factor to add. Continue until none bad enough to remove, # of
factor threshold is satisfied. Remove factors at the end that were
not good enough
variable selection; classical
Combination of forward selection and backwards elimination.
Start with all or no factors. Each step remove/add a factor. As it
Stepwise regression
continues, after adding in new factor we eliminate right away any
factors that may be good. Helps model adjust when new factors
are added, goodness values change
Ways of determining if factors p-value, Rsquared, AIC, BIC
are good enough in variable
selection
, At each step, it does the one thing that looks best
without taking future options into consideration. Good for initial
analysis
Greedy algorithm
1. Forward selection
2. Backwards elimination
3. Stepwise regression
1. LASSO
Global variable selection 2. Elastic Net
approaches
Slower, but tend to give better predictive models
variable selection; global
- SCALE the date (as with any
constrained sum of coefficients)
- add a constraint to the standard
regression equation
- minimize sum of squared errors
- T = limit or "budget" on how large the
sum of squared errors can get. Budget
LASSO
will be used on most important
coefficients
- Method for limiting the number of
variables in a model by limiting the
sum of all coefficients’ absolute values.
Can be very helpful when number of
data points is less than number of
factors.
variable selection; global
- SCALE the date (as with any
constrained sum of coefficients)
- T = limit or "budget" on how large the
sum of squared errors can get. Budget
will be used on most important
Elastic Net
coefficients
- Combination of lasso and ridge
regression.
- Variable selection benefits of LASSO
- Predictive benefits of ridge
regression
- Method of regularization by limiting
the sum of the squares of the
coefficients. Will reduce the
magnitude of coefficients, not the
number of variables chosen.
- The quadratic term in ridge
Ridge Regression regression
tends to shrink the coefficient values
i.e Whatever the basic regression
model coefficients would be,
the quadratic constraint pushes them
toward zero
or regularizes them.
