Midterm Exam 2: ISYE6501 / ISYE 6501 (Latest
Update ) Intro to Analytics
Modeling | Questions & Answers | Grade A |
100% Correct. Georgia Tech
1. Main reasons to limit number of factors in model: Overfitting - when #
of factors is close to or larger than # of data points; Simplicity - simple
models are better; reduce the number of correlated variables; certain
variables might be hard to collect data or expensive; some variables are
missing data or hard to use
2. Why are simple models better?: Less data is required, less chance of
insignificant factors, easier to interpret
3. Examples of factors that are illegal to use: race, sex, religion, marital
status, or any factors that are highly correlated with forbidden ones
4. What is forward selection?: Start with a model with no factors, at each
step, find each best new factor to add to model, and put it in if good
enough (parameter of your choice like p value<=0.15) improvement;
1
, when there is no factor that is good enough or if we add enough factors,
we stop; can remove any factors at the end
5. What is backward selection?: Start with all factors, and at each step, we
find worst factor, and remove from model and we keep going until no
factor bad enough to remove or we reach the number of factors we want
6. What is stepwise regression?: it is a combination of forward selection
and backward elimination. We can either start with all factors or no factors
and at each step we remove or add a factor. As we go through the
procedure after adding each new factor and at the end we eliminate right
away factors that no longer appear; Allows model to adjust if factor we
earlier thought we needed no longer seems necessary thanks to new
factors added
7. What is a greedy algorithm?: At each step, it does the one thing that
looks best without future options are considered
8. What are potential metrics to determine whether to add or remove
factors in variable selection?: p value, AIC, BIC, R^2
9. What are methods that are based on optimization models that make
decisions globally looking at all options at the same time when it
, comes to variable selection?: LASSO, Elastic Net, Ridge Regression
(not variable selection)
10. LASSO Approach: Adds constraint to standard regression equation: we
want to minimize the sum of squared errors, but also the sum of
coefficients cannot be too large; add threshold t - budget to use on
coefficients; need to scale data - uses absolute value of coefficients
11. Factors to consider when picking the right value of t for LASSO
approach?-
: Number of variables and quality of model
12. Elastic Net: Constrain combination of absolute value of coefficients and
their squares, need to scale data, choose t and lambda values
13. Ridge Regression: Constrain the regression equation by the coefficient
squares - doesn't do variable selection
14 LASSO vs Ridge: The equation to choose coefficients that minimize the
total error is the same; however, the restrictions to solutions are different;
LASSO is based on absolute sum of coefficients and Ridge is based on sum of
squares of coefficients 15. In Lasso, why are variables usually removed and
Update ) Intro to Analytics
Modeling | Questions & Answers | Grade A |
100% Correct. Georgia Tech
1. Main reasons to limit number of factors in model: Overfitting - when #
of factors is close to or larger than # of data points; Simplicity - simple
models are better; reduce the number of correlated variables; certain
variables might be hard to collect data or expensive; some variables are
missing data or hard to use
2. Why are simple models better?: Less data is required, less chance of
insignificant factors, easier to interpret
3. Examples of factors that are illegal to use: race, sex, religion, marital
status, or any factors that are highly correlated with forbidden ones
4. What is forward selection?: Start with a model with no factors, at each
step, find each best new factor to add to model, and put it in if good
enough (parameter of your choice like p value<=0.15) improvement;
1
, when there is no factor that is good enough or if we add enough factors,
we stop; can remove any factors at the end
5. What is backward selection?: Start with all factors, and at each step, we
find worst factor, and remove from model and we keep going until no
factor bad enough to remove or we reach the number of factors we want
6. What is stepwise regression?: it is a combination of forward selection
and backward elimination. We can either start with all factors or no factors
and at each step we remove or add a factor. As we go through the
procedure after adding each new factor and at the end we eliminate right
away factors that no longer appear; Allows model to adjust if factor we
earlier thought we needed no longer seems necessary thanks to new
factors added
7. What is a greedy algorithm?: At each step, it does the one thing that
looks best without future options are considered
8. What are potential metrics to determine whether to add or remove
factors in variable selection?: p value, AIC, BIC, R^2
9. What are methods that are based on optimization models that make
decisions globally looking at all options at the same time when it
, comes to variable selection?: LASSO, Elastic Net, Ridge Regression
(not variable selection)
10. LASSO Approach: Adds constraint to standard regression equation: we
want to minimize the sum of squared errors, but also the sum of
coefficients cannot be too large; add threshold t - budget to use on
coefficients; need to scale data - uses absolute value of coefficients
11. Factors to consider when picking the right value of t for LASSO
approach?-
: Number of variables and quality of model
12. Elastic Net: Constrain combination of absolute value of coefficients and
their squares, need to scale data, choose t and lambda values
13. Ridge Regression: Constrain the regression equation by the coefficient
squares - doesn't do variable selection
14 LASSO vs Ridge: The equation to choose coefficients that minimize the
total error is the same; however, the restrictions to solutions are different;
LASSO is based on absolute sum of coefficients and Ridge is based on sum of
squares of coefficients 15. In Lasso, why are variables usually removed and
