ISYE 6501 MIDTERM 2 EXPECTED
QUESTIONS WITH VERIFIED ANSWERS
Question 1.
when might overfitting occur
ANSWER
when the # of factors is close to or larger than the # of data points causing the model to
potentially fit too closely to random effects
Question 2.
Why are simple models better than complex ones
ANSWER
less data is required; less chance of insignificant factors and easier to interpret
Question 3.
what is forward selection
ANSWER
we select the best new factor and see if it's good enough (R^2, AIC, or p-value) add it to our
model and fit the model with the current set of factors. Then at the end we remove factors
that are lower than a certain threshold
Question 4.
what is backward elimination
ANSWER
we start with all factors and find the worst on a supplied threshold (p = 0.15). If it is worse we
remove it and start the process over. We do that until we have the number of factors that we
want and then we move the factors lower than a second threshold (p = .05) and fit the model
with all set of factors
,Question 5.
what is stepwise regression
ANSWER
it is a combination of forward selection and backward elimination. We can either start with all
factors or no factors and at each step we remove or add a factor. As we go through the
procedure after adding each new factor and at the end we eliminate right away factors that no
longer appear.
Question 6.
what type of algorithms are stepwise selection?
ANSWER
Greedy algorithms - at each step they take one thing that looks best
Question 7.
what is LASSO
ANSWER
a variable selection method where the coefficients are determined by both minimizing the
squared error and the sum of their absolute value not being over a certain threshold t
,Question 8.
How do you choose t in LASSO
ANSWER
use the lasso approach with different values of t and see which gives the best trade off
Question 9.
why do we have to scale the data for LASSO
ANSWER
if we don't the measure of the data will artificially affect how big the coefficients need to be
Question 10.
What is elastic net?
ANSWER
A variable selection method that works by minimizing the squared error and constraining the
combination of absolute values of coefficients and their squares
Question 11.
what is a key difference between stepwise regresson and lasso regression
ANSWER
If the data is not scaled, the coefficients can have artificially different orders of magnitude,
which means they'll have unbalanced effects on the lasso constraint.
Question 12.
Why doesn't Ridge Regression perform variable selection?
ANSWER
The coefficients values are squared so they go closer to zero or regularizes them
, Question 13.
What are the pros and cons of Greedy Algorithms (Forward selection, stepwise elimination, stepwise
regression)
ANSWER
Good for initial analysis but often don't perform as well on other data because they fit more to
random effects than you'd like and appear to have a better fit
Question 14.
What are the pros and cons of LASSO and elastic net
ANSWER
They are slower but help make models that make better predictions
Question 15.
Which two methods does elastic net look like it combines and what are the downsides from it?
ANSWER
Ridge Regression and LASSO.
Advantages: variable selection from LASSO and Predictive benefits of LASSO.
Disadvantages: Arbitrarily rules out some correlated variables like LASSO (don't know which
one that is left out should be); Underestimates coefficients of very predictive variables like
Ridge Regresison
Question 16.
What are some downsides of surveys?
ANSWER
Even if you what appears to be a representative sample in simple ways, maybe it isn't in more
complex ways.
Question 17.
If we're testing to see whether red cars sell for higher prices than blue cars, we need to account for
the type and age of the cars in our data set. This is called:
ANSWER
Controlling
QUESTIONS WITH VERIFIED ANSWERS
Question 1.
when might overfitting occur
ANSWER
when the # of factors is close to or larger than the # of data points causing the model to
potentially fit too closely to random effects
Question 2.
Why are simple models better than complex ones
ANSWER
less data is required; less chance of insignificant factors and easier to interpret
Question 3.
what is forward selection
ANSWER
we select the best new factor and see if it's good enough (R^2, AIC, or p-value) add it to our
model and fit the model with the current set of factors. Then at the end we remove factors
that are lower than a certain threshold
Question 4.
what is backward elimination
ANSWER
we start with all factors and find the worst on a supplied threshold (p = 0.15). If it is worse we
remove it and start the process over. We do that until we have the number of factors that we
want and then we move the factors lower than a second threshold (p = .05) and fit the model
with all set of factors
,Question 5.
what is stepwise regression
ANSWER
it is a combination of forward selection and backward elimination. We can either start with all
factors or no factors and at each step we remove or add a factor. As we go through the
procedure after adding each new factor and at the end we eliminate right away factors that no
longer appear.
Question 6.
what type of algorithms are stepwise selection?
ANSWER
Greedy algorithms - at each step they take one thing that looks best
Question 7.
what is LASSO
ANSWER
a variable selection method where the coefficients are determined by both minimizing the
squared error and the sum of their absolute value not being over a certain threshold t
,Question 8.
How do you choose t in LASSO
ANSWER
use the lasso approach with different values of t and see which gives the best trade off
Question 9.
why do we have to scale the data for LASSO
ANSWER
if we don't the measure of the data will artificially affect how big the coefficients need to be
Question 10.
What is elastic net?
ANSWER
A variable selection method that works by minimizing the squared error and constraining the
combination of absolute values of coefficients and their squares
Question 11.
what is a key difference between stepwise regresson and lasso regression
ANSWER
If the data is not scaled, the coefficients can have artificially different orders of magnitude,
which means they'll have unbalanced effects on the lasso constraint.
Question 12.
Why doesn't Ridge Regression perform variable selection?
ANSWER
The coefficients values are squared so they go closer to zero or regularizes them
, Question 13.
What are the pros and cons of Greedy Algorithms (Forward selection, stepwise elimination, stepwise
regression)
ANSWER
Good for initial analysis but often don't perform as well on other data because they fit more to
random effects than you'd like and appear to have a better fit
Question 14.
What are the pros and cons of LASSO and elastic net
ANSWER
They are slower but help make models that make better predictions
Question 15.
Which two methods does elastic net look like it combines and what are the downsides from it?
ANSWER
Ridge Regression and LASSO.
Advantages: variable selection from LASSO and Predictive benefits of LASSO.
Disadvantages: Arbitrarily rules out some correlated variables like LASSO (don't know which
one that is left out should be); Underestimates coefficients of very predictive variables like
Ridge Regresison
Question 16.
What are some downsides of surveys?
ANSWER
Even if you what appears to be a representative sample in simple ways, maybe it isn't in more
complex ways.
Question 17.
If we're testing to see whether red cars sell for higher prices than blue cars, we need to account for
the type and age of the cars in our data set. This is called:
ANSWER
Controlling
