ISYE 6501 - Midterm 2 /
Accurate Expert Verified 150+
Questions & Answers for
Guaranteed Pass | Newest
Update, 2025-2026
Terms in this set (160)
Quiz____?
when might overfitting occur -
Answer✓✓
when the # of factors is close to or larger than the # of data points causing
the model to potentially fit too closely to random effects
Quiz____?
Why are simple models better than complex ones -
Answer✓✓
less data is required; less chance of insignificant factors and easier to
interpret
Quiz____?
what is forward selection -
Answer✓✓
we select the best new factor and see if it's good enough (R^2, AIC, or p-
value) add it to our model and fit the model with the current set of factors.
Then at the end we remove factors that are lower than a certain threshold
, Quiz____?
what is backward elimination -
Answer✓✓
we start with all factors and find the worst on a supplied threshold (p =
0.15). If it is worse we remove it and start the process over. We do that until
we have the number of factors that we want and then we move the factors
lower than a second threshold (p = .05) and fit the model with all set of
factors
Quiz____?
what is stepwise regression -
Answer✓✓
it is a combination of forward selection and backward elimination. We can
either start with all factors or no factors and at each step we remove or add
a factor. As we go through the procedure after adding each new factor and at
the end we eliminate right away factors that no longer appear.
Quiz____?
what type of algorithms are stepwise selection? -
Answer✓✓
Greedy algorithms - at each step they take one thing that looks best
Quiz____?
what is LASSO -
Answer✓✓
a variable selection method where the coefficients are determined by both
minimizing the squared error and the sum of their absolute value not being
over a certain threshold t
, Quiz____?
How do you choose t in LASSO -
Answer✓✓
use the lasso approach with different values of t and see which gives the
best trade off
Quiz____?
why do we have to scale the data for LASSO -
Answer✓✓
if we don't the measure of the data will artificially affect how big the
coefficients need to be
Quiz____?
What is elastic net? -
Answer✓✓
A variable selection method that works by minimizing the squared error and
constraining the combination of absolute values of coefficients and their
squares
Quiz____?
what is a key difference between stepwise regresson and lasso regression -
Answer✓✓
If the data is not scaled, the coefficients can have artificially different orders
of magnitude, which means they'll have unbalanced effects on the lasso
constraint.
Quiz____?
Why doesn't Ridge Regression perform variable selection? -
Answer✓✓
, The coefficients values are squared so they go closer to zero or regularizes
them
Quiz____?
What are the pros and cons of Greedy Algorithms (Forward selection,
stepwise elimination, stepwise regression) -
Answer✓✓
Good for initial analysis but often don't perform as well on other data
because they fit more to random effects than you'd like and appear to have a
better fit
Quiz____?
What are the pros and cons of LASSO and elastic net -
Answer✓✓
They are slower but help make models that make better predictions
Quiz____?
Which two methods does elastic net look like it combines and what are the
downsides from it? -
Answer✓✓
Ridge Regression and LASSO.
Advantages: variable selection from LASSO and Predictive benefits of LASSO.
Disadvantages: Arbitrarily rules out some correlated variables like LASSO
(don't know which one that is left out should be); Underestimates coefficients
of very predictive variables like Ridge Regresison
Quiz____?
Accurate Expert Verified 150+
Questions & Answers for
Guaranteed Pass | Newest
Update, 2025-2026
Terms in this set (160)
Quiz____?
when might overfitting occur -
Answer✓✓
when the # of factors is close to or larger than the # of data points causing
the model to potentially fit too closely to random effects
Quiz____?
Why are simple models better than complex ones -
Answer✓✓
less data is required; less chance of insignificant factors and easier to
interpret
Quiz____?
what is forward selection -
Answer✓✓
we select the best new factor and see if it's good enough (R^2, AIC, or p-
value) add it to our model and fit the model with the current set of factors.
Then at the end we remove factors that are lower than a certain threshold
, Quiz____?
what is backward elimination -
Answer✓✓
we start with all factors and find the worst on a supplied threshold (p =
0.15). If it is worse we remove it and start the process over. We do that until
we have the number of factors that we want and then we move the factors
lower than a second threshold (p = .05) and fit the model with all set of
factors
Quiz____?
what is stepwise regression -
Answer✓✓
it is a combination of forward selection and backward elimination. We can
either start with all factors or no factors and at each step we remove or add
a factor. As we go through the procedure after adding each new factor and at
the end we eliminate right away factors that no longer appear.
Quiz____?
what type of algorithms are stepwise selection? -
Answer✓✓
Greedy algorithms - at each step they take one thing that looks best
Quiz____?
what is LASSO -
Answer✓✓
a variable selection method where the coefficients are determined by both
minimizing the squared error and the sum of their absolute value not being
over a certain threshold t
, Quiz____?
How do you choose t in LASSO -
Answer✓✓
use the lasso approach with different values of t and see which gives the
best trade off
Quiz____?
why do we have to scale the data for LASSO -
Answer✓✓
if we don't the measure of the data will artificially affect how big the
coefficients need to be
Quiz____?
What is elastic net? -
Answer✓✓
A variable selection method that works by minimizing the squared error and
constraining the combination of absolute values of coefficients and their
squares
Quiz____?
what is a key difference between stepwise regresson and lasso regression -
Answer✓✓
If the data is not scaled, the coefficients can have artificially different orders
of magnitude, which means they'll have unbalanced effects on the lasso
constraint.
Quiz____?
Why doesn't Ridge Regression perform variable selection? -
Answer✓✓
, The coefficients values are squared so they go closer to zero or regularizes
them
Quiz____?
What are the pros and cons of Greedy Algorithms (Forward selection,
stepwise elimination, stepwise regression) -
Answer✓✓
Good for initial analysis but often don't perform as well on other data
because they fit more to random effects than you'd like and appear to have a
better fit
Quiz____?
What are the pros and cons of LASSO and elastic net -
Answer✓✓
They are slower but help make models that make better predictions
Quiz____?
Which two methods does elastic net look like it combines and what are the
downsides from it? -
Answer✓✓
Ridge Regression and LASSO.
Advantages: variable selection from LASSO and Predictive benefits of LASSO.
Disadvantages: Arbitrarily rules out some correlated variables like LASSO
(don't know which one that is left out should be); Underestimates coefficients
of very predictive variables like Ridge Regresison
Quiz____?
