ISYE 6501 - Midterm 2
Exam
** Expert-Verified Explanation
** Questions with Verified Answer
** New Edition | 2026-2027 Updated
** 100% Guaranteed Pass
** 100% Correct Answers
, when might overfitting occur when the # of factors is close to or larger than the # of data points causing the
model to potentially fit too closely to random effects
Why are simple models better than complex ones less data is required; less chance of insignificant factors and easier to interpret
what is forward selection we select the best new factor and see if it's good enough (R^2, AIC, or p-value)
add it to our model and fit the model with the current set of factors. Then at the
end we remove factors that are lower than a certain threshold
what is backward elimination we start with all factors and find the worst on a supplied threshold (p = 0.15). If it
is worse we remove it and start the process over. We do that until we have the
number of factors that we want and then we move the factors lower than a
second threshold (p = .05) and fit the model with all set of factors
what is stepwise regression it is a combination of forward selection and backward elimination. We can either
start with all factors or no factors and at each step we remove or add a factor. As
we go through the procedure after adding each new factor and at the end we
eliminate right away factors that no longer appear.
what type of algorithms are stepwise selection? Greedy algorithms - at each step they take one thing that looks best
what is LASSO a variable selection method where the coefficients are determined by both
minimizing the squared error and the sum of their absolute value not being over
a certain threshold t
How do you choose t in LASSO use the lasso approach with different values of t and see which gives the best
trade off
why do we have to scale the data for LASSO if we don't the measure of the data will artificially affect how big the coefficients
need to be
What is elastic net? A variable selection method that works by minimizing the squared error and
constraining the combination of absolute values of coefficients and their squares
what is a key difference between stepwise regresson If the data is not scaled, the coefficients can have artificially different orders of
and lasso regression magnitude, which means they'll have unbalanced effects on the lasso constraint.
Why doesn't Ridge Regression perform variable The coefficients values are squared so they go closer to zero or regularizes them
selection?
What are the pros and cons of Greedy Algorithms Good for initial analysis but often don't perform as well on other data because
(Forward selection, stepwise elimination, stepwise they fit more to random effects than you'd like and appear to have a better fit
regression)
What are the pros and cons of LASSO and elastic net They are slower but help make models that make better predictions
Which two methods does elastic net look like it combines Ridge Regression and LASSO.
and what are the downsides from it?
Advantages: variable selection from LASSO and Predictive benefits of LASSO.
Disadvantages: Arbitrarily rules out some correlated variables like LASSO (don't
know which one that is left out should be); Underestimates coefficients of very
predictive variables like Ridge Regresison
Exam
** Expert-Verified Explanation
** Questions with Verified Answer
** New Edition | 2026-2027 Updated
** 100% Guaranteed Pass
** 100% Correct Answers
, when might overfitting occur when the # of factors is close to or larger than the # of data points causing the
model to potentially fit too closely to random effects
Why are simple models better than complex ones less data is required; less chance of insignificant factors and easier to interpret
what is forward selection we select the best new factor and see if it's good enough (R^2, AIC, or p-value)
add it to our model and fit the model with the current set of factors. Then at the
end we remove factors that are lower than a certain threshold
what is backward elimination we start with all factors and find the worst on a supplied threshold (p = 0.15). If it
is worse we remove it and start the process over. We do that until we have the
number of factors that we want and then we move the factors lower than a
second threshold (p = .05) and fit the model with all set of factors
what is stepwise regression it is a combination of forward selection and backward elimination. We can either
start with all factors or no factors and at each step we remove or add a factor. As
we go through the procedure after adding each new factor and at the end we
eliminate right away factors that no longer appear.
what type of algorithms are stepwise selection? Greedy algorithms - at each step they take one thing that looks best
what is LASSO a variable selection method where the coefficients are determined by both
minimizing the squared error and the sum of their absolute value not being over
a certain threshold t
How do you choose t in LASSO use the lasso approach with different values of t and see which gives the best
trade off
why do we have to scale the data for LASSO if we don't the measure of the data will artificially affect how big the coefficients
need to be
What is elastic net? A variable selection method that works by minimizing the squared error and
constraining the combination of absolute values of coefficients and their squares
what is a key difference between stepwise regresson If the data is not scaled, the coefficients can have artificially different orders of
and lasso regression magnitude, which means they'll have unbalanced effects on the lasso constraint.
Why doesn't Ridge Regression perform variable The coefficients values are squared so they go closer to zero or regularizes them
selection?
What are the pros and cons of Greedy Algorithms Good for initial analysis but often don't perform as well on other data because
(Forward selection, stepwise elimination, stepwise they fit more to random effects than you'd like and appear to have a better fit
regression)
What are the pros and cons of LASSO and elastic net They are slower but help make models that make better predictions
Which two methods does elastic net look like it combines Ridge Regression and LASSO.
and what are the downsides from it?
Advantages: variable selection from LASSO and Predictive benefits of LASSO.
Disadvantages: Arbitrarily rules out some correlated variables like LASSO (don't
know which one that is left out should be); Underestimates coefficients of very
predictive variables like Ridge Regresison
