ISYE 6501 - Midterm 2 Exam 2026
Questions and Answers 100% Pass
Guaranteed
when might overfitting occur - Correct answer-when the # of factors is close to or
larger than the # of data points causing the model to potentially fit too closely to
random effects
Why are simple models better than complex ones - Correct answer-less data is
required; less chance of insignificant factors and easier to interpret
what is forward selection - Correct answer-we select the best new factor and see if
it's good enough (R^2, AIC, or p-value) add it to our model and fit the model with
the current set of factors. Then at the end we remove factors that are lower than a
certain threshold
what is backward elimination - Correct answer-we start with all factors and find
the worst on a supplied threshold (p = 0.15). If it is worse we remove it and start
the process over. We do that until we have the number of factors that we want and
©COPYRIGHT 2025, ALL RIGHTS RESERVE 1
,then we move the factors lower than a second threshold (p = .05) and fit the model
with all set of factors
what is stepwise regression - Correct answer-it is a combination of forward
selection and backward elimination. We can either start with all factors or no
factors and at each step we remove or add a factor. As we go through the procedure
after adding each new factor and at the end we eliminate right away factors that no
longer appear.
what type of algorithms are stepwise selection? - Correct answer-Greedy
algorithms - at each step they take one thing that looks best
what is LASSO - Correct answer-a variable selection method where the
coefficients are determined by both minimizing the squared error and the sum of
their absolute value not being over a certain threshold t
How do you choose t in LASSO - Correct answer-use the lasso approach with
different values of t and see which gives the best trade off
why do we have to scale the data for LASSO - Correct answer-if we don't, the
measure of the data will artificially affect how big the coefficients need to be
©COPYRIGHT 2025, ALL RIGHTS RESERVE 2
, What is elastic net? - Correct answer-A variable selection method that works by
minimizing the squared error and constraining the combination of absolute values
of coefficients and their squares
what is a key difference between stepwise regresson and lasso regression *** -
Correct answer-If the data is not scaled, the coefficients can have artificially
different orders of magnitude, which means they'll have unbalanced effects on the
lasso constraint.
Why doesn't Ridge Regression perform variable selection? - Correct answer-The
coefficients values are squared so they go closer to zero or regularizes them, but
the coefficient values are never equal to zero
What are the pros and cons of Greedy Algorithms (Forward selection, stepwise
elimination, stepwise regression) - Correct answer-Good for initial analysis but
often don't perform as well on other data because they fit more to random effects
than you'd like and appear to have a better fit
What are the pros and cons of LASSO, Ridge and Elastic Net - Correct answer-
They are slower but help make models that make better predictions
Which two methods does elastic net look like it combines and what are the
downsides from it? - Correct answer-Ridge Regression and LASSO.
©COPYRIGHT 2025, ALL RIGHTS RESERVE 3
Questions and Answers 100% Pass
Guaranteed
when might overfitting occur - Correct answer-when the # of factors is close to or
larger than the # of data points causing the model to potentially fit too closely to
random effects
Why are simple models better than complex ones - Correct answer-less data is
required; less chance of insignificant factors and easier to interpret
what is forward selection - Correct answer-we select the best new factor and see if
it's good enough (R^2, AIC, or p-value) add it to our model and fit the model with
the current set of factors. Then at the end we remove factors that are lower than a
certain threshold
what is backward elimination - Correct answer-we start with all factors and find
the worst on a supplied threshold (p = 0.15). If it is worse we remove it and start
the process over. We do that until we have the number of factors that we want and
©COPYRIGHT 2025, ALL RIGHTS RESERVE 1
,then we move the factors lower than a second threshold (p = .05) and fit the model
with all set of factors
what is stepwise regression - Correct answer-it is a combination of forward
selection and backward elimination. We can either start with all factors or no
factors and at each step we remove or add a factor. As we go through the procedure
after adding each new factor and at the end we eliminate right away factors that no
longer appear.
what type of algorithms are stepwise selection? - Correct answer-Greedy
algorithms - at each step they take one thing that looks best
what is LASSO - Correct answer-a variable selection method where the
coefficients are determined by both minimizing the squared error and the sum of
their absolute value not being over a certain threshold t
How do you choose t in LASSO - Correct answer-use the lasso approach with
different values of t and see which gives the best trade off
why do we have to scale the data for LASSO - Correct answer-if we don't, the
measure of the data will artificially affect how big the coefficients need to be
©COPYRIGHT 2025, ALL RIGHTS RESERVE 2
, What is elastic net? - Correct answer-A variable selection method that works by
minimizing the squared error and constraining the combination of absolute values
of coefficients and their squares
what is a key difference between stepwise regresson and lasso regression *** -
Correct answer-If the data is not scaled, the coefficients can have artificially
different orders of magnitude, which means they'll have unbalanced effects on the
lasso constraint.
Why doesn't Ridge Regression perform variable selection? - Correct answer-The
coefficients values are squared so they go closer to zero or regularizes them, but
the coefficient values are never equal to zero
What are the pros and cons of Greedy Algorithms (Forward selection, stepwise
elimination, stepwise regression) - Correct answer-Good for initial analysis but
often don't perform as well on other data because they fit more to random effects
than you'd like and appear to have a better fit
What are the pros and cons of LASSO, Ridge and Elastic Net - Correct answer-
They are slower but help make models that make better predictions
Which two methods does elastic net look like it combines and what are the
downsides from it? - Correct answer-Ridge Regression and LASSO.
©COPYRIGHT 2025, ALL RIGHTS RESERVE 3
