Final Quiz ISYE 6501 | Questions and
Answers | 2026 Update | 100%
Correct.
Section 1: Linear Regression & Model Selection
Q: In a linear regression model, what does the p-value for a
coefficient indicate?
Answer: The probability of observing the coefficient (or more
extreme) if the true coefficient is zero.
Rationale: A low p-value (typically < 0.05) suggests the predictor
is statistically significant, meaning there is evidence that the
coefficient is not zero.
Q: What does the coefficient of determination (R²) represent
in linear regression?
Answer: The proportion of variance in the dependent variable
explained by the model.
Rationale: R² quantifies the percentage of total variation in the
response variable that is accounted for by the predictors. It ranges
from 0 to 1, where higher values indicate better fit.
,Q: Which of the following is a key assumption of ordinary
least squares (OLS) linear regression?
Answer: The errors (residuals) are homoscedastic and
uncorrelated.
Rationale: Key assumptions of OLS include linearity,
independence of errors, homoscedasticity (constant variance of
errors), and normality of errors for inference. Predictors do not
need to be normally distributed.
Q: If there are variables that need to be used to control the
bias selection in the model, they should be:
Answer: Forced to be in the model and not part of the variable
selection process.
Rationale: Control variables must remain in the model regardless
of statistical significance to prevent omitted variable bias.
Q: Penalization in linear regression models means penalizing
for:
Answer: Complex models, that is, models with a large number of
predictors.
Rationale: Penalization adds a penalty term to the loss function
to discourage overfitting by shrinking coefficient estimates.
,Q: Elastic net regression uses which penalties and combines
the benefits of both?
Answer: Both penalties of ridge (L2) and lasso (L1) regression.
Rationale: Elastic net combines L1 and L2 penalties, allowing for
both variable selection (like lasso) and handling of correlated
predictors (like ridge).
Q: Variable selection can be applied to regression problems
when the number of predicting variables is larger than the
number of observations.
Answer: True.
Rationale: Variable selection methods like lasso and stepwise can
handle p > n situations.
Q: The lasso regression performs well under multicollinearity.
Answer: False.
Rationale: Lasso tends to arbitrarily select one variable among a
group of highly correlated predictors and ignore the others.
, Q: The selected variables using best subset regression are the
best ones in explaining and predicting the response variables.
Answer: False.
Rationale: Best subset selection finds the model with the best fit
for a given subset size, but this does not guarantee the best
predictive performance on new data.
Q: The lasso regression requires a numerical algorithm to
minimize the penalized sum of least squares.
Answer: True.
Rationale: There is no closed-form solution for lasso regression
due to the L1 penalty; numerical optimization (e.g., coordinate
descent) is required.
Q: We cannot perform variable selection based on the
statistical significance of the regression coefficients.
Answer: True.
Rationale: Stepwise selection based on p-values is unreliable due
to multiple testing issues and can lead to overfitting.
Answers | 2026 Update | 100%
Correct.
Section 1: Linear Regression & Model Selection
Q: In a linear regression model, what does the p-value for a
coefficient indicate?
Answer: The probability of observing the coefficient (or more
extreme) if the true coefficient is zero.
Rationale: A low p-value (typically < 0.05) suggests the predictor
is statistically significant, meaning there is evidence that the
coefficient is not zero.
Q: What does the coefficient of determination (R²) represent
in linear regression?
Answer: The proportion of variance in the dependent variable
explained by the model.
Rationale: R² quantifies the percentage of total variation in the
response variable that is accounted for by the predictors. It ranges
from 0 to 1, where higher values indicate better fit.
,Q: Which of the following is a key assumption of ordinary
least squares (OLS) linear regression?
Answer: The errors (residuals) are homoscedastic and
uncorrelated.
Rationale: Key assumptions of OLS include linearity,
independence of errors, homoscedasticity (constant variance of
errors), and normality of errors for inference. Predictors do not
need to be normally distributed.
Q: If there are variables that need to be used to control the
bias selection in the model, they should be:
Answer: Forced to be in the model and not part of the variable
selection process.
Rationale: Control variables must remain in the model regardless
of statistical significance to prevent omitted variable bias.
Q: Penalization in linear regression models means penalizing
for:
Answer: Complex models, that is, models with a large number of
predictors.
Rationale: Penalization adds a penalty term to the loss function
to discourage overfitting by shrinking coefficient estimates.
,Q: Elastic net regression uses which penalties and combines
the benefits of both?
Answer: Both penalties of ridge (L2) and lasso (L1) regression.
Rationale: Elastic net combines L1 and L2 penalties, allowing for
both variable selection (like lasso) and handling of correlated
predictors (like ridge).
Q: Variable selection can be applied to regression problems
when the number of predicting variables is larger than the
number of observations.
Answer: True.
Rationale: Variable selection methods like lasso and stepwise can
handle p > n situations.
Q: The lasso regression performs well under multicollinearity.
Answer: False.
Rationale: Lasso tends to arbitrarily select one variable among a
group of highly correlated predictors and ignore the others.
, Q: The selected variables using best subset regression are the
best ones in explaining and predicting the response variables.
Answer: False.
Rationale: Best subset selection finds the model with the best fit
for a given subset size, but this does not guarantee the best
predictive performance on new data.
Q: The lasso regression requires a numerical algorithm to
minimize the penalized sum of least squares.
Answer: True.
Rationale: There is no closed-form solution for lasso regression
due to the L1 penalty; numerical optimization (e.g., coordinate
descent) is required.
Q: We cannot perform variable selection based on the
statistical significance of the regression coefficients.
Answer: True.
Rationale: Stepwise selection based on p-values is unreliable due
to multiple testing issues and can lead to overfitting.
