ISYE 6501 Final Exam Review | Advanced Analytics Modeling
Practice Examination
Comprehensive Mastery Assessment - Third Edition
(Advanced/Complicated)
Exam Title: ISYE 6501 Advanced Final Examination: Comprehensive Analytics Modeling Practice
Assessment Featuring Complex Supervised and Unsupervised Learning Scenarios, Multivariate Time
Series Analysis, Stochastic Optimization, Monte Carlo Simulation, Advanced Experimental Design,
Model Interpretability, and Cross-Validation Techniques for Expert-Level Analytics Practitioners
SECTION 1: ADVANCED REGRESSION AND REGULARIZATION
TECHNIQUES (Questions 1-25)
Question 1:
In the context of regularization, what is the primary difference between L1 (LASSO) and L2 (Ridge)
regularization in terms of their geometric interpretation?
A) L1 creates a diamond-shaped constraint region leading to sparse solutions; L2 creates a circular
constraint region leading to dense solutions
B) L1 creates a circular constraint region leading to dense solutions; L2 creates a diamond-shaped
constraint region leading to sparse solutions
C) Both create identical constraint regions but differ in computational complexity
D) L1 is applicable only to linear regression while L2 works for all models
Correct Answer: A
Rationale: The geometric interpretation is fundamental to understanding regularization. L1
regularization (LASSO) has a diamond-shaped constraint region (|β₁|+|β₂| ≤ t) with corners at the axes,
making it more likely that coefficient estimates will be exactly zero at the intersection with the elliptical
SSE contours. L2 regularization (Ridge) has a circular constraint region (β₁²+β₂² ≤ t) with no corners, so
coefficients are shrunk but rarely exactly zero. This explains why LASSO performs variable selection
while Ridge does not.
Question 2:
When implementing elastic net regularization with parameters α and λ, what happens when α
approaches 1 and λ is large?
,A) The model approaches Ridge regression with strong regularization
B) The model approaches LASSO with strong regularization
C) The model approaches OLS regression
D) The model removes all predictors
Correct Answer: B
Rationale: Elastic net combines L1 and L2 penalties: (1-α)||β||₂² + α||β||₁. When α=1, it becomes pure
LASSO (L1 penalty). With large λ, the penalty is strong, shrinking many coefficients toward zero. This
combination (α→1, λ large) produces a sparse model (LASSO behavior) with aggressive regularization,
potentially selecting only the most important features.
Question 3:
In a high-dimensional regression problem (p >> n), why might Ridge regression be preferred over
LASSO despite LASSO's feature selection capability?
A) Ridge regression is computationally faster for high-dimensional problems
B) LASSO tends to select at most n variables when p >> n, while Ridge provides more stable
coefficient estimates
C) Ridge regression always produces sparser solutions than LASSO
D) LASSO cannot be applied when p >> n
Correct Answer: B
Rationale: When p >> n, LASSO's selection property becomes a limitation—it can select at most n
variables. Since n is small, many relevant predictors may be excluded. Ridge regression, while not
performing feature selection, shrinks coefficients and provides more stable estimates in the p >> n
scenario. Elastic net can sometimes be a better choice as it combines both approaches and can select
more than n variables when there are groups of correlated predictors.
Question 4:
Consider a logistic regression model with highly correlated predictors (multicollinearity). If you apply
LASSO regularization, what is likely to happen to the coefficient estimates compared to Ridge
regression?
A) LASSO will select one variable from each correlated group and set others to zero; Ridge will shrink
all coefficients but keep them non-zero
B) LASSO will keep all correlated variables with similar coefficients; Ridge will select one variable and
set others to zero
C) Both will produce identical coefficient estimates
D) LASSO will produce larger coefficients than Ridge for all variables
Correct Answer: A
,Rationale: This is a crucial distinction in regularization for correlated predictors. LASSO tends to select
one variable from a group of highly correlated predictors and set the others to zero, because L1
regularization doesn't encourage grouping behavior. Ridge regression, with L2 regularization, tends to
shrink coefficients of correlated predictors toward each other, keeping all in the model but with
reduced coefficients. This is why elastic net was developed—to combine LASSO's selection with
Ridge's grouping capability.
Question 5:
What is the effect of standardizing predictors before applying LASSO regression?
A) Standardization is unnecessary because LASSO is scale-invariant
B) Standardization ensures that the L1 penalty is applied fairly across predictors with different scales
C) Standardization should never be performed before LASSO
D) Standardization only affects the intercept term, not the coefficients
Correct Answer: B
Rationale: Standardization is crucial for LASSO (and Ridge) because these methods penalize the
magnitude of coefficients. Without standardization, variables with larger scales would have
proportionally smaller coefficients and thus be penalized differently, regardless of their actual
predictive importance. Standardization (z-score: subtract mean, divide by standard deviation) places all
predictors on the same scale, ensuring the regularization penalty is applied fairly. The intercept is
typically not penalized.
Question 6:
In the context of generalized linear models (GLMs), what is the relationship between the canonical link
function and the natural parameter of the exponential family?
A) The canonical link equates the linear predictor to the natural parameter
B) The canonical link equates the linear predictor to the mean of the response variable
C) The canonical link is always the identity function
D) The canonical link is unrelated to the exponential family
Correct Answer: A
Rationale: In GLMs, the canonical link function g(μ) is the function that equates the linear predictor η
= Xβ to the natural parameter θ of the exponential family distribution. For example, in logistic
regression, the canonical link is the logit function because the natural parameter for the Bernoulli
distribution is log(p/(1-p)). This creates a direct mathematical relationship that ensures the sufficient
statistics are the features themselves and simplifies maximum likelihood estimation.
, Question 7:
What is the variance function in a GLM, and how does it relate to the mean-variance relationship?
A) The variance function specifies how the variance of the response depends on its mean; for Poisson,
Var(Y)=μ
B) The variance function is constant for all GLMs
C) The variance function is only relevant for continuous responses
D) The variance function determines the link function choice
Correct Answer: A
Rationale: The variance function V(μ) describes how the variance depends on the mean for a GLM
family: Var(Y) = φV(μ), where φ is the dispersion parameter. This is crucial because different
distributions have different mean-variance relationships: Poisson has Var(Y)=μ (variance equals mean),
binomial has Var(Y)=μ(1-μ)/n, and normal has constant variance φ. This mean-variance relationship
determines the weights in iteratively reweighted least squares (IRLS) estimation and affects standard
error calculations.
Question 8:
When using quasi-likelihood estimation in GLMs, what assumption is relaxed compared to full
maximum likelihood?
A) The distribution of the response variable need not be fully specified; only the mean-variance
relationship is needed
B) The predictors must be normally distributed
C) The response must be continuous
D) The link function must be canonical
Correct Answer: A
Rationale: Quasi-likelihood estimation relaxes the assumption that the response variable follows a
specific distribution from the exponential family. Instead, it only requires specifying the relationship
between the mean and variance (variance function). This is useful when the response distribution is
unknown or when overdispersion is present. The estimating equations use quasi-likelihood functions
that behave like log-likelihoods but may not correspond to a true probability distribution. This
provides robust inference even when the distributional assumptions are violated.
Question 9:
In the context of generalized additive models (GAMs), what is the advantage of using smoothing
splines over polynomial regression?
A) Smoothing splines are computationally more efficient
B) Smoothing splines provide flexible non-linear fitting without specifying a global polynomial degree
Practice Examination
Comprehensive Mastery Assessment - Third Edition
(Advanced/Complicated)
Exam Title: ISYE 6501 Advanced Final Examination: Comprehensive Analytics Modeling Practice
Assessment Featuring Complex Supervised and Unsupervised Learning Scenarios, Multivariate Time
Series Analysis, Stochastic Optimization, Monte Carlo Simulation, Advanced Experimental Design,
Model Interpretability, and Cross-Validation Techniques for Expert-Level Analytics Practitioners
SECTION 1: ADVANCED REGRESSION AND REGULARIZATION
TECHNIQUES (Questions 1-25)
Question 1:
In the context of regularization, what is the primary difference between L1 (LASSO) and L2 (Ridge)
regularization in terms of their geometric interpretation?
A) L1 creates a diamond-shaped constraint region leading to sparse solutions; L2 creates a circular
constraint region leading to dense solutions
B) L1 creates a circular constraint region leading to dense solutions; L2 creates a diamond-shaped
constraint region leading to sparse solutions
C) Both create identical constraint regions but differ in computational complexity
D) L1 is applicable only to linear regression while L2 works for all models
Correct Answer: A
Rationale: The geometric interpretation is fundamental to understanding regularization. L1
regularization (LASSO) has a diamond-shaped constraint region (|β₁|+|β₂| ≤ t) with corners at the axes,
making it more likely that coefficient estimates will be exactly zero at the intersection with the elliptical
SSE contours. L2 regularization (Ridge) has a circular constraint region (β₁²+β₂² ≤ t) with no corners, so
coefficients are shrunk but rarely exactly zero. This explains why LASSO performs variable selection
while Ridge does not.
Question 2:
When implementing elastic net regularization with parameters α and λ, what happens when α
approaches 1 and λ is large?
,A) The model approaches Ridge regression with strong regularization
B) The model approaches LASSO with strong regularization
C) The model approaches OLS regression
D) The model removes all predictors
Correct Answer: B
Rationale: Elastic net combines L1 and L2 penalties: (1-α)||β||₂² + α||β||₁. When α=1, it becomes pure
LASSO (L1 penalty). With large λ, the penalty is strong, shrinking many coefficients toward zero. This
combination (α→1, λ large) produces a sparse model (LASSO behavior) with aggressive regularization,
potentially selecting only the most important features.
Question 3:
In a high-dimensional regression problem (p >> n), why might Ridge regression be preferred over
LASSO despite LASSO's feature selection capability?
A) Ridge regression is computationally faster for high-dimensional problems
B) LASSO tends to select at most n variables when p >> n, while Ridge provides more stable
coefficient estimates
C) Ridge regression always produces sparser solutions than LASSO
D) LASSO cannot be applied when p >> n
Correct Answer: B
Rationale: When p >> n, LASSO's selection property becomes a limitation—it can select at most n
variables. Since n is small, many relevant predictors may be excluded. Ridge regression, while not
performing feature selection, shrinks coefficients and provides more stable estimates in the p >> n
scenario. Elastic net can sometimes be a better choice as it combines both approaches and can select
more than n variables when there are groups of correlated predictors.
Question 4:
Consider a logistic regression model with highly correlated predictors (multicollinearity). If you apply
LASSO regularization, what is likely to happen to the coefficient estimates compared to Ridge
regression?
A) LASSO will select one variable from each correlated group and set others to zero; Ridge will shrink
all coefficients but keep them non-zero
B) LASSO will keep all correlated variables with similar coefficients; Ridge will select one variable and
set others to zero
C) Both will produce identical coefficient estimates
D) LASSO will produce larger coefficients than Ridge for all variables
Correct Answer: A
,Rationale: This is a crucial distinction in regularization for correlated predictors. LASSO tends to select
one variable from a group of highly correlated predictors and set the others to zero, because L1
regularization doesn't encourage grouping behavior. Ridge regression, with L2 regularization, tends to
shrink coefficients of correlated predictors toward each other, keeping all in the model but with
reduced coefficients. This is why elastic net was developed—to combine LASSO's selection with
Ridge's grouping capability.
Question 5:
What is the effect of standardizing predictors before applying LASSO regression?
A) Standardization is unnecessary because LASSO is scale-invariant
B) Standardization ensures that the L1 penalty is applied fairly across predictors with different scales
C) Standardization should never be performed before LASSO
D) Standardization only affects the intercept term, not the coefficients
Correct Answer: B
Rationale: Standardization is crucial for LASSO (and Ridge) because these methods penalize the
magnitude of coefficients. Without standardization, variables with larger scales would have
proportionally smaller coefficients and thus be penalized differently, regardless of their actual
predictive importance. Standardization (z-score: subtract mean, divide by standard deviation) places all
predictors on the same scale, ensuring the regularization penalty is applied fairly. The intercept is
typically not penalized.
Question 6:
In the context of generalized linear models (GLMs), what is the relationship between the canonical link
function and the natural parameter of the exponential family?
A) The canonical link equates the linear predictor to the natural parameter
B) The canonical link equates the linear predictor to the mean of the response variable
C) The canonical link is always the identity function
D) The canonical link is unrelated to the exponential family
Correct Answer: A
Rationale: In GLMs, the canonical link function g(μ) is the function that equates the linear predictor η
= Xβ to the natural parameter θ of the exponential family distribution. For example, in logistic
regression, the canonical link is the logit function because the natural parameter for the Bernoulli
distribution is log(p/(1-p)). This creates a direct mathematical relationship that ensures the sufficient
statistics are the features themselves and simplifies maximum likelihood estimation.
, Question 7:
What is the variance function in a GLM, and how does it relate to the mean-variance relationship?
A) The variance function specifies how the variance of the response depends on its mean; for Poisson,
Var(Y)=μ
B) The variance function is constant for all GLMs
C) The variance function is only relevant for continuous responses
D) The variance function determines the link function choice
Correct Answer: A
Rationale: The variance function V(μ) describes how the variance depends on the mean for a GLM
family: Var(Y) = φV(μ), where φ is the dispersion parameter. This is crucial because different
distributions have different mean-variance relationships: Poisson has Var(Y)=μ (variance equals mean),
binomial has Var(Y)=μ(1-μ)/n, and normal has constant variance φ. This mean-variance relationship
determines the weights in iteratively reweighted least squares (IRLS) estimation and affects standard
error calculations.
Question 8:
When using quasi-likelihood estimation in GLMs, what assumption is relaxed compared to full
maximum likelihood?
A) The distribution of the response variable need not be fully specified; only the mean-variance
relationship is needed
B) The predictors must be normally distributed
C) The response must be continuous
D) The link function must be canonical
Correct Answer: A
Rationale: Quasi-likelihood estimation relaxes the assumption that the response variable follows a
specific distribution from the exponential family. Instead, it only requires specifying the relationship
between the mean and variance (variance function). This is useful when the response distribution is
unknown or when overdispersion is present. The estimating equations use quasi-likelihood functions
that behave like log-likelihoods but may not correspond to a true probability distribution. This
provides robust inference even when the distributional assumptions are violated.
Question 9:
In the context of generalized additive models (GAMs), what is the advantage of using smoothing
splines over polynomial regression?
A) Smoothing splines are computationally more efficient
B) Smoothing splines provide flexible non-linear fitting without specifying a global polynomial degree
