ISYE 6501 Final Quiz | Questions and Answers | 2025
Update | 100% Correct.
1. When using leave-one-out cross-validation (LOOCV) for a linear regression model on a
dataset of size n, the CV estimate of prediction error is known to have high variance. This
is primarily because:
A. LOOCV uses n-1 training points, leading to overfitting
B. The training sets are highly similar, causing correlated predictions
C. The test set size is too small, leading to unstable estimates
D. LOOCV is equivalent to AIC, which has high variance
Answer: B
Rationale: In LOOCV, training sets differ by only one point, so the fitted models are highly
correlated. This correlation inflates the variance of the CV estimate, even though the bias is
typically low. Option A is incorrect because using n-1 points reduces overfitting, not causes it.
Option C is a common misconception; the test set size is one, but the issue is correlation, not test
set size. Option D is incorrect because LOOCV is not equivalent to AIC in general.
2. A dataset contains clusters of varying densities and non-spherical shapes. Applying
K-means clustering with Euclidean distance is likely to produce:
A. Reasonable clusters if the data is scaled appropriately
B. Spherical clusters that may merge distinct groups
C. Elliptical clusters aligned with principal components
D. Clusters that reflect the global density pattern
Answer: B
Rationale: K-means implicitly assumes spherical clusters of similar size due to its use of
Euclidean distance and minimization of within-cluster variance. It fails to capture complex
shapes or varying densities, often merging distinct groups into one. Scaling only affects variable
magnitude, not shape. Elliptical clusters (option C) would require a different algorithm (e.g.,
Gaussian Mixture Models). Option D is incorrect because K-means does not model density.
3. In principal component analysis on four standardized variables, the loadings of the first
principal component are all 0.5. This implies:
A. The first PC explains 25% of the total variance
B. All variables contribute equally to the first PC
C. The variables are uncorrelated
D. The data lies primarily along the diagonal direction
Page 1
,Answer: B
Rationale: Equal loadings (0.5 each) indicate that the first principal component is an equally
weighted linear combination of the variables. The variance explained depends on the
eigenvalue, which is not determined solely by loadings (option A). Uncorrelated variables would
yield loadings of 1 and 0s. Option D describes the direction but does not directly follow from
equal loadings alone.
4. For an ARIMA(1,1,1) model fitted to a time series, the residuals show significant
autocorrelation at lag 1. Which model adjustment is most appropriate?
A. Increase the differencing order
B. Increase the autoregressive order
C. Increase the moving average order
D. Apply a Box-Cox transformation
Answer: B
Rationale: Residual autocorrelation at lag 1 suggests that the autoregressive component is
insufficient. Increasing the AR order (e.g., to ARIMA(2,1,1)) can capture the remaining linear
dependence. Differencing (option A) would remove trend but not residual autocorrelation.
Increasing MA order (option C) might help for autocorrelation at longer lags, but lag 1 is
typically handled by AR. Transformation (option D) addresses variance issues, not
autocorrelation.
5. In a support vector machine with a Gaussian RBF kernel, a very high gamma value will
most likely lead to:
A. Underfitting
B. Overfitting
C. Linear decision boundary
D. Robustness to outliers
Answer: B
Rationale: Gamma controls the influence of a single training example; high gamma makes the
kernel narrow, causing the model to be overly sensitive to individual points, resulting in a highly
wiggly decision boundary that overfits. Low gamma leads to underfitting. A linear boundary
(option C) occurs with a linear kernel. High gamma reduces robustness (option D).
6. A deep neural network with ReLU activation functions is initialized with very small
random weights. The training loss stagnates at a high value from the start. This is most
likely due to:
A. Vanishing gradients
B. Dead ReLU units
C. Exploding gradients
D. Overfitting
Page 2
,Answer: B
Rationale: With very small weights, the linear transformations produce small outputs, and ReLU
units may output zero for many inputs, especially if biases are also small. Once a unit becomes
dead (always zero), its gradients vanish, preventing updates. Vanishing gradients (option A)
typically occur with sigmoid/tanh, not ReLU. Exploding gradients (option C) arise from large
weights. Overfitting (option D) would show low training loss.
7. If a linear programming primal is infeasible, then its dual must be:
A. Infeasible
B. Unbounded
C. Feasible
D. Either infeasible or unbounded
Answer: D
Rationale: By the duality theorem, if the primal is infeasible, the dual is either infeasible or
unbounded. It cannot be feasible with a finite optimum because that would imply primal
feasibility. Thus D is correct; A and B are only specific cases, but not always.
8. In discrete-event simulation, to obtain a valid estimate of the steady-state mean, the
method of batch means involves:
A. Dividing a single long run into batches and treating batch means as independent observations
B. Running multiple independent short replications
C. Using a moving average to smooth the data
D. Removing the first part of the simulation output without further analysis
Answer: A
Rationale: Batch means addresses autocorrelation by partitioning a single long run into
contiguous batches and computing means; if batches are large enough, the batch means are
approximately independent. Independent replications (option B) are also valid but not batch
means. Smoothing (option C) does not produce independent observations. Just removing initial
data (option D) is insufficient without assessing correlation.
9. For a binary classifier on an imbalanced dataset (1% positives), the accuracy is 99%,
precision is 0.8, and recall is 0.5. The Matthews correlation coefficient (MCC) is computed.
Based on these values, which statement is most accurate?
A. The model is better than random because accuracy is high
B. The model has a high F1 score but low MCC
C. The MCC value will be close to 0.8
D. The MCC value will be lower than the F1 score
Answer: D
Rationale: MCC accounts for all four confusion matrix categories and is more informative with
imbalance. Given precision=0.8 and recall=0.5, F1 0.615. MCC typically is lower than F1
when there is a significant imbalance and misclassifications. Option A is misleading because
Page 3
, accuracy is inflated. Option B is incorrect; F1 is moderate. Option C is incorrect; MCC is
usually lower than precision.
10. In a dataset with 1000 features and 500 samples, a ridge regression model with =0.1
yields training R²=0.99 and test R²=0.65. Which adjustment most likely improves test
performance?
A. Increase to reduce overfitting
B. Decrease to allow more flexibility
C. Add L1 regularization instead of L2
D. Remove regularization entirely
Answer: A
Rationale: High training R² with much lower test R² indicates overfitting. Ridge regression with
L2 penalty; increasing (regularization strength) shrinks coefficients more, reducing variance at
cost of bias, thus improving test performance. Decreasing or removing regularization would
worsen overfitting. Switching to L1 (Lasso) could help via feature selection but is not
guaranteed; increasing is a direct remedy.
11. A logistic regression model for binary classification yields coefficients: = -2, = 0.5, =
-0.3 for features x and x. For a new observation with x=1, x=2, what is the predicted
probability of the positive class?
A. 0.119
B. 0.269
C. 0.731
D. 0.881
Answer: A
Rationale: Logistic regression probability = 1/(1+exp(-(² €+² •x •+² ‚x ‚))). Compute linear
predictor: -2+0.5*1+(-0.3)*2 = -2+0.5-0.6 = -2.1. Then probability = 1/(1+exp(2.1))
1/(1+8.166) 0.109. Closest option is 0.119 (due to rounding). Option B results from sign error;
C and D from incorrect exponent handling.
12. A decision tree trained on a regression problem has leaves with mean squared errors.
Which split criterion is used to decide where to split?
A. Gini impurity
B. Entropy reduction
C. Variance reduction
D. Classification error
Answer: C
Rationale: For regression trees, the split criterion is typically variance reduction (or reduction in
mean squared error). Gini impurity and entropy are used for classification. Classification error
is also for classification but rarely used due to insensitivity. Variance reduction minimizes the
weighted sum of variances in child nodes.
Page 4
Update | 100% Correct.
1. When using leave-one-out cross-validation (LOOCV) for a linear regression model on a
dataset of size n, the CV estimate of prediction error is known to have high variance. This
is primarily because:
A. LOOCV uses n-1 training points, leading to overfitting
B. The training sets are highly similar, causing correlated predictions
C. The test set size is too small, leading to unstable estimates
D. LOOCV is equivalent to AIC, which has high variance
Answer: B
Rationale: In LOOCV, training sets differ by only one point, so the fitted models are highly
correlated. This correlation inflates the variance of the CV estimate, even though the bias is
typically low. Option A is incorrect because using n-1 points reduces overfitting, not causes it.
Option C is a common misconception; the test set size is one, but the issue is correlation, not test
set size. Option D is incorrect because LOOCV is not equivalent to AIC in general.
2. A dataset contains clusters of varying densities and non-spherical shapes. Applying
K-means clustering with Euclidean distance is likely to produce:
A. Reasonable clusters if the data is scaled appropriately
B. Spherical clusters that may merge distinct groups
C. Elliptical clusters aligned with principal components
D. Clusters that reflect the global density pattern
Answer: B
Rationale: K-means implicitly assumes spherical clusters of similar size due to its use of
Euclidean distance and minimization of within-cluster variance. It fails to capture complex
shapes or varying densities, often merging distinct groups into one. Scaling only affects variable
magnitude, not shape. Elliptical clusters (option C) would require a different algorithm (e.g.,
Gaussian Mixture Models). Option D is incorrect because K-means does not model density.
3. In principal component analysis on four standardized variables, the loadings of the first
principal component are all 0.5. This implies:
A. The first PC explains 25% of the total variance
B. All variables contribute equally to the first PC
C. The variables are uncorrelated
D. The data lies primarily along the diagonal direction
Page 1
,Answer: B
Rationale: Equal loadings (0.5 each) indicate that the first principal component is an equally
weighted linear combination of the variables. The variance explained depends on the
eigenvalue, which is not determined solely by loadings (option A). Uncorrelated variables would
yield loadings of 1 and 0s. Option D describes the direction but does not directly follow from
equal loadings alone.
4. For an ARIMA(1,1,1) model fitted to a time series, the residuals show significant
autocorrelation at lag 1. Which model adjustment is most appropriate?
A. Increase the differencing order
B. Increase the autoregressive order
C. Increase the moving average order
D. Apply a Box-Cox transformation
Answer: B
Rationale: Residual autocorrelation at lag 1 suggests that the autoregressive component is
insufficient. Increasing the AR order (e.g., to ARIMA(2,1,1)) can capture the remaining linear
dependence. Differencing (option A) would remove trend but not residual autocorrelation.
Increasing MA order (option C) might help for autocorrelation at longer lags, but lag 1 is
typically handled by AR. Transformation (option D) addresses variance issues, not
autocorrelation.
5. In a support vector machine with a Gaussian RBF kernel, a very high gamma value will
most likely lead to:
A. Underfitting
B. Overfitting
C. Linear decision boundary
D. Robustness to outliers
Answer: B
Rationale: Gamma controls the influence of a single training example; high gamma makes the
kernel narrow, causing the model to be overly sensitive to individual points, resulting in a highly
wiggly decision boundary that overfits. Low gamma leads to underfitting. A linear boundary
(option C) occurs with a linear kernel. High gamma reduces robustness (option D).
6. A deep neural network with ReLU activation functions is initialized with very small
random weights. The training loss stagnates at a high value from the start. This is most
likely due to:
A. Vanishing gradients
B. Dead ReLU units
C. Exploding gradients
D. Overfitting
Page 2
,Answer: B
Rationale: With very small weights, the linear transformations produce small outputs, and ReLU
units may output zero for many inputs, especially if biases are also small. Once a unit becomes
dead (always zero), its gradients vanish, preventing updates. Vanishing gradients (option A)
typically occur with sigmoid/tanh, not ReLU. Exploding gradients (option C) arise from large
weights. Overfitting (option D) would show low training loss.
7. If a linear programming primal is infeasible, then its dual must be:
A. Infeasible
B. Unbounded
C. Feasible
D. Either infeasible or unbounded
Answer: D
Rationale: By the duality theorem, if the primal is infeasible, the dual is either infeasible or
unbounded. It cannot be feasible with a finite optimum because that would imply primal
feasibility. Thus D is correct; A and B are only specific cases, but not always.
8. In discrete-event simulation, to obtain a valid estimate of the steady-state mean, the
method of batch means involves:
A. Dividing a single long run into batches and treating batch means as independent observations
B. Running multiple independent short replications
C. Using a moving average to smooth the data
D. Removing the first part of the simulation output without further analysis
Answer: A
Rationale: Batch means addresses autocorrelation by partitioning a single long run into
contiguous batches and computing means; if batches are large enough, the batch means are
approximately independent. Independent replications (option B) are also valid but not batch
means. Smoothing (option C) does not produce independent observations. Just removing initial
data (option D) is insufficient without assessing correlation.
9. For a binary classifier on an imbalanced dataset (1% positives), the accuracy is 99%,
precision is 0.8, and recall is 0.5. The Matthews correlation coefficient (MCC) is computed.
Based on these values, which statement is most accurate?
A. The model is better than random because accuracy is high
B. The model has a high F1 score but low MCC
C. The MCC value will be close to 0.8
D. The MCC value will be lower than the F1 score
Answer: D
Rationale: MCC accounts for all four confusion matrix categories and is more informative with
imbalance. Given precision=0.8 and recall=0.5, F1 0.615. MCC typically is lower than F1
when there is a significant imbalance and misclassifications. Option A is misleading because
Page 3
, accuracy is inflated. Option B is incorrect; F1 is moderate. Option C is incorrect; MCC is
usually lower than precision.
10. In a dataset with 1000 features and 500 samples, a ridge regression model with =0.1
yields training R²=0.99 and test R²=0.65. Which adjustment most likely improves test
performance?
A. Increase to reduce overfitting
B. Decrease to allow more flexibility
C. Add L1 regularization instead of L2
D. Remove regularization entirely
Answer: A
Rationale: High training R² with much lower test R² indicates overfitting. Ridge regression with
L2 penalty; increasing (regularization strength) shrinks coefficients more, reducing variance at
cost of bias, thus improving test performance. Decreasing or removing regularization would
worsen overfitting. Switching to L1 (Lasso) could help via feature selection but is not
guaranteed; increasing is a direct remedy.
11. A logistic regression model for binary classification yields coefficients: = -2, = 0.5, =
-0.3 for features x and x. For a new observation with x=1, x=2, what is the predicted
probability of the positive class?
A. 0.119
B. 0.269
C. 0.731
D. 0.881
Answer: A
Rationale: Logistic regression probability = 1/(1+exp(-(² €+² •x •+² ‚x ‚))). Compute linear
predictor: -2+0.5*1+(-0.3)*2 = -2+0.5-0.6 = -2.1. Then probability = 1/(1+exp(2.1))
1/(1+8.166) 0.109. Closest option is 0.119 (due to rounding). Option B results from sign error;
C and D from incorrect exponent handling.
12. A decision tree trained on a regression problem has leaves with mean squared errors.
Which split criterion is used to decide where to split?
A. Gini impurity
B. Entropy reduction
C. Variance reduction
D. Classification error
Answer: C
Rationale: For regression trees, the split criterion is typically variance reduction (or reduction in
mean squared error). Gini impurity and entropy are used for classification. Classification error
is also for classification but rarely used due to insensitivity. Variance reduction minimizes the
weighted sum of variances in child nodes.
Page 4
