ISYE 6501 MIDTERM 2 TERMS UPDATED
ACTUAL QUESTIONS WITH CORRECT ANSWERS
Question 1.
Backward elimination
ANSWER
Variable selection process that starts with all variables and then iteratively removes the least-
immediately-relevant variables from the model.
Question 2.
Elastic net
ANSWER
Combination of lasso and ridge regression.
Question 3.
Forward selection
ANSWER
Variable selection process that starts with no variables and then iteratively adds the most-
immediately-relevant variables to the model.
Question 4.
Lasso/Lasso regression
ANSWER
Method for limiting the number of variables in a model by limiting the sum of all coefficients'
absolute values. Can be very helpful when number of data points is less than number of
factors.
,Question 5.
Overfitting
ANSWER
Building a model that describes random effects instead of or in significant addition to the real
effects; often caused by having too many factors or parameters compared to the number of
data points. ____________ models will have high prediction errors.
Question 6.
Regularization
ANSWER
Addition of term(s) to the model to reduce model complexity or overfitting. For example,
adding a penalty to the objective function in regression can help reduce overfitting (see ridge
regression).
Question 7.
Ridge regression
ANSWER
Method of regularization by limiting the sum of the squares of the coefficients. Will reduce the
magnitude of coefficients, not the number of variables chosen.
Question 8.
Simplicity (of a model)
ANSWER
Having fewer parameters; opposite of complexity of a model. Often helpful for avoiding
overfitting and increasing interpretability.
Question 9.
Stepwise regression
ANSWER
Variable selection process that can combine forward selection and backward regression.
,Question 10.
Variable selection
ANSWER
Process of selecting the best subset of predictors to explain variance in data; involves
eliminating unnecessary or redundant or less-important variables from
a potential set of predictors.
Question 11.
A/B testing
ANSWER
Test of two alternatives to see if either one leads to better outcomes.
Question 12.
Analysis of Variance/ANOVA
ANSWER
Statistical method for dividing the variation in observations among different sources.
Question 13.
Balanced design
ANSWER
Set of combinations of factor values across multiple factors, that has the same number of
runs for all combinations of levels of one or more factors.
Question 14.
Blocking
ANSWER
Factor introduced to an experimental design that interacts with the effect of the factors to be
studied. The effect of the factors is studied within the same level (block) of the blocking
factor.
, Question 15.
Control
ANSWER
(1) A variable whose value remains constant for all runs of an experiment, so changes in this
variable don't affect the experiment. (2) Design an experiment where some factors ("controls"
by definition (1)) are held constant to avoid them affecting the outcome.
Question 16.
Design of experiments
ANSWER
Choosing a set of tests to be made to find the effect of input variables on an outcome.
Question 17.
Exploitation
ANSWER
Using known information to get good outcomes.
Question 18.
Exploration
ANSWER
Finding new/better/more information to determine how to optimize output.
Question 19.
Factorial design
ANSWER
Tests of different combinations of factor values over multiple factors, to find each one's
effect, and interaction effects, on the outcome.
ACTUAL QUESTIONS WITH CORRECT ANSWERS
Question 1.
Backward elimination
ANSWER
Variable selection process that starts with all variables and then iteratively removes the least-
immediately-relevant variables from the model.
Question 2.
Elastic net
ANSWER
Combination of lasso and ridge regression.
Question 3.
Forward selection
ANSWER
Variable selection process that starts with no variables and then iteratively adds the most-
immediately-relevant variables to the model.
Question 4.
Lasso/Lasso regression
ANSWER
Method for limiting the number of variables in a model by limiting the sum of all coefficients'
absolute values. Can be very helpful when number of data points is less than number of
factors.
,Question 5.
Overfitting
ANSWER
Building a model that describes random effects instead of or in significant addition to the real
effects; often caused by having too many factors or parameters compared to the number of
data points. ____________ models will have high prediction errors.
Question 6.
Regularization
ANSWER
Addition of term(s) to the model to reduce model complexity or overfitting. For example,
adding a penalty to the objective function in regression can help reduce overfitting (see ridge
regression).
Question 7.
Ridge regression
ANSWER
Method of regularization by limiting the sum of the squares of the coefficients. Will reduce the
magnitude of coefficients, not the number of variables chosen.
Question 8.
Simplicity (of a model)
ANSWER
Having fewer parameters; opposite of complexity of a model. Often helpful for avoiding
overfitting and increasing interpretability.
Question 9.
Stepwise regression
ANSWER
Variable selection process that can combine forward selection and backward regression.
,Question 10.
Variable selection
ANSWER
Process of selecting the best subset of predictors to explain variance in data; involves
eliminating unnecessary or redundant or less-important variables from
a potential set of predictors.
Question 11.
A/B testing
ANSWER
Test of two alternatives to see if either one leads to better outcomes.
Question 12.
Analysis of Variance/ANOVA
ANSWER
Statistical method for dividing the variation in observations among different sources.
Question 13.
Balanced design
ANSWER
Set of combinations of factor values across multiple factors, that has the same number of
runs for all combinations of levels of one or more factors.
Question 14.
Blocking
ANSWER
Factor introduced to an experimental design that interacts with the effect of the factors to be
studied. The effect of the factors is studied within the same level (block) of the blocking
factor.
, Question 15.
Control
ANSWER
(1) A variable whose value remains constant for all runs of an experiment, so changes in this
variable don't affect the experiment. (2) Design an experiment where some factors ("controls"
by definition (1)) are held constant to avoid them affecting the outcome.
Question 16.
Design of experiments
ANSWER
Choosing a set of tests to be made to find the effect of input variables on an outcome.
Question 17.
Exploitation
ANSWER
Using known information to get good outcomes.
Question 18.
Exploration
ANSWER
Finding new/better/more information to determine how to optimize output.
Question 19.
Factorial design
ANSWER
Tests of different combinations of factor values over multiple factors, to find each one's
effect, and interaction effects, on the outcome.
