ISYE 6501 COMPREHENSIVE TEST PAPER
WITH SOLVED QUESTIONS
◉ Box-Cox Transformation
Answer: Logarithmic transformation technique used to eliminate
heteroskedasticity (unequal variance) across a data set to make it
fit a normal distribution. This will help a model avoid being biased
towards parts of the data with greater variance.
◉ PCA (principal component analysis)
Answer: Feature extraction technique that can reduce the
dimensionality of the feature space in a model, creating new
components ordered by importance, while also transforming the
data around new orthogonal axis in order to remove correlation.
◉ PCA Pros and Cons
Answer: Pros: Removes correlated features
Can improve algorithm performance by omitting correlated
variables
Reduces over fitting
Improves visualization
Cons: Harder to interpret effects of individual independent
variables
Requires data standardization
, ◉ CART (classification and regression trees)
Answer: Regression technique that creates different models for
different subset of data. This is useful when different factors may
have different ways they interact that makes them more effective
to model separately. For example when modeling likeliness of
response to advertisement, may be useful to divide population
into age group and marital status.
◉ Branching
Answer: The process of deciding how to split the data, and which
factors to include in a CART model. Can go either backwards,
removing leaves that don't decrease variance more than a certain
threshold, or forward by adding leaves until they either don't
improve the model enough or don't contain enough data.
◉ Random Forests
Answer: Method where a multitude of CARTs are produced, using
randomized subset of data and randomized factors. For regression
you then use the average predicted response across the trees, and
for classification you use the most common predicted response/
◉ Random Forest Pros/Cons
Answer: Pros: Tends to give better estimates overall because
while each tree might be overfitting in one place or another, they
don't necessarily over-fit the same way
Average over all trees corrects overfitting
Cons: Harder to explain the output of these models.
WITH SOLVED QUESTIONS
◉ Box-Cox Transformation
Answer: Logarithmic transformation technique used to eliminate
heteroskedasticity (unequal variance) across a data set to make it
fit a normal distribution. This will help a model avoid being biased
towards parts of the data with greater variance.
◉ PCA (principal component analysis)
Answer: Feature extraction technique that can reduce the
dimensionality of the feature space in a model, creating new
components ordered by importance, while also transforming the
data around new orthogonal axis in order to remove correlation.
◉ PCA Pros and Cons
Answer: Pros: Removes correlated features
Can improve algorithm performance by omitting correlated
variables
Reduces over fitting
Improves visualization
Cons: Harder to interpret effects of individual independent
variables
Requires data standardization
, ◉ CART (classification and regression trees)
Answer: Regression technique that creates different models for
different subset of data. This is useful when different factors may
have different ways they interact that makes them more effective
to model separately. For example when modeling likeliness of
response to advertisement, may be useful to divide population
into age group and marital status.
◉ Branching
Answer: The process of deciding how to split the data, and which
factors to include in a CART model. Can go either backwards,
removing leaves that don't decrease variance more than a certain
threshold, or forward by adding leaves until they either don't
improve the model enough or don't contain enough data.
◉ Random Forests
Answer: Method where a multitude of CARTs are produced, using
randomized subset of data and randomized factors. For regression
you then use the average predicted response across the trees, and
for classification you use the most common predicted response/
◉ Random Forest Pros/Cons
Answer: Pros: Tends to give better estimates overall because
while each tree might be overfitting in one place or another, they
don't necessarily over-fit the same way
Average over all trees corrects overfitting
Cons: Harder to explain the output of these models.
