ISYE 6501: Introduction to Analytics
Modeling - Comprehensive Final
Exam Practice (Part 2)[ With Verified
Complete Answers]
Section 1: Foundational Concepts
Q1: What are the three main types of analytics questions?
Answer: Descriptive (what happened?), Predictive (what will happen?),
and Prescriptive (what action should be taken?). Descriptive analytics
summarizes historical data, predictive analytics forecasts future
outcomes, and prescriptive analytics recommends optimal decisions .
Q2: What is a model in the context of analytics?
Answer: A mathematical representation of a real-life situation. Models
simplify reality to enable analysis, prediction, and decision-making .
Q3: What is the difference between structured and unstructured
data?
Answer: Structured data can be stored in tables with defined rows and
columns (e.g., spreadsheets, databases), while unstructured data lacks a
predefined format (e.g., text, images, video, audio) .
Q4: What is the difference between quantitative and categorical
data?
Answer: Quantitative data are numbers with meaningful values (e.g.,
age, income, temperature), while categorical data classify observations
without quantitative meaning (e.g., colors, product types) or where
numbers are labels without inherent meaning (e.g., postal codes) .
Q5: What is binary data?
,Answer: Data that can take only two possible values, such as true/false,
0/1, or on/off. It is a special case of categorical data with exactly two
categories .
Q6: What is A/B testing and when is it appropriate?
Answer: A/B testing is a method for choosing between two alternatives
by randomly assigning subjects to one of two groups and comparing
outcomes. It is appropriate when: (1) data can be collected quickly
enough to use the answer, (2) the data comes from a representative
sample, and (3) enough data can be collected to achieve statistical
significance .
Q7: What does the p-value for a coefficient in linear regression
indicate?
Answer: The probability of observing the coefficient value (or a more
extreme value) if the true coefficient were actually zero. A low p-value
(typically < 0.05) suggests the predictor is statistically significant .
Q8: What is the bias-variance trade-off?
Answer: Total error = Bias² + Variance + Irreducible Error. As model
complexity increases, bias decreases but variance increases. Simple
models have high bias (underfitting), while complex models have high
variance (overfitting). The goal is to find the sweet spot that minimizes
total error .
Q9: What is overfitting and how can it be detected?
Answer: Overfitting occurs when a model fits training data too closely,
capturing noise rather than true patterns. It is detected when a model
has low training error but high test error, indicating the model has
memorized the training data but fails to generalize to new data .
Q10: What is the curse of dimensionality?
Answer: The phenomenon where data becomes sparse in high-
dimensional spaces, making distance metrics less meaningful and
requiring exponentially more data to maintain statistical significance.
,This affects algorithms that rely on distance measures like k-NN and
clustering .
Q11: What is regularization and what is its purpose?
Answer: Regularization is a technique that adds a penalty term to the
loss function to prevent overfitting by discouraging large coefficients. It
helps models generalize better by trading some training accuracy for
improved test performance .
Q12: What is cross-validation and why is it used?
Answer: Cross-validation is a resampling technique used to evaluate
model performance on unseen data. It involves splitting data into folds,
training on some folds, and validating on the held-out fold. k-fold cross-
validation (typically k=5 or 10) provides a more robust estimate of model
performance than a single train-test split .
Q13: What is the difference between supervised and unsupervised
learning?
Answer: Supervised learning uses labeled data with known outcomes
(responses) to train models for prediction or classification. Unsupervised
learning finds patterns in unlabeled data without predefined outcomes,
such as clustering or dimensionality reduction .
Q14: What is AIC (Akaike Information Criterion)?
Answer: A model selection criterion that trades off between model fit
and model complexity. When comparing models, the model with lower
AIC is preferred. AIC generally penalizes complexity less than BIC .
Q15: What is the difference between causation and correlation?
Answer: Correlation indicates a statistical association between variables,
while causation means one variable directly causes changes in another.
Correlation does not imply causation, as confounding variables may
explain the relationship .
, Section 2: Regression Models
Q16: What is the difference between linear and logistic regression?
Answer: Linear regression predicts continuous numeric outcomes using
a linear combination of predictors. Logistic regression predicts binary
outcomes (probabilities) using the logit link function (log-odds) and
outputs probabilities between 0 and 1 .
Q17: What does the R² value measure in linear regression?
Answer: R² measures the proportion of variance in the response variable
explained by the predictors. It ranges from 0 to 1, with higher values
indicating better fit. Adjusted R² penalizes for adding unnecessary
predictors .
Q18: What is the difference between Ridge and LASSO regression?
Answer: Ridge regression uses L2 penalty (sum of squared coefficients)
which shrinks coefficients but does not set them exactly to zero. LASSO
uses L1 penalty (sum of absolute coefficients) which can shrink
coefficients exactly to zero, performing automatic variable selection and
producing sparse solutions .
Q19: When is Ridge regression preferred over LASSO?
Answer: Ridge regression is preferred when there is multicollinearity
among predictors, as it shrinks correlated coefficients together. LASSO
tends to arbitrarily select one variable from a correlated group, while
Ridge retains all variables with reduced coefficients .
Q20: What is the link function in logistic regression?
Answer: The logit (log-odds) function transforms probabilities from [0,1]
to the full real line, allowing linear combination of predictors to predict
log-odds: logit(p) = β₀ + β₁X₁ + ... + βₚXₚ .
Q21: What does the coefficient represent in logistic regression?
Modeling - Comprehensive Final
Exam Practice (Part 2)[ With Verified
Complete Answers]
Section 1: Foundational Concepts
Q1: What are the three main types of analytics questions?
Answer: Descriptive (what happened?), Predictive (what will happen?),
and Prescriptive (what action should be taken?). Descriptive analytics
summarizes historical data, predictive analytics forecasts future
outcomes, and prescriptive analytics recommends optimal decisions .
Q2: What is a model in the context of analytics?
Answer: A mathematical representation of a real-life situation. Models
simplify reality to enable analysis, prediction, and decision-making .
Q3: What is the difference between structured and unstructured
data?
Answer: Structured data can be stored in tables with defined rows and
columns (e.g., spreadsheets, databases), while unstructured data lacks a
predefined format (e.g., text, images, video, audio) .
Q4: What is the difference between quantitative and categorical
data?
Answer: Quantitative data are numbers with meaningful values (e.g.,
age, income, temperature), while categorical data classify observations
without quantitative meaning (e.g., colors, product types) or where
numbers are labels without inherent meaning (e.g., postal codes) .
Q5: What is binary data?
,Answer: Data that can take only two possible values, such as true/false,
0/1, or on/off. It is a special case of categorical data with exactly two
categories .
Q6: What is A/B testing and when is it appropriate?
Answer: A/B testing is a method for choosing between two alternatives
by randomly assigning subjects to one of two groups and comparing
outcomes. It is appropriate when: (1) data can be collected quickly
enough to use the answer, (2) the data comes from a representative
sample, and (3) enough data can be collected to achieve statistical
significance .
Q7: What does the p-value for a coefficient in linear regression
indicate?
Answer: The probability of observing the coefficient value (or a more
extreme value) if the true coefficient were actually zero. A low p-value
(typically < 0.05) suggests the predictor is statistically significant .
Q8: What is the bias-variance trade-off?
Answer: Total error = Bias² + Variance + Irreducible Error. As model
complexity increases, bias decreases but variance increases. Simple
models have high bias (underfitting), while complex models have high
variance (overfitting). The goal is to find the sweet spot that minimizes
total error .
Q9: What is overfitting and how can it be detected?
Answer: Overfitting occurs when a model fits training data too closely,
capturing noise rather than true patterns. It is detected when a model
has low training error but high test error, indicating the model has
memorized the training data but fails to generalize to new data .
Q10: What is the curse of dimensionality?
Answer: The phenomenon where data becomes sparse in high-
dimensional spaces, making distance metrics less meaningful and
requiring exponentially more data to maintain statistical significance.
,This affects algorithms that rely on distance measures like k-NN and
clustering .
Q11: What is regularization and what is its purpose?
Answer: Regularization is a technique that adds a penalty term to the
loss function to prevent overfitting by discouraging large coefficients. It
helps models generalize better by trading some training accuracy for
improved test performance .
Q12: What is cross-validation and why is it used?
Answer: Cross-validation is a resampling technique used to evaluate
model performance on unseen data. It involves splitting data into folds,
training on some folds, and validating on the held-out fold. k-fold cross-
validation (typically k=5 or 10) provides a more robust estimate of model
performance than a single train-test split .
Q13: What is the difference between supervised and unsupervised
learning?
Answer: Supervised learning uses labeled data with known outcomes
(responses) to train models for prediction or classification. Unsupervised
learning finds patterns in unlabeled data without predefined outcomes,
such as clustering or dimensionality reduction .
Q14: What is AIC (Akaike Information Criterion)?
Answer: A model selection criterion that trades off between model fit
and model complexity. When comparing models, the model with lower
AIC is preferred. AIC generally penalizes complexity less than BIC .
Q15: What is the difference between causation and correlation?
Answer: Correlation indicates a statistical association between variables,
while causation means one variable directly causes changes in another.
Correlation does not imply causation, as confounding variables may
explain the relationship .
, Section 2: Regression Models
Q16: What is the difference between linear and logistic regression?
Answer: Linear regression predicts continuous numeric outcomes using
a linear combination of predictors. Logistic regression predicts binary
outcomes (probabilities) using the logit link function (log-odds) and
outputs probabilities between 0 and 1 .
Q17: What does the R² value measure in linear regression?
Answer: R² measures the proportion of variance in the response variable
explained by the predictors. It ranges from 0 to 1, with higher values
indicating better fit. Adjusted R² penalizes for adding unnecessary
predictors .
Q18: What is the difference between Ridge and LASSO regression?
Answer: Ridge regression uses L2 penalty (sum of squared coefficients)
which shrinks coefficients but does not set them exactly to zero. LASSO
uses L1 penalty (sum of absolute coefficients) which can shrink
coefficients exactly to zero, performing automatic variable selection and
producing sparse solutions .
Q19: When is Ridge regression preferred over LASSO?
Answer: Ridge regression is preferred when there is multicollinearity
among predictors, as it shrinks correlated coefficients together. LASSO
tends to arbitrarily select one variable from a correlated group, while
Ridge retains all variables with reduced coefficients .
Q20: What is the link function in logistic regression?
Answer: The logit (log-odds) function transforms probabilities from [0,1]
to the full real line, allowing linear combination of predictors to predict
log-odds: logit(p) = β₀ + β₁X₁ + ... + βₚXₚ .
Q21: What does the coefficient represent in logistic regression?
