10/15/24, 9:03 Example Final Questions Answ
AM ers
The following process was followed to predict sales of a product each month for the next three years:
1. Split past sales data randomly into three sets: training, validation, and test
2. Build 20 different models using the training data
3. Evaluate all 20 models on the validation data
4. Select the model that performed best on the validation data
5. Evaluate the selected model on the test data
6. Use the selected model to predict monthly sales for the next three years based on real time
data and observe its true performance.
Which of the following three statements is correct?
- It is unclear how the selected model’s expected performance on test data compares to its
observed performances on real time data because the training data and the test data
were taken from the same population, but the real time data might be different.
- The selected model’s expected performance on test data must be worse than its observed
performance on real time data because the training data and the test data were taken from
the same population, but the real time data might be different
- The selected model’s expected performance on the test data must be better than its
observed performance on real time data, because the training data and test data were taken
from the same population, but the real time data might be different
Which of the following three statements is correct?
- Every model’s expected performance on training data will be the same as its expected
performance on the validation data, because both the training data and the validation data
are taken from the same population.
- Every model’s expected performance on training data will be worse than its expected
performance on the validation data, because the training data and validation data are
different
- Every model’s expected performance on training data will be better than its expected
performance on the validation data, because model fits partly to random patterns in the
training data.
Which of the following three statements is correct?
- The selected model’s expected performance on test data will be better than its expected
performance on the validation data, because there is a selection bias; the selected model is
more likely to have worse than average performance on random patterns in the validation
data.
- The selected model’s expected performance on test data will be the same as its expected
performance on the validation data, because the validation data and the test data are the
same.
- The selected model’s expected performance on test data will be worse than its expected
performance on the validation data, because there is a selection bias; the selected model is
more likely to have better-than-average performance on random patterns in the validation
data.
about:bl 1/
,10/15/24, 9:03 Example Final Questions Answ
AM ers
A positive correlation has been observed between health and wealth among older Americans (healthier
people are wealthier on average, and wealthier people are healthier on average).
Based on that observed correlation, select all of the following statements about the direction of causality
between health and wealth that are true.
a. Wealth causes health: a healthy lifestyle is more expensive, so wealthier people are more able
to use their money to make healthier choices.
b. Health causes wealth: medical care for serious injuryttillness is expensive and worse
health restricts earning potential, so poor health drains people’s wealth.
c. Both health and wealth are positively correlated with another factor, which causes both
d. Can’t tell without more analysis
A positive correlation has been observed between number of police and amount of crime reported
(where there are more police per capita more crime is reported, and where more crime is reported
there are more police per capita)
Based on that observed correlation, select all of the following statements about the direction of causality
between police and crime reports that are true.
a. Police cause crime reports: Where more police are working, citizens report more crime to them.
b. Crime reports cause police: where there is more crime reported, more police are hired to stop it.
c. Both more police and more crimes report are positively correlated with another factor,
which causes both.
d. Can’t tell without more analysis.
For each of the four situations below, specify which would be better: including a “data missing”
binary variable or imputing missing data.
a. 2% of the data points have missing values, and you can build a good predictive model for
the missing data. Impute missing data
b. 2% of the data points have missing values, and you cannot build a good predictive model for the
missing data. “Data Missing” binary variable
c. 50% of the data points have missing values for this variable, and you believe that points
with missing data have a different distribution of values from points where data is present.
“Data Missing” binary variable
d. 50% of the data points have missing values for this variable, and you cannot build a
good predictive model for the missing data. “Data Missing” binary variable
A trucking company that has focused on 50 major cities nationally would like to make its network
more efficient by having better predictions of day-to-day demand from each city to each other city,
and then reallocating its resources (drivers, vehicles, etc.) to better-meet that demand.
about:bl 2/
, 10/15/24, 9:03 Example Final Questions Answ
AM ers
about:bl 3/
AM ers
The following process was followed to predict sales of a product each month for the next three years:
1. Split past sales data randomly into three sets: training, validation, and test
2. Build 20 different models using the training data
3. Evaluate all 20 models on the validation data
4. Select the model that performed best on the validation data
5. Evaluate the selected model on the test data
6. Use the selected model to predict monthly sales for the next three years based on real time
data and observe its true performance.
Which of the following three statements is correct?
- It is unclear how the selected model’s expected performance on test data compares to its
observed performances on real time data because the training data and the test data
were taken from the same population, but the real time data might be different.
- The selected model’s expected performance on test data must be worse than its observed
performance on real time data because the training data and the test data were taken from
the same population, but the real time data might be different
- The selected model’s expected performance on the test data must be better than its
observed performance on real time data, because the training data and test data were taken
from the same population, but the real time data might be different
Which of the following three statements is correct?
- Every model’s expected performance on training data will be the same as its expected
performance on the validation data, because both the training data and the validation data
are taken from the same population.
- Every model’s expected performance on training data will be worse than its expected
performance on the validation data, because the training data and validation data are
different
- Every model’s expected performance on training data will be better than its expected
performance on the validation data, because model fits partly to random patterns in the
training data.
Which of the following three statements is correct?
- The selected model’s expected performance on test data will be better than its expected
performance on the validation data, because there is a selection bias; the selected model is
more likely to have worse than average performance on random patterns in the validation
data.
- The selected model’s expected performance on test data will be the same as its expected
performance on the validation data, because the validation data and the test data are the
same.
- The selected model’s expected performance on test data will be worse than its expected
performance on the validation data, because there is a selection bias; the selected model is
more likely to have better-than-average performance on random patterns in the validation
data.
about:bl 1/
,10/15/24, 9:03 Example Final Questions Answ
AM ers
A positive correlation has been observed between health and wealth among older Americans (healthier
people are wealthier on average, and wealthier people are healthier on average).
Based on that observed correlation, select all of the following statements about the direction of causality
between health and wealth that are true.
a. Wealth causes health: a healthy lifestyle is more expensive, so wealthier people are more able
to use their money to make healthier choices.
b. Health causes wealth: medical care for serious injuryttillness is expensive and worse
health restricts earning potential, so poor health drains people’s wealth.
c. Both health and wealth are positively correlated with another factor, which causes both
d. Can’t tell without more analysis
A positive correlation has been observed between number of police and amount of crime reported
(where there are more police per capita more crime is reported, and where more crime is reported
there are more police per capita)
Based on that observed correlation, select all of the following statements about the direction of causality
between police and crime reports that are true.
a. Police cause crime reports: Where more police are working, citizens report more crime to them.
b. Crime reports cause police: where there is more crime reported, more police are hired to stop it.
c. Both more police and more crimes report are positively correlated with another factor,
which causes both.
d. Can’t tell without more analysis.
For each of the four situations below, specify which would be better: including a “data missing”
binary variable or imputing missing data.
a. 2% of the data points have missing values, and you can build a good predictive model for
the missing data. Impute missing data
b. 2% of the data points have missing values, and you cannot build a good predictive model for the
missing data. “Data Missing” binary variable
c. 50% of the data points have missing values for this variable, and you believe that points
with missing data have a different distribution of values from points where data is present.
“Data Missing” binary variable
d. 50% of the data points have missing values for this variable, and you cannot build a
good predictive model for the missing data. “Data Missing” binary variable
A trucking company that has focused on 50 major cities nationally would like to make its network
more efficient by having better predictions of day-to-day demand from each city to each other city,
and then reallocating its resources (drivers, vehicles, etc.) to better-meet that demand.
about:bl 2/
, 10/15/24, 9:03 Example Final Questions Answ
AM ers
about:bl 3/
