WGU D501 MACHINE LEARNING DEVOPS PROJECT 1: BUILD AN ML
PIPELINE FOR SHORT-TERM RENTAL PRICES IN NYC | MLFLOW,
RANDOM FOREST, DATA VALIDATION & MODEL RELEASE | 2026/2027
UPDATED | COMPLETE PROJECT SOLUTION [QUESTION 1-200] AND
ANSWERS UPDATED 2026/2027 | DETAILED RATIONALES
INTRODUCTION
The WGU D501 Machine Learning DevOps Project 1 focuses on applying machine-
learning engineering and DevOps principles to an end-to-end predictive analytics
workflow. A short-term rental price prediction project provides an opportunity to
integrate data preparation, validation, feature engineering, model development,
experiment tracking, reproducibility, and controlled model release into a cohesive ML
pipeline.
This practice bank is designed for learners preparing to demonstrate applied
understanding of concepts associated with a project using Random Forest, MLflow,
automated or repeatable data validation, and model-release practices. The questions
emphasize difficult, scenario-based decision making rather than simple terminology.
They are intended to test whether a student can identify appropriate engineering
choices when faced with data-quality problems, model-performance tradeoffs,
experiment-management issues, reproducibility requirements, and deployment
concerns.
The questions should be used as study material alongside the student's current D501
task instructions, rubric, course materials, and project-specific requirements. They
emphasize reasoning about an end-to-end ML workflow and the relationship between
data quality, model development, experiment tracking, validation, and release.
CORE DOMAINS TESTED
1.
ML Project Architecture — Designing a reproducible end-to-end machine-
learning workflow for short-term rental price prediction.
2.
3.
Data Acquisition and Validation — Detecting missing, invalid, inconsistent,
duplicated, or structurally incorrect rental data.
4.
5.
Data Preprocessing — Handling categorical variables, numerical features,
missing values, and transformations without introducing leakage.
1
, 6.
7.
Exploratory Data Analysis — Interpreting rental-market patterns and
identifying issues that can affect modeling.
8.
9.
Feature Engineering — Creating meaningful predictors while maintaining
appropriate temporal and leakage controls.
10.
11.
Train/Validation/Test Strategy — Separating datasets correctly and
preserving the integrity of model evaluation.
12.
13.
Random Forest Regression — Selecting, tuning, interpreting, and evaluating
a tree-based regression model.
14.
15.
Model Evaluation — Applying MAE, RMSE, R², residual analysis, and
business-oriented evaluation appropriately.
16.
17.
MLflow Experiment Tracking — Tracking parameters, metrics, artifacts,
datasets, and model versions systematically.
18.
19.
Reproducibility — Ensuring that experiments and model artifacts can be
reconstructed consistently.
20.
21.
Model Release and Lifecycle Management — Moving validated models
through controlled release stages.
22.
2
, 23.
ML DevOps and Governance — Applying validation gates, version control,
documentation, monitoring, and rollback principles.
24.
25.
Model Interpretation and Business Use — Translating model results into
responsible recommendations for the rental-price use case.
26.
27.
Troubleshooting and Scenario Analysis — Diagnosing pipeline, data,
experiment-tracking, and model-release failures.
28.
QUESTIONS 1-100
Q1:
A data scientist builds a Random Forest regression model to predict nightly rental
prices. The model achieves an R² of 0.91 on the training data but only 0.48 on the
held-out test data. The dataset contains many listings with highly similar
characteristics. Which interpretation is MOST appropriate?
A) The model is necessarily underfitting because the test R² is lower than the training
R²
B) The large train-test performance gap is evidence of potential overfitting and
warrants validation of model complexity and data-splitting procedures
C) The model should be released because an R² above 0.40 is automatically
production-ready
D) The test set should be added to the training set until the two R² values become
equal
Rationale: The correct answer is B because a substantial difference between training
and unseen-test performance is a classic warning sign of overfitting or an evaluation-
design problem. Option A is incorrect because underfitting generally produces poor
performance on both training and unseen data. Option C is incorrect because no
universal R² threshold establishes production readiness; business requirements and
additional metrics matter. Option D is incorrect because incorporating the test set
into training destroys its role as an unbiased evaluation set.
Q2:
3
, During data validation, the pipeline finds several rental records with negative values
for minimum_nights. What should the pipeline do FIRST?
A) Replace every negative value with zero without recording the change
B) Apply a predefined validation rule that identifies the records as invalid and
routes them according to the project's data-quality policy
C) Train the Random Forest anyway because tree models can tolerate invalid values
D) Delete the entire dataset because one invalid field exists
Rationale: The correct answer is B because data validation should enforce explicit
quality rules before modeling and should handle failures according to a documented
policy. Option A is inappropriate because an arbitrary correction can alter the
meaning of the data and should not occur without a defined rule. Option C is
incorrect because model tolerance does not make invalid business values valid.
Option D is unnecessarily destructive because individual invalid records can often be
quarantined, corrected, or removed according to policy.
Q3:
A feature called review_score_after_checkout appears highly predictive of rental
price. Investigation shows that the value is generated only after the guest completes a
stay. The model is intended for pricing a listing before a reservation occurs. What is
the primary problem?
A) Multicollinearity
B) Underfitting
C) Data leakage caused by using information unavailable at prediction time
D) Insufficient Random Forest depth
Rationale: The correct answer is C because the feature contains information that
would not exist when the prediction is supposed to be made. This gives the model
access to future information and can produce unrealistically strong validation
performance. Option A is incorrect because correlation among predictors is not the
central problem. Option B is incorrect because the issue is not excessive model
simplicity. Option D is incorrect because changing tree depth cannot make a post-
outcome feature valid.
Q4:
A project requires every training run to be traceable to the exact model parameters,
evaluation metrics, and artifacts used to produce it. Which MLflow capability most
directly supports this requirement?
A) Random Forest bootstrap sampling
B) Experiment and run tracking
C) One-hot encoding
D) Residual normalization
4
PIPELINE FOR SHORT-TERM RENTAL PRICES IN NYC | MLFLOW,
RANDOM FOREST, DATA VALIDATION & MODEL RELEASE | 2026/2027
UPDATED | COMPLETE PROJECT SOLUTION [QUESTION 1-200] AND
ANSWERS UPDATED 2026/2027 | DETAILED RATIONALES
INTRODUCTION
The WGU D501 Machine Learning DevOps Project 1 focuses on applying machine-
learning engineering and DevOps principles to an end-to-end predictive analytics
workflow. A short-term rental price prediction project provides an opportunity to
integrate data preparation, validation, feature engineering, model development,
experiment tracking, reproducibility, and controlled model release into a cohesive ML
pipeline.
This practice bank is designed for learners preparing to demonstrate applied
understanding of concepts associated with a project using Random Forest, MLflow,
automated or repeatable data validation, and model-release practices. The questions
emphasize difficult, scenario-based decision making rather than simple terminology.
They are intended to test whether a student can identify appropriate engineering
choices when faced with data-quality problems, model-performance tradeoffs,
experiment-management issues, reproducibility requirements, and deployment
concerns.
The questions should be used as study material alongside the student's current D501
task instructions, rubric, course materials, and project-specific requirements. They
emphasize reasoning about an end-to-end ML workflow and the relationship between
data quality, model development, experiment tracking, validation, and release.
CORE DOMAINS TESTED
1.
ML Project Architecture — Designing a reproducible end-to-end machine-
learning workflow for short-term rental price prediction.
2.
3.
Data Acquisition and Validation — Detecting missing, invalid, inconsistent,
duplicated, or structurally incorrect rental data.
4.
5.
Data Preprocessing — Handling categorical variables, numerical features,
missing values, and transformations without introducing leakage.
1
, 6.
7.
Exploratory Data Analysis — Interpreting rental-market patterns and
identifying issues that can affect modeling.
8.
9.
Feature Engineering — Creating meaningful predictors while maintaining
appropriate temporal and leakage controls.
10.
11.
Train/Validation/Test Strategy — Separating datasets correctly and
preserving the integrity of model evaluation.
12.
13.
Random Forest Regression — Selecting, tuning, interpreting, and evaluating
a tree-based regression model.
14.
15.
Model Evaluation — Applying MAE, RMSE, R², residual analysis, and
business-oriented evaluation appropriately.
16.
17.
MLflow Experiment Tracking — Tracking parameters, metrics, artifacts,
datasets, and model versions systematically.
18.
19.
Reproducibility — Ensuring that experiments and model artifacts can be
reconstructed consistently.
20.
21.
Model Release and Lifecycle Management — Moving validated models
through controlled release stages.
22.
2
, 23.
ML DevOps and Governance — Applying validation gates, version control,
documentation, monitoring, and rollback principles.
24.
25.
Model Interpretation and Business Use — Translating model results into
responsible recommendations for the rental-price use case.
26.
27.
Troubleshooting and Scenario Analysis — Diagnosing pipeline, data,
experiment-tracking, and model-release failures.
28.
QUESTIONS 1-100
Q1:
A data scientist builds a Random Forest regression model to predict nightly rental
prices. The model achieves an R² of 0.91 on the training data but only 0.48 on the
held-out test data. The dataset contains many listings with highly similar
characteristics. Which interpretation is MOST appropriate?
A) The model is necessarily underfitting because the test R² is lower than the training
R²
B) The large train-test performance gap is evidence of potential overfitting and
warrants validation of model complexity and data-splitting procedures
C) The model should be released because an R² above 0.40 is automatically
production-ready
D) The test set should be added to the training set until the two R² values become
equal
Rationale: The correct answer is B because a substantial difference between training
and unseen-test performance is a classic warning sign of overfitting or an evaluation-
design problem. Option A is incorrect because underfitting generally produces poor
performance on both training and unseen data. Option C is incorrect because no
universal R² threshold establishes production readiness; business requirements and
additional metrics matter. Option D is incorrect because incorporating the test set
into training destroys its role as an unbiased evaluation set.
Q2:
3
, During data validation, the pipeline finds several rental records with negative values
for minimum_nights. What should the pipeline do FIRST?
A) Replace every negative value with zero without recording the change
B) Apply a predefined validation rule that identifies the records as invalid and
routes them according to the project's data-quality policy
C) Train the Random Forest anyway because tree models can tolerate invalid values
D) Delete the entire dataset because one invalid field exists
Rationale: The correct answer is B because data validation should enforce explicit
quality rules before modeling and should handle failures according to a documented
policy. Option A is inappropriate because an arbitrary correction can alter the
meaning of the data and should not occur without a defined rule. Option C is
incorrect because model tolerance does not make invalid business values valid.
Option D is unnecessarily destructive because individual invalid records can often be
quarantined, corrected, or removed according to policy.
Q3:
A feature called review_score_after_checkout appears highly predictive of rental
price. Investigation shows that the value is generated only after the guest completes a
stay. The model is intended for pricing a listing before a reservation occurs. What is
the primary problem?
A) Multicollinearity
B) Underfitting
C) Data leakage caused by using information unavailable at prediction time
D) Insufficient Random Forest depth
Rationale: The correct answer is C because the feature contains information that
would not exist when the prediction is supposed to be made. This gives the model
access to future information and can produce unrealistically strong validation
performance. Option A is incorrect because correlation among predictors is not the
central problem. Option B is incorrect because the issue is not excessive model
simplicity. Option D is incorrect because changing tree depth cannot make a post-
outcome feature valid.
Q4:
A project requires every training run to be traceable to the exact model parameters,
evaluation metrics, and artifacts used to produce it. Which MLflow capability most
directly supports this requirement?
A) Random Forest bootstrap sampling
B) Experiment and run tracking
C) One-hot encoding
D) Residual normalization
4
