WESTERN GOVERNORS UNIVERSITY
WGU D608
Data Processing | DTAN 6221
ETL PIPELINE IMPLEMENTATION
Data Extraction, Transformation, Loading
and Validation
Project: Harborline Retail Automated Sales Data Pipeline
DOCUMENT INFORMATION DETAIL
Course WGU D608 - Data Processing (DTAN 6221)
Assessment component ETL implementation; official task numbering unverified
Case organization Harborline Retail (fictional)
Technologies Python, SQLite, Apache Airflow, Amazon S3, Amazon Redshift
Verified work Locally executed ETL and seven automated checks
Not verified Airflow/AWS deployment, cloud execution, current private WGU rubric
Original technical case study using synthetic input records. The local Python/SQLite implementation was executed and
tested. The companion AWS/Airflow project is supplied as an unexecuted deployment example.
Harborline Retail | ETL Implementation Case Study 1
, WGU D608 | DATA PROCESSING ETL PIPELINE IMPLEMENTATION
Contents
A. Assessment Scope and Project Summary
B. Business Problem and Requirements
C. Architecture and ETL Workflow
D. Data Sources and Contracts
E. Data Model and Storage Design
F. Local ETL Implementation
G. AWS and Apache Airflow Implementation
H. Data Quality and Reconciliation
I. Local Testing and Verified Results
J. Pipeline Operations, Security and Recovery
K. Implementation Risks and Limitations
L. Deployment Validation and Evidence Requirements
M. Evaluation and Conclusion
N. References
Appendix A. Representative Python Implementation
Appendix B. Airflow and Redshift Code Excerpts
Appendix C. Test Summary and Execution Commands
Document conventions. Code blocks marked example describe the proposed AWS path. Recorded local results apply only to
the six-event synthetic dataset. Complete executable source files are retained in the companion ZIP package.
Harborline Retail | ETL Implementation Case Study 2
WGU D608
Data Processing | DTAN 6221
ETL PIPELINE IMPLEMENTATION
Data Extraction, Transformation, Loading
and Validation
Project: Harborline Retail Automated Sales Data Pipeline
DOCUMENT INFORMATION DETAIL
Course WGU D608 - Data Processing (DTAN 6221)
Assessment component ETL implementation; official task numbering unverified
Case organization Harborline Retail (fictional)
Technologies Python, SQLite, Apache Airflow, Amazon S3, Amazon Redshift
Verified work Locally executed ETL and seven automated checks
Not verified Airflow/AWS deployment, cloud execution, current private WGU rubric
Original technical case study using synthetic input records. The local Python/SQLite implementation was executed and
tested. The companion AWS/Airflow project is supplied as an unexecuted deployment example.
Harborline Retail | ETL Implementation Case Study 1
, WGU D608 | DATA PROCESSING ETL PIPELINE IMPLEMENTATION
Contents
A. Assessment Scope and Project Summary
B. Business Problem and Requirements
C. Architecture and ETL Workflow
D. Data Sources and Contracts
E. Data Model and Storage Design
F. Local ETL Implementation
G. AWS and Apache Airflow Implementation
H. Data Quality and Reconciliation
I. Local Testing and Verified Results
J. Pipeline Operations, Security and Recovery
K. Implementation Risks and Limitations
L. Deployment Validation and Evidence Requirements
M. Evaluation and Conclusion
N. References
Appendix A. Representative Python Implementation
Appendix B. Airflow and Redshift Code Excerpts
Appendix C. Test Summary and Execution Commands
Document conventions. Code blocks marked example describe the proposed AWS path. Recorded local results apply only to
the six-event synthetic dataset. Complete executable source files are retained in the companion ZIP package.
Harborline Retail | ETL Implementation Case Study 2