Ultimate Guide – Complete Data
Engineering Project with AWS, Apache
Airflow, Python, SQL, and Automated Data
Quality Validation Tests (2026/2027
Updated Edition)
1. Which phase of an ETL pipeline is responsible for reading raw data
from source systems such as APIs, databases, or flat files?
A) Transformation
B) Loading
C) Extraction
D) Validation
Answer: C) Extraction
Rationale: Extraction is the first chronological stage where data is pulled from
operational source systems before any cleaning or structural changes occur.
Transformation (B) occurs afterward, and Loading (D) is the final step .
2. In a traditional ETL architecture, which step immediately follows the
Transformation phase?
A) Extraction
B) Loading
C) Profiling
D) Aggregation
,Answer: B) Loading
Rationale: The ETL sequence is Extract → Transform → Load. Loading writes
the processed data into the target data warehouse or data lake. Aggregation
(D) is a transformation task, not a separate phase .
3. Which ETL subsystem is specifically designed to manage surrogate
keys for dimension tables?
A) Aggregate Builder
B) Surrogate Key Generator
C) Lineage and Dependency Analyzer
D) Recover and Restart System
Answer: B) Surrogate Key Generator
Rationale: The Surrogate Key Generator creates and maintains unique,
system-assigned keys for dimension records to ensure referential integrity
independent of source-system natural keys .
4. A company requires near real-time processing for point-of-sale
transactions. Which processing paradigm is most appropriate?
A) Batch ETL
B) Streaming Data Processing
C) Full Refresh Loading
D) Nightly Snapshot
Answer: B) Streaming Data Processing
Rationale: Streaming pipelines process events as they occur, minimizing
latency. Batch ETL (A) and full refreshes (C) introduce delays unsuitable for
real-time POS systems .
5. Which ETL subsystem ensures that a pipeline can resume from the
point of failure without data loss after a power outage?
A) Surrogate Key Generator
B) Recover and Restart System
,C) Aggregate Builder
D) Data Cleansing Subsystem
Answer: B) Recover and Restart System
Rationale: This subsystem maintains checkpoints and restart logic. Backups
alone are insufficient because they do not automatically resume pipeline
state .
6. What term describes the ability to trace the origin, movement, and
transformation of data throughout its lifecycle?
A) Data Democratization
B) Data Lineage
C) Data Profiling
D) Data Partitioning
Answer: B) Data Lineage
Rationale: Data lineage provides visibility into data flow and usage, which is
critical for regulatory compliance and debugging pipeline errors .
7. Which ETL subsystem precomputes summary statistics to improve
downstream query performance?
A) Surrogate Key Generator
B) Aggregate Builder
C) Slowly Changing Dimension Manager
D) Sorting System
Answer: B) Aggregate Builder
Rationale: The Aggregate Builder pre-calculates groupings and sums during
the ETL process so that BI dashboards do not need to compute them at query
time .
8. When customer attributes such as address or loyalty tier change
over time, which subsystem should be utilized?
, A) Surrogate Key Generator
B) Slowly Changing Dimension (SCD) Manager
C) Aggregate Builder
D) File-Based Extractor
Answer: B) Slowly Changing Dimension (SCD) Manager
Rationale: SCD techniques preserve historical attribute values while allowing
current records to reflect updates, which is essential for accurate historical
reporting .
9. Which of the following statements about data cleansing in ETL is
TRUE?
A) Data cleansing is optional and only needed for unstructured data.
B) Data cleansing is a mandatory transformation step to handle errors and
missing values.
C) Data cleansing occurs during the Loading phase.
D) Data cleansing is performed only after data reaches the warehouse.
Answer: B) Data cleansing is a mandatory transformation step to
handle errors and missing values.
Rationale: Cleaning errors and handling nulls are core transformation
activities. Treating cleansing as optional leads to unreliable analytics .
10. What is the primary purpose of Change Data Capture (CDC) in
extraction?
A) To reload the entire source table every night.
B) To identify and extract only rows that have been inserted, updated, or
deleted since the last run.
C) To compress data before loading.
D) To generate surrogate keys automatically.
Answer: B) To identify and extract only rows that have been inserted,
updated, or deleted since the last run.