Pre-Test 2
What is the best way to describe a data Lakehouse compared to a data warehouse? -
answer A data Lakehouse enables both batch and streaming analytics.
What are the 8 characteristics of a data Lakehouse? - answer1. Transaction support
2. Schema enforcement and governance
3. BI support
4. Storage is decoupled from compute
5. Openness
6. Support for diverse data types ranging from unstructured to structured data
7. Support for diverse workloads
8. End-to-end streaming
You are designing an analytical to store structured data from your e-commerce platform
and unstructured data from website traffic and app store, how would you approach
where you store this data? - answer Data Lakehouse can store structured and
unstructured data and can enforce schema
You are currently working on a production job failure with a job set up in job clusters due
to a data issue, what cluster do you need to start to investigate and analyze the data? -
answerAll-purpose cluster/ interactive cluster is the recommended way to run
commands and view the data.
Which of the following describes how Databricks Repos can help facilitate CI/CD
workflows on the Databricks Lakehouse Platform? - answerDatabricks Repos can
commit or push code changes to trigger a CI/CD process
You noticed that colleague is manually copying the notebook with _bkp to store the
previous versions, which of the following feature would you recommend instead. -
answerDatabricks notebooks support automatic change tracking and versioning.
Newly joined data analyst requested read-only access to tables, assuming you are
owner/admin which section of Databricks platform is going to facilitate granting select
access to the user - answerData Explorer
How does a Delta Lake differ from a traditional data lake? - answerDelta lake is an open
storage format like parquet with additional capabilities that can provide reliability,
security, and performance
Delta lake IS - answer· Open source
· Builds up on standard data format
· Optimized for cloud object storage
, · Built for scalable metadata handling
Delta lake is NOT - answer· Proprietary technology
· Storage format
· Storage medium
· Database service or data warehouse
As a Data Engineer, you were asked to create a delta table to store below transaction
data? - answerCREATE TABLE tablename (
column type
)
Which of the following is a correct statement on how the data is organized in the storage
when when managing a DELTA table? - answerAll of the data is broken down into one
or many parquet files, log files are broken down into one or many json files, and each
transaction creates a new data file(s) and log file.
What is the underlying technology that makes the Auto Loader work? -
answerStructured Streaming
You are currently working to ingest millions of files that get uploaded to the cloud object
storage for consumption, and you are asked to build a process to ingest this data, the
schema of the file is expected to change over time, and the ingestion process should be
able to handle these changes automatically. Which of the following method can be used
to ingest the data incrementally? - answerAUTO LOADER
When should you use AUTO LOADER instead of COPY INTO? - answer- You want to
load data from a file location that contains files in the order of millions or higher. Auto
Loader can discover files more efficiently than the COPY INTO SQL command and can
split file processing into multiple batches.
- COPY INTO only directory listing but AUTO LOADER supports File notification method
where the Auto Loader continues to ingest files as they arrive in cloud object storage
leveraging cloud provider(Queues and triggers) and Spark's structured streaming.
- Your data schema evolves frequently. Auto Loader provides better support for schema
inference and evolution.
What is the purpose of the bronze layer in a Multi-hop architecture? - answer- Raw copy
of ingested data
- Replaces traditional data lake
- Provides efficient storage and querying of full, unprocessed history of data
- No schema is applied at this layer
What is the purpose of a silver layer in Multi hop architecture? - answer1. Reduces data
storage complexity, latency, and redundency
2. Optimizes ETL throughput and analytic query performance
3. Preserves grain of original data (without aggregation)
What is the best way to describe a data Lakehouse compared to a data warehouse? -
answer A data Lakehouse enables both batch and streaming analytics.
What are the 8 characteristics of a data Lakehouse? - answer1. Transaction support
2. Schema enforcement and governance
3. BI support
4. Storage is decoupled from compute
5. Openness
6. Support for diverse data types ranging from unstructured to structured data
7. Support for diverse workloads
8. End-to-end streaming
You are designing an analytical to store structured data from your e-commerce platform
and unstructured data from website traffic and app store, how would you approach
where you store this data? - answer Data Lakehouse can store structured and
unstructured data and can enforce schema
You are currently working on a production job failure with a job set up in job clusters due
to a data issue, what cluster do you need to start to investigate and analyze the data? -
answerAll-purpose cluster/ interactive cluster is the recommended way to run
commands and view the data.
Which of the following describes how Databricks Repos can help facilitate CI/CD
workflows on the Databricks Lakehouse Platform? - answerDatabricks Repos can
commit or push code changes to trigger a CI/CD process
You noticed that colleague is manually copying the notebook with _bkp to store the
previous versions, which of the following feature would you recommend instead. -
answerDatabricks notebooks support automatic change tracking and versioning.
Newly joined data analyst requested read-only access to tables, assuming you are
owner/admin which section of Databricks platform is going to facilitate granting select
access to the user - answerData Explorer
How does a Delta Lake differ from a traditional data lake? - answerDelta lake is an open
storage format like parquet with additional capabilities that can provide reliability,
security, and performance
Delta lake IS - answer· Open source
· Builds up on standard data format
· Optimized for cloud object storage
, · Built for scalable metadata handling
Delta lake is NOT - answer· Proprietary technology
· Storage format
· Storage medium
· Database service or data warehouse
As a Data Engineer, you were asked to create a delta table to store below transaction
data? - answerCREATE TABLE tablename (
column type
)
Which of the following is a correct statement on how the data is organized in the storage
when when managing a DELTA table? - answerAll of the data is broken down into one
or many parquet files, log files are broken down into one or many json files, and each
transaction creates a new data file(s) and log file.
What is the underlying technology that makes the Auto Loader work? -
answerStructured Streaming
You are currently working to ingest millions of files that get uploaded to the cloud object
storage for consumption, and you are asked to build a process to ingest this data, the
schema of the file is expected to change over time, and the ingestion process should be
able to handle these changes automatically. Which of the following method can be used
to ingest the data incrementally? - answerAUTO LOADER
When should you use AUTO LOADER instead of COPY INTO? - answer- You want to
load data from a file location that contains files in the order of millions or higher. Auto
Loader can discover files more efficiently than the COPY INTO SQL command and can
split file processing into multiple batches.
- COPY INTO only directory listing but AUTO LOADER supports File notification method
where the Auto Loader continues to ingest files as they arrive in cloud object storage
leveraging cloud provider(Queues and triggers) and Spark's structured streaming.
- Your data schema evolves frequently. Auto Loader provides better support for schema
inference and evolution.
What is the purpose of the bronze layer in a Multi-hop architecture? - answer- Raw copy
of ingested data
- Replaces traditional data lake
- Provides efficient storage and querying of full, unprocessed history of data
- No schema is applied at this layer
What is the purpose of a silver layer in Multi hop architecture? - answer1. Reduces data
storage complexity, latency, and redundency
2. Optimizes ETL throughput and analytic query performance
3. Preserves grain of original data (without aggregation)
