Databricks Certified Data Engineer
Profesional Exam and Answers |Latest
Update|
A data analyst is running a shell script in all the notebooks attached to the cluster. The shell
script contains a long set of commands which is taking a lot of time to complete. As a data
engineer, which of the following statements will you suggest to the data analyst?
Use the init script to execute the shell script faster
Run the script as a workspace admin
Use %md to run the script faster
Increase the number of worker nodes to speed up the script
Run the notebook using Databricks API - ✔✔Use the init script to execute the shell script faster
By using %sh to run a shell script in a Databricks notebook, you end up using only the driver
node of the cluster. If you need to run the shell script on the worker nodes as well, you will
need to add it as an init script on the cluster.
Which of the following is a valid response to a JSON workload passed to 2.0/jobs/create
endpoint of Databricks REST API?
"job name":"name"
"status":200
"job id": 13746
"job id":"13746"
job_id:13746 - ✔✔"job id":13746
A data engineer wants to unmount a mount point mounted at /mnt/mountPoint1 Which of the
following commands can be used by the data engineer?
,dbutils.unmount("/mnt/mountPoint1")
dbutils.mount.unmount("/mnt/mountPoint1")
dbutils.fs.unmount("/mnt/mountPoint1")
dbutils.widgets("/mnt/mountPoint1")
dbutils.remove_mount("/mnt/mountPoint1") - ✔✔dbutils.fs.unmount("/mnt/mountPoint1")
A Databricks admin has created a Cluster and provided Can Restart permission to a group of
users named data_analysts Which of the following cluster-related actions cannot be performed
by the members of the data_analysts group?
Terminate the cluster
View Spark UI
Restart the cluster
View the cluster metrics
Edit the cluster - ✔✔Edit the cluster
Cluster-level permission is usually set to limit the accessibility to a cluster. There are 4 different
permission levels that can be set to a cluster.
No permissions, Can Attach to , Can Restart, Can Manage
A data engineer has created a new job with a single notebook task intended to run daily at 5
PM. The task intends to run the notebook named show_regular.py which contains 5 cells where
each cell outputs data in the form of a DataFrame. The size of the outputs are :
Cell 1 5.2 MB
Cell 2 6.1 MB
Cell 3 4.4 MB
Cell 4 5.3 MB
,Cell 5 2.5 MB
The job fails on its initial run and the data engineer is clueless about the failure. Which of the
constraint forced the job to fail?
The individual cell output is limited to 6 MB which caused the job to fail
The output size of all the cells in a notebook should not exceed 20 MB. Since it exceeds 20MB, it
failed
The output size of the first cell cannot exceed 5 MB in size, forcing the job to fail.
The job failed due to multiple reasons as the size of any individual cell output in a notebook
cannot increase by 6 MB and the total size of all the outputs cannot be more than 18 MB. -
✔✔The output size of all the cells in a notebook should not exceed 20 MB. Since it exceeds
20MB, it failed
In Databricks, when you add a notebook task to a job, the size limit can force the job to fail. The
size of an individual cell cannot exceed 8 MB whereas the total size of all the cells cannot be
more than 20 MB.
While creating a cluster, the admin added the ENV=PROD in Env variables. The engineering
team is using this env variable for fetching data from the S3 bucket in the fetch_details
notebook.
A new member joins the team and needs to fetch data from the DEV env instead of PROD. The
member runs the following command in a new notebook to update the env variable:
import os
os.environ["ENV"] = 'DEV'
Which of the following statements describes the effect of the command?
The command will fail with an error message as the env variable cannot be changed.
The command will be executed successfully but the env variable will not be changed.
The env variable's value will be changed but the value in all other notebooks will remain PROD.
, The env variable's value will be updated in the cluster properties and all the notebooks
attached to the cluster will start using the updated value. - ✔✔The env variable's value will be
changed but the value in all other notebooks will remain PROD.
The environment variables can be defined while creating or updating a cluster. These
environment variables can be accessed using the following statements:
import os
os.environ["ENV"]
Once the variables are defined, they can be changed but the change will not reflect in other
notebooks. If an environment variable is changed in notebook A, the effect of this change
cannot be experienced in notebook B.
Two tables were created with the following SQL statements:
CREATE TABLE flights
...
using DELTA;
CREATE TABLE airports
....
USING PARQUET
LOCATION 'dbfs:/airports/data/';
The DBFS location dbfs:/airports/data/ contains static data for the airports while the data in the
flights table is updated on a daily basis. A data engineer runs the following SQL statements.
DROP TABLE flights;
DROP TABLE airports;
Which of the following statements is true about the data present in the tables?
The data in both tables can be retrieved by using the Time Travel feature of Delta tables.
Profesional Exam and Answers |Latest
Update|
A data analyst is running a shell script in all the notebooks attached to the cluster. The shell
script contains a long set of commands which is taking a lot of time to complete. As a data
engineer, which of the following statements will you suggest to the data analyst?
Use the init script to execute the shell script faster
Run the script as a workspace admin
Use %md to run the script faster
Increase the number of worker nodes to speed up the script
Run the notebook using Databricks API - ✔✔Use the init script to execute the shell script faster
By using %sh to run a shell script in a Databricks notebook, you end up using only the driver
node of the cluster. If you need to run the shell script on the worker nodes as well, you will
need to add it as an init script on the cluster.
Which of the following is a valid response to a JSON workload passed to 2.0/jobs/create
endpoint of Databricks REST API?
"job name":"name"
"status":200
"job id": 13746
"job id":"13746"
job_id:13746 - ✔✔"job id":13746
A data engineer wants to unmount a mount point mounted at /mnt/mountPoint1 Which of the
following commands can be used by the data engineer?
,dbutils.unmount("/mnt/mountPoint1")
dbutils.mount.unmount("/mnt/mountPoint1")
dbutils.fs.unmount("/mnt/mountPoint1")
dbutils.widgets("/mnt/mountPoint1")
dbutils.remove_mount("/mnt/mountPoint1") - ✔✔dbutils.fs.unmount("/mnt/mountPoint1")
A Databricks admin has created a Cluster and provided Can Restart permission to a group of
users named data_analysts Which of the following cluster-related actions cannot be performed
by the members of the data_analysts group?
Terminate the cluster
View Spark UI
Restart the cluster
View the cluster metrics
Edit the cluster - ✔✔Edit the cluster
Cluster-level permission is usually set to limit the accessibility to a cluster. There are 4 different
permission levels that can be set to a cluster.
No permissions, Can Attach to , Can Restart, Can Manage
A data engineer has created a new job with a single notebook task intended to run daily at 5
PM. The task intends to run the notebook named show_regular.py which contains 5 cells where
each cell outputs data in the form of a DataFrame. The size of the outputs are :
Cell 1 5.2 MB
Cell 2 6.1 MB
Cell 3 4.4 MB
Cell 4 5.3 MB
,Cell 5 2.5 MB
The job fails on its initial run and the data engineer is clueless about the failure. Which of the
constraint forced the job to fail?
The individual cell output is limited to 6 MB which caused the job to fail
The output size of all the cells in a notebook should not exceed 20 MB. Since it exceeds 20MB, it
failed
The output size of the first cell cannot exceed 5 MB in size, forcing the job to fail.
The job failed due to multiple reasons as the size of any individual cell output in a notebook
cannot increase by 6 MB and the total size of all the outputs cannot be more than 18 MB. -
✔✔The output size of all the cells in a notebook should not exceed 20 MB. Since it exceeds
20MB, it failed
In Databricks, when you add a notebook task to a job, the size limit can force the job to fail. The
size of an individual cell cannot exceed 8 MB whereas the total size of all the cells cannot be
more than 20 MB.
While creating a cluster, the admin added the ENV=PROD in Env variables. The engineering
team is using this env variable for fetching data from the S3 bucket in the fetch_details
notebook.
A new member joins the team and needs to fetch data from the DEV env instead of PROD. The
member runs the following command in a new notebook to update the env variable:
import os
os.environ["ENV"] = 'DEV'
Which of the following statements describes the effect of the command?
The command will fail with an error message as the env variable cannot be changed.
The command will be executed successfully but the env variable will not be changed.
The env variable's value will be changed but the value in all other notebooks will remain PROD.
, The env variable's value will be updated in the cluster properties and all the notebooks
attached to the cluster will start using the updated value. - ✔✔The env variable's value will be
changed but the value in all other notebooks will remain PROD.
The environment variables can be defined while creating or updating a cluster. These
environment variables can be accessed using the following statements:
import os
os.environ["ENV"]
Once the variables are defined, they can be changed but the change will not reflect in other
notebooks. If an environment variable is changed in notebook A, the effect of this change
cannot be experienced in notebook B.
Two tables were created with the following SQL statements:
CREATE TABLE flights
...
using DELTA;
CREATE TABLE airports
....
USING PARQUET
LOCATION 'dbfs:/airports/data/';
The DBFS location dbfs:/airports/data/ contains static data for the airports while the data in the
flights table is updated on a daily basis. A data engineer runs the following SQL statements.
DROP TABLE flights;
DROP TABLE airports;
Which of the following statements is true about the data present in the tables?
The data in both tables can be retrieved by using the Time Travel feature of Delta tables.
