• Wrong document? Swap it for free
  • Written by students who passed
  • Immediately available after payment
  • Read online or as PDF
Sell
Where do you study
Your language
Document preview thumbnail
Preview 2 out of 11 pages
Exam (elaborations)

Databricks Competition Study

Document preview thumbnail
Preview 2 out of 11 pages

What does the Lakehouse consist of? - answer-The lakehouse brings the scalability and cost effectiveness of data lakes with the reliability and performance of data warehouses How does Databricks differentiate from Snowflake? - answer-Data Science/Machine Learning Data Ingestion ETL Streaming Capabilities Data Sharing With Snowflake, you will have high costs, limited capabilities, limited unstructured data support, inefficient engineering and vendor lock in For Data Science- how does Databricks differ from Snowflake? - answer-Snowflake is built for SQL workloads. They either use Snowpark or rely on 3rd party tools for DS and ML-like Dataiku or DataRobot Databricks can allow your data scientist to use any libraries/languages in the same platform as their DE, and manage the end-to-end ML pipeline with MLflow For Data Ingestion- how does Databricks differ from Snowflake - answer-With Snowflake ingestion go through various Snowflake stages, often needing to utilize limited SQL pipelines or Snowpipe which is inefficient and expensive 2) Snowflake tax - egress tax moving data to and from and as that data is being stored Databricks - ingestion is simple with AutoLoader where data is automatically transformed into Delta Tables For ETL- how does Databricks differ from Snowflake - answer-Snowflake has no ETL tools rely heavily on 3rd party vendors which increases cost and complexity Databricks - Delta Live Tables What is Delta Live Tables? - answer-Delta Live Tables is the first ETL framework that allows you to build reliable data pipelines (accelerate ETL development) - automatically manages your infrastructure at scale so data analysts and engineers can spend less time on tooling and focus on getting value from data. DLT- fully supports Python and SQL and works with both batch and streaming DLT - manages task oircehstration, cluster management, monitoring, data quality and error handling supports CDC (change data capture) Streaming Capabilities- how does Databricks differ from Snowflake - answer-Databricks reads in any streaming services while Snowflake only supports Kafka and was not designed for high velocity data but rather structured data at rest What Databricks features helps out with observability and governance? - answer-Expectations- help prevent bad data from flowing into tables, track data quality over time and provide tools to troubleshoot bad data with granular pipeline observability so you get a high fidelity lineage diagram of your pipeline, track dependencies and aggregate data quality metrics across all of your pipeline. For Data Sharing- how does Databricks differ from Snowflake? - answer-Snowflake's data format is proprietary, users can only share data with other Snowflake accounts (vendor lock in) -Snowflake would take the data from your cloud storage, conduct transformations and then push the data back so you are paying an egress tax to and from and as that data is going to be stored -pay for compute to send data Delta Sharing - an open standard for data sharing, no replication of datasets How does Delta Lake differ Iceberg? - answer-1) Overall performance - loading and querying data is 3.5x faster 2) Load Performance- load from Parquet to intended formats (delta is faster) 3) Query performance - Delta is 4.5x faster What are Data cleanrooms? - answer-It's a secure environment to run computations on joint data Run any computation on Python, SQL, R, or java No data replications Scalability What is Databricks Marketplace? - answer-open marketplace for data solutions, built on Delta Sharing Consist of Notebooks Data files, Data Tables Solution Accelerators ML Models Dashboards Why have data marketplaces seen limited use? - answer-Closed platforms (one per vendor) Limited to just datasets What is Project Lightspeed? - answer-Faster and simpler stream processing 1) predictable low latency 2) enhanced functionality 3) Operations and Troubleshooting 4) Connectors & Ecosystem Databricks Workflows - answer-Fully managed orchestration service Orchestration is the configuration of multiple tasks into one complete end-to-end process the goal is to streamline and optimize the execution of frequent, repeatable processes EMR - answer-ETL/stream processing many open sourced projects (Hadoop, Presto, Jupyter) are packaged and rolled into a single click which integrates well with AWS ecosystem cloud big data platform for running large scale distributed data processing jobs, interactive SQL queries EMR- EMR runtime which provides performance enhancements What does EMR

Content preview

Databricks Competition Study
What does the Lakehouse consist of? - answer-The lakehouse brings the scalability and cost
effectiveness of data lakes with the reliability and performance of data warehouses

How does Databricks differentiate from Snowflake? - answer-Data Science/Machine Learning
Data Ingestion
ETL
Streaming Capabilities
Data Sharing
With Snowflake, you will have high costs, limited capabilities, limited unstructured data
support, inefficient engineering and vendor lock in

For Data Science- how does Databricks differ from Snowflake? - answer-Snowflake is built for
SQL workloads. They either use Snowpark or rely on 3rd party tools for DS and ML-like Dataiku
or DataRobot
Databricks can allow your data scientist to use any libraries/languages in the same platform as
their DE, and manage the end-to-end ML pipeline with MLflow

For Data Ingestion- how does Databricks differ from Snowflake - answer-With Snowflake
ingestion go through various Snowflake stages, often needing to utilize limited SQL pipelines or
Snowpipe which is inefficient and expensive
2) Snowflake tax - egress tax moving data to and from and as that data is being stored
Databricks - ingestion is simple with AutoLoader where data is automatically transformed into
Delta Tables

For ETL- how does Databricks differ from Snowflake - answer-Snowflake has no ETL tools rely
heavily on 3rd party vendors which increases cost and complexity
Databricks - Delta Live Tables

What is Delta Live Tables? - answer-Delta Live Tables is the first ETL framework that allows you
to build reliable data pipelines (accelerate ETL development)
- automatically manages your infrastructure at scale so data analysts and engineers can spend
less time on tooling and focus on getting value from data.
DLT- fully supports Python and SQL and works with both batch and streaming
DLT - manages task oircehstration, cluster management, monitoring, data quality and error
handling
supports CDC (change data capture)

Streaming Capabilities- how does Databricks differ from Snowflake - answer-Databricks reads in
any streaming services while Snowflake only supports Kafka and was not designed for high
velocity data but rather structured data at rest

, What Databricks features helps out with observability and governance? - answer-Expectations-
help prevent bad data from flowing into tables, track data quality over time and provide tools
to troubleshoot bad data with granular pipeline observability so you get a high fidelity lineage
diagram of your pipeline, track dependencies and aggregate data quality metrics across all of
your pipeline.

For Data Sharing- how does Databricks differ from Snowflake? - answer-Snowflake's data
format is proprietary, users can only share data with other Snowflake accounts (vendor lock in)
-Snowflake would take the data from your cloud storage, conduct transformations and then
push the data back so you are paying an egress tax to and from and as that data is going to be
stored
-pay for compute to send data

Delta Sharing - an open standard for data sharing, no replication of datasets

How does Delta Lake differ Iceberg? - answer-1) Overall performance - loading and querying
data is 3.5x faster
2) Load Performance- load from Parquet to intended formats (delta is faster)
3) Query performance - Delta is 4.5x faster

What are Data cleanrooms? - answer-It's a secure environment to run computations on joint
data
Run any computation on Python, SQL, R, or java
No data replications
Scalability

What is Databricks Marketplace? - answer-open marketplace for data solutions, built on Delta
Sharing
Consist of
Notebooks
Data files, Data Tables
Solution Accelerators
ML Models
Dashboards

Why have data marketplaces seen limited use? - answer-Closed platforms (one per vendor)
Limited to just datasets

What is Project Lightspeed? - answer-Faster and simpler stream processing
1) predictable low latency
2) enhanced functionality
3) Operations and Troubleshooting
4) Connectors & Ecosystem

Document information

Uploaded on
September 29, 2024
Number of pages
11
Written in
2024/2025
Type
Exam (elaborations)
Contains
Questions & answers
$12.49

Wrong document? Swap it for free Within 14 days of purchase and before downloading, you can choose a different document. You can simply spend the amount again.
Written by students who passed
Immediately available after payment
Read online or as PDF

Seller avatar
Reputation scores are based on the amount of documents a seller has sold for a fee and the reviews they have received for those documents. There are three levels: Bronze, Silver and Gold. The better the reputation, the more your can rely on the quality of the sellers work.
TOPDOCTOR
5.0
(1)
Sold
10
Followers
5
Items
3387
Last sold
1 year ago



Why students choose Stuvia

Created by fellow students, verified by reviews

Quality you can trust: written by students who passed their tests and reviewed by others who've used these notes.

Didn't get what you expected? Choose another document

No worries! You can instantly pick a different document that better fits what you're looking for.

Pay as you like, start learning right away

No subscription, no commitments. Pay the way you're used to via credit card and download your PDF document instantly.

Student with book image

“Bought, downloaded, and aced it. It really can be that simple.”

Alisha Student

Working on your references?

Create accurate citations in APA, MLA and Harvard with our free citation generator.

Working on your references?

Frequently asked questions