DP900 - Explore data storage and
processing in Azure
What is Azure Synapse Analytics - answer Azure Synapse Analytics is a generalized
analytics service. You can use it to read data from many sources, process this data,
generate various analyses and models, and save the results.
what two technologies can process data for Azure Synapse Analytics - answerTransact-
SQL
Spark
what is Transact-SQL - answerThis is the same dialect of SQL used by Azure SQL
Database, with some extensions for reading data from external sources, such as
databases, files, and Azure Data Lake storage. You can use these extensions to load
data quickly, generate aggregations and other analytics, create tables and views, and
store information using these tables and views. You can use the results for later
reporting and processing.
what is Spark - answerThis is the same open-source technology used to power Azure
Databricks. You write your analytical code using notebooks in a programming language
such as C#, Scala, Python, or SQL. The Spark libraries provided with Azure Synapse
Analytics enable you to read data from external sources, and also write out data in a
variety of different formats if you need to save your results for further analysis.
what clustered architecture can Azure Synapse Analytics use - answerAzure Synapse
Analytics uses a clustered architecture. Each cluster has a control node that is used as
the entry point to the system. When you run Transact-SQL statements or start Spark
jobs from a notebook, the request is sent to the control node.
what is the control node in Azure Synapse Analytics - answerThe control node runs a
parallel processing engine that splits the operation into a set of tasks that can be run
concurrently. Each task performs part of the workload over a subset of the source data.
Each task is sent to a compute node to actually do the processing. The control node
gathers the results from the compute nodes and combines them into an overall result.
What is Azure Databricks - answerAzure Databricks is an analytics platform optimized
for the Microsoft Azure cloud services platform. Designed with the founders of Apache
Spark, Databricks is integrated with Azure to provide one-click setup, streamlined
workflows, and an interactive workspace that enables collaboration between data
scientists, data engineers, and business analysts.
, What data can databricks process - answerDatabricks can process data held in many
different types of storage, including Azure Blob storage, Azure Data Lake Store,
Hadoop storage, flat files, databases, and data warehouses.
Can databricks process streaming data - answerDatabricks can also process streaming
data
What architecture does databricks use - answerDatabricks uses an extensible
architecture based on drivers.
What is a driver - answerA driver is a piece of code that connects to a specific data
source and enables you to read and write that source. A driver is typically provided as
part of a library that you can load into the Databricks environment. Drivers are available
for many Azure services, including Azure SQL Database, Azure Cosmos DB, Azure
Blob storage, and Azure Data Lake storage, as well as many services and databases
produced by third-parties, such as MySQL and PostgreSQL.
The databricks processing engine is provided by what - answerThe processing engine
is provided by Apache Spark. Spark is a parallel-processing engine that supports large-
scale analytics. You write application code that consumes data from one or more
sources, and merge, reformat, filter, and remodel this data, and then store the results.
Spark distributes the work across a cluster of computers. Each computer can process
its data in parallel with the other computers. The strategy helps to reduce the time
required to perform the work. Spark is designed to handle massive quantities of data.
What languages can you write spark code in - answerYou can write the Spark
application code using several languages, such as Python, R, Scala, Java, and SQL.
Spark has a number of libraries for these languages, providing complex analytical
routines that have been optimized for the clustered environment. These libraries include
modules for machine learning, statistical analysis, linear and non-linear modeling,
predictive analytics, and graphics.
What is a databricks notebook - answerYou write Databricks applications using a
Notebook. A notebook contains a series of steps (cells), each of which contains a block
of code. For example, one cell might contain the code that connects to a data source,
the next cell reads the data from that source and converts it into a model in-memory, the
next cell plots a graph, and a final cell saves the data from the in-memory model to a
repository. The first line in the cell is %language. For example, %scala.
What is Azure HDInsight - answerAzure HDInsight is a managed analytics service in the
cloud. It's based on Apache Hadoop, a collection of open-source tools and utilities that
enable you to run processing tasks over large amounts of data. HDInsight uses a
clustered model, similar to that of Synapse Analytics. HDInsight stores data using Azure
Data Lake storage. You can use HDInsight to analyze data using frameworks such as
Hadoop Map/Reduce, Apache Spark, Apache Hive, Apache Kafka, Apache Storm, R,
and more.
processing in Azure
What is Azure Synapse Analytics - answer Azure Synapse Analytics is a generalized
analytics service. You can use it to read data from many sources, process this data,
generate various analyses and models, and save the results.
what two technologies can process data for Azure Synapse Analytics - answerTransact-
SQL
Spark
what is Transact-SQL - answerThis is the same dialect of SQL used by Azure SQL
Database, with some extensions for reading data from external sources, such as
databases, files, and Azure Data Lake storage. You can use these extensions to load
data quickly, generate aggregations and other analytics, create tables and views, and
store information using these tables and views. You can use the results for later
reporting and processing.
what is Spark - answerThis is the same open-source technology used to power Azure
Databricks. You write your analytical code using notebooks in a programming language
such as C#, Scala, Python, or SQL. The Spark libraries provided with Azure Synapse
Analytics enable you to read data from external sources, and also write out data in a
variety of different formats if you need to save your results for further analysis.
what clustered architecture can Azure Synapse Analytics use - answerAzure Synapse
Analytics uses a clustered architecture. Each cluster has a control node that is used as
the entry point to the system. When you run Transact-SQL statements or start Spark
jobs from a notebook, the request is sent to the control node.
what is the control node in Azure Synapse Analytics - answerThe control node runs a
parallel processing engine that splits the operation into a set of tasks that can be run
concurrently. Each task performs part of the workload over a subset of the source data.
Each task is sent to a compute node to actually do the processing. The control node
gathers the results from the compute nodes and combines them into an overall result.
What is Azure Databricks - answerAzure Databricks is an analytics platform optimized
for the Microsoft Azure cloud services platform. Designed with the founders of Apache
Spark, Databricks is integrated with Azure to provide one-click setup, streamlined
workflows, and an interactive workspace that enables collaboration between data
scientists, data engineers, and business analysts.
, What data can databricks process - answerDatabricks can process data held in many
different types of storage, including Azure Blob storage, Azure Data Lake Store,
Hadoop storage, flat files, databases, and data warehouses.
Can databricks process streaming data - answerDatabricks can also process streaming
data
What architecture does databricks use - answerDatabricks uses an extensible
architecture based on drivers.
What is a driver - answerA driver is a piece of code that connects to a specific data
source and enables you to read and write that source. A driver is typically provided as
part of a library that you can load into the Databricks environment. Drivers are available
for many Azure services, including Azure SQL Database, Azure Cosmos DB, Azure
Blob storage, and Azure Data Lake storage, as well as many services and databases
produced by third-parties, such as MySQL and PostgreSQL.
The databricks processing engine is provided by what - answerThe processing engine
is provided by Apache Spark. Spark is a parallel-processing engine that supports large-
scale analytics. You write application code that consumes data from one or more
sources, and merge, reformat, filter, and remodel this data, and then store the results.
Spark distributes the work across a cluster of computers. Each computer can process
its data in parallel with the other computers. The strategy helps to reduce the time
required to perform the work. Spark is designed to handle massive quantities of data.
What languages can you write spark code in - answerYou can write the Spark
application code using several languages, such as Python, R, Scala, Java, and SQL.
Spark has a number of libraries for these languages, providing complex analytical
routines that have been optimized for the clustered environment. These libraries include
modules for machine learning, statistical analysis, linear and non-linear modeling,
predictive analytics, and graphics.
What is a databricks notebook - answerYou write Databricks applications using a
Notebook. A notebook contains a series of steps (cells), each of which contains a block
of code. For example, one cell might contain the code that connects to a data source,
the next cell reads the data from that source and converts it into a model in-memory, the
next cell plots a graph, and a final cell saves the data from the in-memory model to a
repository. The first line in the cell is %language. For example, %scala.
What is Azure HDInsight - answerAzure HDInsight is a managed analytics service in the
cloud. It's based on Apache Hadoop, a collection of open-source tools and utilities that
enable you to run processing tasks over large amounts of data. HDInsight uses a
clustered model, similar to that of Synapse Analytics. HDInsight stores data using Azure
Data Lake storage. You can use HDInsight to analyze data using frameworks such as
Hadoop Map/Reduce, Apache Spark, Apache Hive, Apache Kafka, Apache Storm, R,
and more.
