• Wrong document? Swap it for free
  • Written by students who passed
  • Immediately available after payment
  • Read online or as PDF
Sell
Where do you study
Your language
Document preview thumbnail
Preview 2 out of 10 pages
Other

Buan 6346 Big Data Study Guide

Document preview thumbnail
Preview 2 out of 10 pages

Buan 6346 Big Data Study Guide

Content preview

BUAN 6346 BIG DATA STUDY GUIDE

Partitioning (data storage) - Answers - Subdividing table storage into HDFS
subdirectories by column value to avoid full scans.

Full table scan - Answers - Default non-partitioned behavior where every file in the
directory is read during analysis.

Partition column - Answers - A virtual column whose value is encoded in the directory
name, not in the data file.

Partition directory naming format - Answers - column=value format, e.g., state=CA or
call_date=2014-10-02.

Dynamic partitioning - Answers - Automatic creation of partitions at load time based on
the last column in SELECT/INSERT.

Static partitioning - Answers - Manually specifying the target partition using ALTER
TABLE ... ADD PARTITION.

hive.exec.dynamic.partition - Answers - Hive property that must be set to true to
enable dynamic partitioning.

hive.exec.dynamic.partition.mode - Answers - Hive property set to nonstrict to allow all
partitions in a query to be dynamic.

hive.exec.max.dynamic.partitions.pernode - Answers - Hive limit on dynamic partitions
created per node in a query; default is 100.

hive.exec.max.dynamic.partitions - Answers - Hive limit on total dynamic partitions
created per HiveQL statement; default is 1000.

hive.exec.max.created.files - Answers - Hive limit on total files created across all
nodes by a query; default is 100000.

MSCK REPAIR TABLE - Answers - Hive command that recovers partition metadata
created outside Hive (e.g., via Spark or MapReduce).

LOAD DATA INPATH ... PARTITION(...) - Answers - Hive command that moves an
HDFS file into the specified partition subdirectory.

Hive-only LOAD shortcut - Answers - Hive automatically creates non-existent partition
metadata and subdirectories on LOAD; Impala requires ADD PARTITION first.

, SHOW PARTITIONS tablename - Answers - Command used to view all existing
partitions on a table.

When NOT to partition - Answers - Avoid partitioning on high-cardinality columns (e.g.,
first name) to prevent creating too many tiny files.

When TO partition - Answers - Partition when full scans are slow, queries frequently
filter on that column, and distinct values are reasonable.

Tools with built-in partition support - Answers - Hive and Impala natively manage
partitions; Sqoop supports partitioned writes via CLI flags.

Apache Flume - Answers - A high-performance system for collecting, aggregating, and
streaming large log and event data into HDFS.

Flume's origin - Answers - Created by Cloudera, donated to Apache in 2011, and
became a top-level project in 2012.

Flume NG vs Flume OG - Answers - NG refers to Next Generation (current
architecture); OG refers to the legacy Original Generation.

Flume's three design goals - Answers - Reliability, Scalability, and Extensibility.

How Flume achieves reliability - Answers - Uses transactional transfers through
Channels, rolling back and retrying on failures.

How Flume achieves scalability - Answers - Scales horizontally by adding more
machines and agents to handle increased load.

How Flume achieves extensibility - Answers - Allows developers to create custom
Sources and Sinks for new storage layers or platforms.

Flume Event - Answers - The basic data unit in Flume, consisting of a payload body
and metadata headers.

Source (Flume component) - Answers - Receives events from external producers and
writes them to one or more Channels.

Sink (Flume component) - Answers - Drains events from a Channel and delivers them
to an external destination.

Channel (Flume component) - Answers - Buffers events between the Source and the
Sink.

Agent (Flume component) - Answers - A JVM process hosting and managing Sources,
Channels, and Sinks.

Document information

Uploaded on
October 24, 2026
Number of pages
10
Written in
2026/2027
Type
Other
Person
Unknown
$14.99

Wrong document? Swap it for free Within 14 days of purchase and before downloading, you can choose a different document. You can simply spend the amount again.
Written by students who passed
Immediately available after payment
Read online or as PDF

Seller avatar
Reputation scores are based on the amount of documents a seller has sold for a fee and the reviews they have received for those documents. There are three levels: Bronze, Silver and Gold. The better the reputation, the more your can rely on the quality of the sellers work.
GEEKA
3.8
(379)
Sold
2239
Followers
1451
Items
63634
Last sold
1 day ago



Why students choose Stuvia

Created by fellow students, verified by reviews

Quality you can trust: written by students who passed their tests and reviewed by others who've used these notes.

Didn't get what you expected? Choose another document

No worries! You can instantly pick a different document that better fits what you're looking for.

Pay as you like, start learning right away

No subscription, no commitments. Pay the way you're used to via credit card and download your PDF document instantly.

Student with book image

“Bought, downloaded, and aced it. It really can be that simple.”

Alisha Student

Working on your references?

Create accurate citations in APA, MLA and Harvard with our free citation generator.

Working on your references?

Frequently asked questions