I85PREP
⌘ K
HCIA-Big Data study materials
HCIA-Big Data study resources

HCIA-Big Data practice questions

One question at a time. Build your HCIA-Big Data understanding.

First 50 of 200 questions. Each one lists the choices, the correct answer, and an explanation. The checker at the top of this page saves progress in this browser when it is available.

Continue in the practice tool above to load more questions, 50 at a time.

Question 1Big Data overview and ecosystem

In the Hadoop ecosystem, which component is primarily responsible for reliable distributed storage of very large files across a cluster of commodity machines?

  1. MapReduce
  2. Hive
  3. YARN
  4. HDFS Correct answer.

Explanation

HDFS (Hadoop Distributed File System) is the storage layer of Hadoop. It splits files into blocks, replicates them across DataNodes, and relies on a NameNode to track metadata, providing fault-tolerant storage on commodity hardware.

Question 2Big Data overview and ecosystem

Which Hadoop component acts as the cluster resource manager that allocates CPU and memory containers to distributed applications?

  1. HBase RegionServer
  2. YARN ResourceManager Correct answer.
  3. HDFS NameNode
  4. ZooKeeper

Explanation

YARN separates resource management from computation. The ResourceManager is the global authority that tracks cluster capacity and hands out containers to application masters, which then run tasks on NodeManagers.

Question 3Big Data overview and ecosystem

A data engineer must run an iterative machine-learning algorithm that repeatedly reads the same dataset into memory across many passes. Which execution engine design best suits this requirement?

  1. A disk-based two-stage MapReduce pipeline
  2. An in-memory cluster computing engine such as Spark Correct answer.
  3. A message queue such as Kafka
  4. A columnar warehouse such as Hive on Tez

Explanation

Spark caches intermediate data in memory across iterations, which avoids re-reading from disk on every pass. That characteristic makes it well suited to iterative algorithms such as gradient descent and graph processing.

Question 4Big Data overview and ecosystem

What is the primary role of the NameNode in an HDFS cluster?

  1. Serving SQL queries over Hive tables
  2. Shuffling intermediate key-value pairs between map and reduce
  3. Maintaining file-system metadata and block-location mappings Correct answer.
  4. Storing the actual data blocks on local disks

Explanation

The NameNode is the master that holds the namespace and the mapping of blocks to DataNodes. It does not stream file data itself; clients ask it where blocks live and then read directly from DataNodes.

Question 5Big Data overview and ecosystem

HDFS is described as write-once, read-many. Which workload pattern is therefore a poor fit for plain HDFS?

  1. Random in-place updates to individual records of an existing file Correct answer.
  2. Appending large log files once and scanning them later
  3. Batch scanning of terabytes of historical clickstream data
  4. Storing large immutable archive files

Explanation

HDFS is optimized for streaming reads of large files and does not support efficient random in-place modification of records. Systems needing frequent record-level updates typically use a store such as HBase on top of HDFS.

Question 6Big Data overview and ecosystem

In MapReduce, what does the shuffle phase accomplish?

  1. It groups and transfers map output by key so each reducer receives all values for a key Correct answer.
  2. It replicates HDFS blocks to a second rack for durability
  3. It compiles user code into bytecode before execution
  4. It compresses the final output file written to HDFS

Explanation

The shuffle partitions map output by key and moves it across the network so that all values sharing a key reach the same reducer. Sorting and merging during this phase make the reducer input ordered by key.

Question 7Big Data overview and ecosystem

Which statement best describes the relationship between Hadoop and the broader big data ecosystem?

  1. Hadoop is a single monolithic product that contains every big data tool
  2. Hadoop provides core storage and resource management on which many separate projects build Correct answer.
  3. Hadoop replaces the need for any streaming or SQL engine
  4. Hadoop is a query language used to access NoSQL databases

Explanation

Hadoop supplies foundational pieces, chiefly HDFS for storage and YARN for resource management. Projects such as Hive, HBase, Spark, and Flink integrate with or run on these foundations rather than being part of one monolith.

Question 8Big Data overview and ecosystem

A company needs low-latency random reads and writes on billions of rows keyed by user ID, while keeping the data on HDFS. Which component is the most appropriate choice?

  1. Sqoop
  2. Flume
  3. Hive
  4. HBase Correct answer.

Explanation

HBase is a distributed, column-family NoSQL store built on HDFS that supports random real-time reads and writes by row key. Its design targets large sparse tables with fast key-based access.

Question 9Big Data overview and ecosystem

Which tool is designed to move bulk data between a relational database and HDFS using MapReduce-based parallel transfers?

  1. Flume
  2. Sqoop Correct answer.
  3. Oozie
  4. Kafka

Explanation

Sqoop imports and exports structured data between relational databases and Hadoop, splitting work into parallel map tasks. It uses JDBC connectivity to read from or write to the source system.

Question 10Big Data overview and ecosystem

A team wants to ingest continuously generated web server logs into HDFS with minimal custom code. Which component is purpose-built for this?

  1. Sqoop
  2. Flume Correct answer.
  3. Pig
  4. Hive

Explanation

Flume is an agent-based data collection system for streaming event and log data into sinks such as HDFS. Its source-channel-sink model lets it tail logs and buffer events reliably before writing them.

Question 11Big Data overview and ecosystem

In the Hadoop ecosystem, what is the main purpose of ZooKeeper?

  1. Distributed coordination such as configuration management, naming, and leader election Correct answer.
  2. Distributed storage of large binary files
  3. Real-time stream processing of events
  4. Batch SQL query execution

Explanation

ZooKeeper offers a consistent, highly available coordination service. Distributed systems use it for configuration, naming, synchronization, and leader election so they do not each reimplement these primitives.

Question 12Big Data overview and ecosystem

Which statement correctly distinguishes Hive from HBase?

  1. Hive is a NoSQL store, while HBase is a SQL warehouse
  2. Hive provides SQL-like batch analytics over stored data, while HBase provides low-latency key-based access Correct answer.
  3. Hive replaces HDFS, while HBase replaces YARN
  4. Hive stores data in memory, while HBase always writes to local disk only

Explanation

Hive maps structured files to tables and translates SQL-like queries into batch jobs, favoring throughput over latency. HBase is a NoSQL store optimized for fast random access by row key, so the two serve different access patterns.

Question 13Big Data overview and ecosystem

What does the replication factor setting in HDFS control?

  1. The number of copies of each block maintained across DataNodes Correct answer.
  2. The number of ZooKeeper ensemble members
  3. The number of reducers launched per job
  4. The number of partitions in a Hive table

Explanation

The replication factor determines how many copies of each block HDFS keeps, typically three by default. More replicas increase durability and read parallelism at the cost of storage.

Question 14Big Data overview and ecosystem

A cluster has a replication factor of three and one DataNode holding a block replica fails permanently. What does HDFS do to restore the intended replication level?

  1. It blocks all reads until an administrator manually copies the block
  2. It creates a new replica from an existing copy on another DataNode Correct answer.
  3. It reduces the replication factor of that file to two
  4. It restores the lost block from the NameNode's edit log

Explanation

The NameNode detects that a block is under-replicated and schedules replication from a surviving replica to another DataNode. Reads continue uninterrupted because other copies remain available.

Question 15Big Data overview and ecosystem

Which description best captures the difference between batch and stream processing in the big data ecosystem?

  1. Batch processing requires no storage, while stream processing always stores all data first
  2. Batch processes bounded stored data in scheduled jobs, while stream processing handles unbounded data continuously as it arrives Correct answer.
  3. Stream processing can only run on a single machine, while batch processing is always distributed
  4. Batch and stream processing are identical except for the programming language used

Explanation

Batch engines operate over finite datasets, often on a schedule, and prioritize throughput. Stream engines treat data as an unbounded flow and process each event with low latency as it arrives.

Question 16Big Data overview and ecosystem

What is the function of the Secondary NameNode in HDFS?

  1. It acts as a hot standby that takes over immediately when the NameNode fails
  2. It periodically merges the edit log with the fsimage to keep NameNode startup fast Correct answer.
  3. It stores a second replica of every block
  4. It schedules MapReduce tasks on DataNodes

Explanation

The Secondary NameNode performs checkpointing: it merges the namespace image with the edit log and returns a compacted image, reducing NameNode restart time. It is not a failover standby.

Question 17Big Data overview and ecosystem

A user launches a MapReduce job and observes that all map tasks finish quickly but reducers run for a very long time. Which condition most likely explains this?

  1. Too many DataNodes in the cluster
  2. Data skew causing a few keys to dominate reducer input Correct answer.
  3. The NameNode running in safe mode
  4. A missing HDFS replication factor

Explanation

Reducer duration is often dominated by the largest key group, since a single reducer must process all values for a key. Skewed key distribution concentrates work in a few reducers while others finish early.

Question 18Big Data overview and ecosystem

Which component in the Hadoop ecosystem provides workflow scheduling and orchestration of dependent jobs?

  1. HBase
  2. Pig
  3. Flume
  4. Oozie Correct answer.

Explanation

Oozie defines and runs workflows of dependent actions, such as chained MapReduce, Hive, and Sqoop jobs. It triggers actions based on completion and supports time- or data-based coordination.

Question 19Big Data overview and ecosystem

Why does HDFS typically place the first block replica on the local node when a client writes from within the cluster?

  1. To bypass the DataNode pipeline entirely
  2. To reduce network traffic and write latency for the initial replica Correct answer.
  3. To avoid storing metadata in the NameNode
  4. To guarantee the block is never replicated again

Explanation

Writing the first replica locally avoids sending that copy over the network and shortens the write path. The remaining replicas are then placed on other nodes, often across racks, for durability.

Question 20Big Data overview and ecosystem

Which statement about YARN's NodeManager is accurate?

  1. It stores the HDFS namespace metadata
  2. It is the SQL gateway for Hive queries
  3. It performs leader election for the cluster
  4. It runs on each worker node and launches containers on behalf of the ResourceManager Correct answer.

Explanation

A NodeManager agent runs on every worker node, monitors local resources, and starts containers as instructed. It reports to the ResourceManager and manages the lifecycle of tasks on its node.

Question 21Big Data overview and ecosystem

A team stores small files of a few kilobytes each in HDFS and notices poor NameNode scalability. What is the underlying reason?

  1. Small files cannot be replicated across DataNodes
  2. Each file, block, and directory consumes NameNode memory for metadata Correct answer.
  3. The Secondary NameNode refuses to checkpoint small files
  4. Small files bypass the DataNode write pipeline

Explanation

The NameNode keeps metadata for every file, directory, and block in memory, so many tiny files create a large object count and strain heap capacity. Consolidating small files into larger containers reduces this metadata pressure.

Question 22Big Data overview and ecosystem

Which ecosystem component is best described as a distributed publish-subscribe messaging system for high-throughput event streams?

  1. Kafka Correct answer.
  2. Oozie
  3. Sqoop
  4. Hive

Explanation

Kafka is a distributed log that lets producers publish records to topics and consumers subscribe to them at high throughput. It decouples data producers from downstream processing systems.

Question 23Big Data overview and ecosystem

What does Hive's metastore primarily store?

  1. The HDFS block checksums
  2. The MapReduce job history logs
  3. The actual row data of every table
  4. The table definitions, partitions, and schema information Correct answer.

Explanation

The Hive metastore is a repository of metadata describing databases, tables, columns, and partitions. Query engines consult it to resolve table names and locate the underlying HDFS paths.

Question 24Big Data overview and ecosystem

A candidate compares Pig and Hive. Which statement is accurate?

  1. Pig replaces HDFS, while Hive replaces ZooKeeper
  2. Pig is a NoSQL store, while Hive is a message queue
  3. Pig provides low-latency random access, while Hive provides only streaming ingestion
  4. Pig offers a data-flow scripting language, while Hive offers a SQL-like declarative language Correct answer.

Explanation

Pig Latin expresses transformations as a procedural data-flow script, whereas HiveQL is a declarative SQL-like language. Both ultimately compile work into distributed execution jobs.

Question 25Big Data overview and ecosystem

Why is data locality a key design principle in Hadoop's distributed processing model?

  1. It guarantees that no data is ever replicated
  2. It eliminates the need for a resource manager
  3. Moving computation to where data resides avoids costly network transfer of large inputs Correct answer.
  4. It ensures every job runs on a single node

Explanation

Large datasets are expensive to move across the network, so Hadoop schedules tasks on nodes that already hold the needed blocks. When locality cannot be satisfied, the framework falls back to nearby nodes to limit transfer.

Question 26Big Data overview and ecosystem

Which characteristic of big data refers to the speed at which data is generated and must be processed?

  1. Veracity
  2. Volume
  3. Velocity Correct answer.
  4. Variety

Explanation

Velocity describes how fast data arrives and how quickly it must be ingested and analyzed, such as continuous clickstreams or sensor readings. Volume concerns the sheer size of the data, while variety concerns the mix of formats and veracity concerns trustworthiness. These are distinct dimensions, so only velocity matches the notion of generation and processing speed.

Question 27Big Data overview and ecosystem

Which data type is most naturally handled by a distributed file system such as HDFS rather than a traditional relational database?

  1. Short-lived session state accessed with millisecond latency
  2. Small configuration tables joined frequently
  3. Unstructured log files written once and read many times Correct answer.
  4. Highly normalized transactional records updated row by row

Explanation

HDFS is optimized for append-only, write-once-read-many access to very large files, which suits raw unstructured logs. Relational databases are better suited to frequent row-level updates, small joined tables, and low-latency random access. Choosing HDFS for transactional updates or millisecond session lookups would conflict with its design goals.

Question 28Big Data overview and ecosystem

In the Hadoop ecosystem, which component is a column-oriented NoSQL store built on HDFS and intended for random read and write access to very large tables?

  1. HBase Correct answer.
  2. Sqoop
  3. Flume
  4. Hive

Explanation

HBase is a distributed, column-family-oriented NoSQL database that runs on HDFS and supports random reads and writes at scale by key. Hive provides SQL-like querying over batch data rather than low-latency random access. Sqoop and Flume are data movement tools, not storage engines.

Question 29Big Data overview and ecosystem

Which statement describes the role of Hadoop Common in the Apache Hadoop project?

  1. It stores file metadata and namespace information
  2. It performs parallel data processing with map and reduce tasks
  3. It supplies shared libraries and utilities used by the other Hadoop modules Correct answer.
  4. It schedules containers across cluster nodes

Explanation

Hadoop Common contains the shared libraries, configuration utilities, and abstractions that the other Hadoop modules depend on. Container scheduling belongs to YARN, namespace metadata to HDFS, and parallel processing to MapReduce. It is therefore the common substrate rather than a processing or storage engine.

Question 30Big Data overview and ecosystem

A team must run SQL-like queries over petabytes of existing HDFS files without writing Java programs. Which ecosystem component best fits this requirement?

  1. ZooKeeper
  2. Oozie
  3. Kafka
  4. Hive Correct answer.

Explanation

Hive provides a SQL-like interface that compiles queries into distributed jobs over data stored in HDFS, so analysts can query without writing Java. Kafka is a messaging system, ZooKeeper provides coordination, and Oozie orchestrates workflows. Only Hive offers declarative query over existing HDFS data.

Question 31Big Data overview and ecosystem

Which statement accurately describes how YARN separates cluster responsibilities?

  1. The NodeManager allocates cluster resources while the ResourceManager executes tasks
  2. The ResourceManager allocates cluster resources while per-application ApplicationMasters negotiate containers Correct answer.
  3. The ResourceManager runs tasks while the NameNode allocates containers
  4. The JobTracker schedules resources and the TaskTracker stores block replicas

Explanation

In YARN, the ResourceManager arbitrates cluster-wide resources, and each application has an ApplicationMaster that requests containers and coordinates its own execution. NodeManagers run on worker nodes and manage containers locally. This split removes the single JobTracker bottleneck of the earlier MapReduce architecture.

Question 32Big Data overview and ecosystem

An engineer wants to copy a large table from a MySQL database into HDFS in parallel using generated MapReduce jobs. Which tool is appropriate?

  1. Flume
  2. Pig
  3. Sqoop Correct answer.
  4. HBase

Explanation

Sqoop is designed to transfer bulk data between relational databases and Hadoop, splitting the import into parallel map tasks. Flume targets streaming log and event ingestion rather than relational tables. HBase is a store and Pig is a data-flow scripting language, so neither performs relational imports.

Question 33Big Data overview and ecosystem

Which statement best explains why HDFS tolerates the loss of a DataNode without data loss?

  1. The NameNode stores a full copy of every block in memory
  2. The Secondary NameNode restores lost blocks from its checkpoint
  3. Blocks are compressed so that corruption cannot occur
  4. Each block is replicated to multiple DataNodes according to the replication factor Correct answer.

Explanation

HDFS divides files into blocks and stores multiple replicas of each block on different DataNodes, so losing one node leaves other replicas available. The NameNode holds metadata rather than block data, and the Secondary NameNode only merges edit logs into checkpoints. Compression does not provide fault tolerance.

Question 34Big Data overview and ecosystem

A pipeline must collect application log lines from hundreds of servers and deliver them reliably into HDFS with configurable channels and sinks. Which component is designed for this?

  1. Oozie
  2. Hive
  3. Mahout
  4. Flume Correct answer.

Explanation

Flume is an agent-based data collection system whose sources, channels, and sinks move streaming log data reliably into destinations such as HDFS. Oozie coordinates workflows, Hive queries stored data, and Mahout provides machine learning algorithms. Only Flume is built around reliable event collection and delivery.

Question 35Big Data overview and ecosystem

Which statement correctly distinguishes Pig from MapReduce?

  1. Pig provides a SQL engine with a mandatory metastore
  2. Pig executes only on a single machine and cannot use a cluster
  3. Pig expresses data flows in a scripting language that compiles into MapReduce or other execution jobs Correct answer.
  4. Pig replaces HDFS as the storage layer for processed data

Explanation

Pig Latin is a data-flow language whose scripts are compiled into execution plans, typically MapReduce or other engines, so developers avoid writing raw map and reduce code. Pig does not replace HDFS storage and does not require a metastore, which is a Hive trait. It is designed for distributed cluster execution.

Question 36Big Data overview and ecosystem

A data engineer needs a single SQL engine that can query data stored in HDFS, HBase, and external relational systems through one interface. Which component best fits?

  1. Hive with its metastore and storage handlers Correct answer.
  2. Flume with its channel configuration
  3. YARN with its capacity scheduler
  4. ZooKeeper with its znodes

Explanation

Hive separates its metastore, which describes tables and schemas, from storage handlers that can point at HDFS, HBase, or external databases, giving one SQL interface across sources. Flume is an ingestion agent, ZooKeeper is a coordination service, and YARN is a resource manager. None of the others provides a unified query layer.

Question 37Big Data overview and ecosystem

Which scenario is the strongest reason to choose a stream processing engine instead of a batch engine?

  1. A nightly job must join two large historical datasets
  2. Monthly financial reports must aggregate archived records
  3. Alerts must fire within seconds of a fraudulent transaction occurring Correct answer.
  4. A one-time migration must rewrite historical files into a new format

Explanation

Stream processing handles unbounded data continuously, so events can be analyzed as they arrive and alerts raised within seconds. Monthly reports, one-time migrations, and nightly joins operate on bounded historical data and are naturally batch workloads. Latency requirements, not data size alone, drive the choice.

Question 38Big Data overview and ecosystem

A cluster writes a 300 MB file with a block size of 128 MB. How many blocks does HDFS create for this file, ignoring replication?

  1. 2
  2. 4
  3. 3 Correct answer.
  4. 1

Explanation

HDFS splits a file into full blocks plus a final partial block, so 300 MB yields two full 128 MB blocks and one remaining 44 MB block. The remainder still occupies a block, giving three blocks in total. Replication affects the number of stored copies, not the block count.

Question 39Big Data overview and ecosystem

A NameNode has been running for months while edit logs keep growing. Why does the cluster rely on checkpointing rather than replaying the entire edit log at every restart?

  1. Checkpointing compacts the namespace image so startup replays only recent edits Correct answer.
  2. Checkpointing removes the need for DataNodes to send block reports
  3. Checkpointing copies data blocks to a standby node for faster reads
  4. Checkpointing converts HDFS files into Hive tables automatically

Explanation

The NameNode persists namespace changes in an edit log, and replaying a very long log at startup would be slow. Checkpointing merges the fsimage with accumulated edits into a compact new image, so only edits after the checkpoint must be replayed. This shortens recovery time without changing block placement or block reporting.

Question 40Big Data overview and ecosystem

A MapReduce job reads a 1 GB input split stored on a remote DataNode because no local replica is available. What is the most direct performance consequence?

  1. The NameNode refuses to schedule the task on that node
  2. Map tasks must transfer input over the network, reducing throughput compared with local reads Correct answer.
  3. The job fails because MapReduce requires strict data locality
  4. The replication factor is automatically increased to restore locality

Explanation

Hadoop prefers to schedule map tasks where a replica resides so reads come from local disk. When locality cannot be satisfied, the framework still runs the task but streams the split across the network, consuming bandwidth and slowing throughput. A missing local replica degrades performance rather than causing failure.

Question 41Hadoop and MapReduce fundamentals

In HDFS, which process on the NameNode is responsible for periodically merging the edit log into the fsimage so that the edit log does not grow without bound?

  1. Heartbeat processing performed by the JobTracker
  2. Block replication performed by the DataNodes
  3. Shuffle and sort performed by the MapReduce framework
  4. Checkpointing performed by the SecondaryNameNode or a Standby NameNode Correct answer.

Explanation

The SecondaryNameNode (or the Standby NameNode in an HA cluster) periodically downloads the fsimage and edit log, merges them into a new fsimage, and returns it to the active NameNode. This checkpointing bounds the size of the edit log and shortens restart time. Block replication and heartbeats are unrelated data-plane and liveness functions.

Question 42Hadoop and MapReduce fundamentals

A file of 300 MB is stored in HDFS with a block size of 128 MB and a replication factor of 3. How many block replicas does HDFS store in total for this file?

  1. 12
  2. 9 Correct answer.
  3. 6
  4. 3

Explanation

The file occupies three blocks (128 MB + 128 MB + 44 MB). With a replication factor of 3, each block has three replicas, so the total is 3 blocks x 3 replicas = 9 replicas. The partial final block is still replicated three times.

Question 43Hadoop and MapReduce fundamentals

What is the primary role of the DataNode in an HDFS cluster?

  1. Scheduling MapReduce tasks across worker nodes
  2. Storing and serving block replicas and reporting block lists to the NameNode Correct answer.
  3. Merging the edit log with the fsimage
  4. Resolving the mapping of file paths to inodes for clients

Explanation

DataNodes hold the actual block replicas on local disks, serve read and write requests from clients, and send periodic block reports and heartbeats to the NameNode. They do not manage namespace metadata or schedule jobs.

Question 44Hadoop and MapReduce fundamentals

Which HDFS component stores the namespace metadata (the directory tree and file-to-block mapping) in memory to serve client requests quickly?

  1. The NodeManager
  2. The DataNode
  3. The NameNode Correct answer.
  4. The JournalNode

Explanation

The NameNode keeps the entire namespace image in memory, which is why HDFS is limited by NameNode memory rather than by total cluster disk capacity. DataNodes store blocks, NodeManagers run containers, and JournalNodes share the edit log in HA deployments.

Question 45Hadoop and MapReduce fundamentals

In the MapReduce programming model, what does a Combiner do?

  1. It partitions the final output into a fixed number of reduce files
  2. It sorts the intermediate keys on the reduce side
  3. It merges the outputs of several mappers on the map side before they are sent across the network Correct answer.
  4. It splits the input file into logical input splits

Explanation

A Combiner performs a local partial aggregation of mapper output on the map node, reducing the volume of data shuffled to reducers. It is only safe when the aggregation is commutative and associative, such as sum or max. Partitioning, reduce-side sorting, and input splitting are separate framework functions.

Question 46Hadoop and MapReduce fundamentals

Which YARN component negotiates cluster resources and launches the per-application ApplicationMaster?

  1. The NodeManager
  2. The ResourceManager Correct answer.
  3. The JobHistoryServer
  4. The DataNode

Explanation

The ResourceManager is the cluster-level authority that tracks available resources and, on application submission, allocates a container in which the ApplicationMaster starts. The ApplicationMaster then negotiates further containers with the ResourceManager through the scheduler.

Question 47Hadoop and MapReduce fundamentals

Which file in a Hadoop configuration directory is read to set cluster-wide properties such as fs.defaultFS?

  1. mapred-site.xml
  2. core-site.xml Correct answer.
  3. yarn-site.xml
  4. hdfs-site.xml

Explanation

core-site.xml holds common Hadoop properties, including fs.defaultFS, which defines the default filesystem URI. hdfs-site.xml covers HDFS-specific settings such as replication, while mapred-site.xml and yarn-site.xml configure the MapReduce and YARN frameworks.

Question 48Hadoop and MapReduce fundamentals

A developer needs to run a word count job on data already stored in HDFS without writing custom Java code. Which built-in command is appropriate?

  1. hadoop fs -count /input
  2. hdfs dfsadmin -report
  3. yarn application -list
  4. hadoop jar hadoop-mapreduce-examples.jar wordcount /input /output Correct answer.

Explanation

The hadoop-mapreduce-examples jar bundles a ready-made wordcount program that can be run with hadoop jar, taking an HDFS input path and output path. The other commands inspect file counts, cluster status, or running applications but do not run an aggregation job.

Question 49Hadoop and MapReduce fundamentals

During the MapReduce shuffle, how is the intermediate key-value pair (k, v) routed to a reducer?

  1. The key is written to the NameNode, which assigns a reducer
  2. The pair is broadcast to every reducer for filtering
  3. The value is compared against a threshold set in mapred-site.xml
  4. The key is hashed and reduced modulo the number of reduce tasks by the Partitioner Correct answer.

Explanation

The default HashPartitioner computes key.hashCode() modulo the number of reduce tasks, so all values for the same key reach the same reducer. This guarantees that a reducer sees a complete key group. Broadcasting would duplicate work and destroy the aggregation semantics.

Question 50Hadoop and MapReduce fundamentals

An input file is 1 GB and the default split size is 128 MB. Approximately how many map tasks will be created, assuming the file is splittable and stored as HDFS blocks of 128 MB?

  1. 8 Correct answer.
  2. 4
  3. 1
  4. 16

Explanation

Splits are aligned with block boundaries, so a 1 GB file (1024 MB) yields about 1024/128 = 8 splits, and each split normally becomes one map task. The number of map tasks therefore follows the number of input splits, not the file count.