HCIA-Big Data common mistakes
HCIA-Big Data · Understand the traps and how to avoid them.
When asked where a MapReduce job's shuffle phase runs, the candidate answers 'on the NameNode' or 'on the client', instead of on the map and reduce task nodes (the map output is partitioned, sorted and spilled on mapper nodes and fetched/merged on reducer nodes).
Candidates memorize the HDFS master/slave layout and then reuse 'master node does the coordination work' as a default answer for every component, without separating storage roles (NameNode, DataNode) from compute roles (ResourceManager, NodeManager, task containers).
What to do instead
Build a role table before the exam: NameNode = metadata; DataNode = block storage; ResourceManager = cluster resource scheduling; NodeManager = per-node agent; Mapper/Reducer = task containers. Then trace one record through map -> circular buffer/spill -> partition/sort -> fetch/merge -> reduce, and state explicitly which host performs each step.
The candidate treats an HDFS block as a unit that is always fully written on one DataNode and assumes the default block size is 64 MB, or claims a small file 'wastes no space' because blocks are logical.
Older tutorials and exam dumps still circulate the 64 MB default, and small-file problems are explained vaguely as 'NameNode memory' without the candidate ever learning the mechanism: each file, block and replica consumes NameNode heap objects, so millions of small files exhaust NameNode memory regardless of DataNode disk space.
What to do instead
Memorize the current defaults (128 MB block, 3 replicas) and the reason for them: a block is the unit of storage, replication and scheduling, sized to amortize seek time. For small files, state the mechanism (per-file/per-block metadata in NameNode memory) and the remedies (HAR archives, sequence files, CombineFileInputFormat, or a storage layer such as HBase/object storage).
When asked to choose a component for a scenario, the candidate picks by name familiarity rather than by access pattern: e.g. choosing HBase for a nightly batch aggregation over terabytes of flat logs, or choosing Hive for millisecond point lookups on a single row key.
The exam states scenarios in business language ('real-time query by customer ID', 'daily report'), and candidates map keywords like 'big data' or 'query' to whichever product they studied last, ignoring latency, schema flexibility and update requirements.
What to do instead
Use a decision checklist: Hive = SQL over HDFS, high-latency batch/OLAP, schema-on-read, append-oriented; HBase = column-family NoSQL on HDFS, row-key point get/scan, low latency, random read/write; MapReduce = custom batch ETL; Spark/Flink = in-memory iterative or streaming. For each practice scenario, write the required latency, access key and update pattern first, then select.
The candidate writes a Hive query using a partition column in a way that disables partition pruning, for example wrapping it in a function (WHERE substr(dt,1,7)='2024-01') or filtering with an incompatible type, and still expects only one partition to be scanned.
Candidates learn that 'partitioning improves performance' as a slogan but never learn that pruning works only when the filter is a direct, sargable comparison on the partition column; they also forget that static partition inserts require the partition column to be last in the SELECT list and excluded from the insert column list.
What to do instead
Practice rewriting filters into direct comparisons on partition columns and confirm with EXPLAIN that the partition list is reduced. Memorize the loading rules: static partition INSERT ... PARTITION (dt='...') requires the partition column not appear in the select list; dynamic partition inserts require the partition column to be last and hive.exec.dynamic.partition set appropriately.
The candidate confuses Hive internal (managed) tables with external tables, claiming that DROP TABLE always deletes the underlying HDFS data, or that an external table cannot be partitioned.
The distinction is taught as a definition ('external keeps data') without the operational consequence, and candidates never connect it to the shared-data scenario the exam usually presents (multiple engines reading the same HDFS directory).
What to do instead
Memorize the single decisive difference: DROP TABLE on a managed table removes metadata and data; on an external table it removes only metadata, leaving HDFS files. Both types support partitioning and bucketing. When a scenario mentions data shared with other tools or raw data that must survive, choose EXTERNAL and say why.
The candidate designs an HBase row key that is a monotonically increasing value (timestamp, auto-increment ID, or a sequential order number) and then claims the table will scale linearly across RegionServers.
Relational habits carry over: a sequential primary key is normal in RDBMS, and candidates do not connect HBase's sorted row-key storage to hotspotting, where all new writes land on the single region holding the highest key range.
What to do instead
State the mechanism: rows are sorted by row key and split into regions by key range, so sequential keys concentrate writes on one region. Then apply standard remedies in answers: salted/hashed prefix, reversed key, or a composite key with a high-cardinality prefix, plus pre-splitting when the key distribution is known.
The candidate explains HBase reads and writes as if data goes straight to HDFS or directly to disk, omitting the MemStore and Write-Ahead Log (WAL) on write and the BlockCache plus StoreFile/HFile lookup on read.
Candidates memorize the table -> region -> column family -> store hierarchy as a static diagram and never learn the data path, so they cannot answer questions about durability, flush, compaction or read latency.
What to do instead
Learn one write path and one read path as sequences: write -> WAL append -> MemStore -> flush to HFile when thresholds are reached -> compaction merges HFiles; read -> check MemStore and BlockCache -> then HFiles (Bloom filters and block index reduce I/O). Be able to say which step provides durability (WAL) and which causes read amplification (many small HFiles, fixed by compaction).
The candidate answers questions about Hadoop's replication or fault tolerance by saying 'the NameNode stores a copy of the data' or that a lost block is recovered from the secondary NameNode.
The term 'secondary NameNode' strongly suggests a standby master, and candidates never learn that it only performs checkpointing of the edit log and namespace image; they also blur metadata with data.
What to do instead
Write the correction explicitly: the NameNode holds metadata only; DataNodes hold replicas; block loss is detected by block reports and heartbeat timeouts, and the NameNode schedules re-replication from surviving replicas. The Secondary NameNode/Checkpoint Node merges fsimage and edits to bound restart time and is not a failover node; HA failover needs an Active/Standby pair with shared edits (e.g. JournalNode quorum).
The candidate describes FusionInsight/Huawei big data operations only in generic terms ('install the cluster, monitor it, add nodes') and cannot name the manager component, the service roles it manages, or the routine operational tasks the exam asks about (capacity expansion, alarm handling, tenant/resource management, backup).
Operations content is studied last and skimmed, so candidates retain marketing-level descriptions instead of concrete component names and procedures, and they cannot map a symptom (DataNode disk full, RegionServer down, NameNode in safe mode) to an action.
What to do instead
Build a one-page operations map: cluster manager for installation, configuration, monitoring, alarms and host/role scaling; services such as HDFS, YARN, Hive, HBase, Spark, ZooKeeper with their key roles; and a symptom-to-action table (safe mode -> check block reports and free space; disk imbalance -> rebalance; RegionServer down -> check logs, region reassignment). Practice stating the ordered steps for adding a node and for handling one alarm.
The candidate computes HDFS usable capacity or replication impact incorrectly, for example saying a 10-node cluster with 10 TB disks and 3 replicas stores 100 TB of data, or ignoring that replication factor multiplies raw storage consumption.
Candidates treat 'cluster capacity' as the sum of disks and forget that each block has three replicas, and they also forget reserved space for non-HDFS use and the temporary overhead of re-replication during a node failure.
What to do instead
Use the formula: usable data capacity = raw HDFS disk capacity / replication factor, minus reserved space. For 10 nodes x 10 TB with replication 3, usable is about 33 TB, not 100 TB. Also state the failure case: losing one node temporarily raises the effective replication demand while blocks are re-copied, which is why clusters are not filled to 100%.