HCIP-Big Data Developer study materials
HCIP-Big Data Developer study resources

HCIP-Big Data Developer study plan

HCIP-Big Data Developer · Give each week a clear focus.

8 weeks · About 7 hours per week

Week 1

Big Data Architecture and Huawei FusionInsight Overview & Hadoop Distributed File System (HDFS) Design and Development

  • Map every FusionInsight component (HDFS, YARN, Hive, HBase, Spark, Flink, Kafka, Loader, Flume, Sqoop, ZooKeeper, Manager) to its role, so you can state what each one stores or schedules without notes

  • Describe the HDFS write pipeline and read path end to end, naming NameNode, DataNode, JournalNode, and client responsibilities at each step

  • Configure replication factor, block size, and rack awareness for a stated workload, and justify each value you choose

  • Diagnose three HDFS failure scenarios (DataNode loss, NameNode failover, full disk) and state the recovery behavior

Your checkpoint

A one-page architecture diagram plus a 15-row component table you can reproduce from memory, scored against your own notes.

Week 2

Hadoop Distributed File System (HDFS) Design and Development & MapReduce and YARN Application Development

  • Write a Java or Scala HDFS client program that creates, appends, lists, and deletes files, and handles permissions errors explicitly

  • Implement a MapReduce job with a custom Writable, a Combiner, and a Partitioner, then explain how each changes shuffle volume

  • Configure a YARN application with memory, vCore, and queue settings, and predict what happens when the container request exceeds the queue capacity

  • Trace one job through ResourceManager, NodeManager, ApplicationMaster, and containers, and identify where a hang would surface

Your checkpoint

A working MapReduce job plus a written shuffle trace showing input splits, map output, partition counts, and reduce input.

Week 3

Hive Data Warehouse and SQL-on-Big-Data Development & HBase Distributed Database Design and Development

  • Model a Hive warehouse with internal and external tables, partitions, and buckets, and state when each storage choice is correct

  • Write HiveQL covering joins, window functions, lateral views, and dynamic partitioning, and explain the execution plan for one of them

  • Design an HBase rowkey for a stated access pattern that avoids hot spots, and defend the salt or hash prefix you chose

  • Build an HBase table with column families sized for the read/write mix, and explain how MemStore, BlockCache, and compaction affect latency

  • Compare Hive-on-HBase, Phoenix-style access, and direct HBase API access for one query requirement

Your checkpoint

A schema document containing one Hive DDL set with partitions and one HBase rowkey design with a written hot-spot analysis.

Week 4

Spark Core and Spark SQL Development

  • Write Spark transformations and actions that avoid unnecessary shuffles, and identify wide dependencies in your own code

  • Persist a DataFrame at the right storage level and measure or reason about the effect on a repeated aggregation

  • Explain Spark's memory model, executor and driver sizing, and how a skewed key produces a straggler task

  • Rewrite a Spark SQL query using broadcast join hints and partition pruning, and state the expected plan change

Your checkpoint

Two Spark jobs on the same data, one naive and one tuned, with a written comparison of stages, shuffle bytes, and runtime reasoning.

Week 5

Flink Stream Processing and Real-Time Development & Kafka and Data Ingestion / Integration (Flume, Loader, Sqoop)

  • Build a Flink DataStream pipeline with event time, watermarks, and a windowed aggregation, and explain late-event handling

  • Choose between keyed state and operator state for a stated use case, and configure checkpointing and restart strategy

  • Define a Kafka topic with partitions and replication suited to a stated throughput, and explain consumer group rebalancing

  • Assemble an ingestion path with Flume or Sqoop or Loader into HDFS or Hive, and state where duplicates or partial files can appear

  • Explain exactly-once versus at-least-once across the Kafka-to-Flink boundary and what each requires

Your checkpoint

A running Flink job consuming from Kafka plus a diagram of the full ingestion path with delivery-guarantee annotations at each hop.

Week 6

Spark Core and Spark SQL Development & Flink Stream Processing and Real-Time Development

  • Compare Spark Structured Streaming and Flink for one stated latency and correctness requirement, and justify a choice

  • Tune a streaming job's parallelism, checkpoint interval, and state backend, and predict the effect of each change

  • Convert one batch Spark job into a streaming equivalent and identify the state you must carry

  • Reproduce and fix a backpressure or checkpoint-timeout symptom in a streaming pipeline

Your checkpoint

A decision memo comparing Spark and Flink for one scenario, plus a tuned streaming job with a before-and-after configuration table.

Week 7

Big Data Security, Permissions, and Multi-Tenancy & Performance Tuning, Troubleshooting, and Operations for Big Data Applications

  • Configure HDFS and Hive permissions with roles, ACLs, and Ranger-style policies, and verify a denied access produces the expected error

  • Design a multi-tenant layout with queues, directories, and resource limits, and state what isolates tenants from each other

  • Diagnose a slow job using stage timing, spill metrics, GC behavior, and data skew evidence, and name the fix for each cause

  • Explain how to read a NameNode heap problem, a small-files problem, and a YARN queue saturation problem from symptoms alone

Your checkpoint

A troubleshooting runbook covering five symptoms with cause, evidence to collect, and remediation for each.

Week 8

Big Data Architecture and Huawei FusionInsight Overview & Hadoop Distributed File System (HDFS) Design and Development & MapReduce and YARN Application Development & Hive Data Warehouse and SQL-on-Big-Data Development & HBase Distributed Database Design and Development & Spark Core and Spark SQL Development & Flink Stream Processing and Real-Time Development & Kafka and Data Ingestion / Integration (Flume, Loader, Sqoop) & Big Data Security, Permissions, and Multi-Tenancy & Performance Tuning, Troubleshooting, and Operations for Big Data Applications

  • Answer timed practice sets weighted by domain percentage, and log every miss by domain and cause

  • Re-derive the weakest three domain summaries from memory without opening your notes

  • Redo every missed item and state why the correct answer beats each distractor

  • Review the security, tuning, and ingestion domains last, since they are the smallest and easiest to let slip

Your checkpoint

A scored practice log with per-domain accuracy and a one-page final review sheet of your ten most-missed facts.

Milestones

Week 2: You can draw the HDFS write pipeline and explain a MapReduce shuffle from memory, and your client program runs without reference material.

Week 4: You can size a Spark job and explain why one version shuffles less than another, with evidence from your own runs.

Week 6: You can defend a Spark-versus-Flink choice for a concrete scenario and describe the delivery guarantee your pipeline actually provides.

Week 8: Practice accuracy is at or above your target in every domain, and no domain sits below 60 percent.