AWS Data Engineer study plan

Compare Kinesis Data Streams, Data Firehose, and Amazon MSK on throughput limits, delivery semantics, and consumer models, then select one for each of four scenarios

Build a Firehose delivery stream that buffers, compresses, and dynamically partitions records into S3

Configure a Kinesis Data Streams producer and consumer with enhanced fan-out and validate record ordering

Calculate shard capacity for a 5 MB/s ingest workload and document the resize trigger

Build an AWS Glue ETL job that reads CSV from S3, enforces a schema, and writes partitioned Parquet

Implement a DynamoDB-to-S3 extract with row-count reconciliation against the source table

Add job bookmarks and a repartition step to an existing Glue job and verify incremental runs

Tune a Spark job by adjusting worker sizing, shuffle partitions, and partition pruning, then measure the runtime delta

Select file format, compression codec, and partition scheme for three workload profiles (BI reporting, streaming landing, ML feature prep)

Build a Redshift load using COPY from S3 with a manifest file and error handling

Design idempotent writes and deduplication logic for a re-run scenario

Compact small files produced by a streaming sink into right-sized objects

Design an S3 prefix and partition layout for a 1 TB/day dataset and justify it against three known query patterns

Model a Redshift star schema with distribution keys, sort keys, and compression encodings

Compare Redshift provisioned versus Serverless and Redshift Spectrum versus Athena for four query scenarios

Configure S3 lifecycle rules and storage-class transitions for a tiered retention policy

Register S3 data in the Glue Data Catalog and query it from Athena using partition projection

Configure Lake Formation permissions to restrict access to a single column of a catalog table

Design a DynamoDB table with partition and sort keys plus GSIs for two access patterns, and estimate RCU/WCU

Tune Redshift with VACUUM, ANALYZE, and workload management queues, and capture before/after query timings

Orchestrate a multi-step pipeline with Step Functions or MWAA including retries, backoff, and a failure branch

Instrument the pipeline with CloudWatch metrics, alarms, and a dashboard covering latency and failure rate

Diagnose three injected failures (throttling, schema drift, partition skew) and document root cause and fix for each

Estimate the monthly cost of the pipeline and identify two concrete savings levers

Write least-privilege IAM policies for the ingestion role, ETL role, and analyst role

Enable SSE-KMS with a customer-managed key on S3 and Glue, and constrain the key policy

Apply Lake Formation tag-based access control and column masking to a table containing PII

Enable CloudTrail data events, Macie PII discovery, and catalog audit logging, then verify each produces evidence

Configure cross-account data sharing and validate that the grantee can read only the intended table

Complete two full-length timed practice exams and log every missed question by domain and sub-topic

Re-study the two weakest sub-areas from the error log and re-test them with targeted questions

Redraw the end-to-end reference architecture from memory and defend each service choice

Review the operations runbook and security matrix for gaps against the four domain weightings