I85PREP
⌘ K
DEA Level 3 study materials
DEA Level 3 study resources

The Lakehouse Migration With a Frozen Reporting Layer

DEA Level 3 · Put your knowledge to work in context.

All case studies

The Lakehouse Migration With a Frozen Reporting Layer

A logistics firm is migrating from an on-premises warehouse to a cloud lakehouse. The legacy warehouse has 1,800 tables, of which 240 feed regulatory filings and 60 feed a customer-facing SLA dashboard with a 15-minute freshness requirement. The migration team proposes a 'lift and shift' of all 1,800 tables with a like-for-like star schema, then optimization later. The data engineering lead warns that 70% of the tables have no documented downstream consumers and that the legacy model has 14 overlapping customer dimensions. The CFO has approved 14 months of funding and expects decommissioning of on-premises hardware at month 12 to hit a savings target. A parallel initiative is introducing a new order-management system in month 8 that changes the order entity's granularity.

1. Evaluate the lift-and-shift proposal and recommend a target architecture approach.

Read the worked answer

Assessment. Lift-and-shift is defensible for the 240 regulatory tables (minimizing behavior change reduces filing risk) but indefensible as a blanket strategy. The proposal ignores three facts in the setup:

  • Undocumented consumers (70%): migrating them preserves unknown dependencies and carries cost with no proven value. Some are almost certainly dead.
  • 14 overlapping customer dimensions: a like-for-like copy perpetuates the exact modeling debt that a migration is the rare opportunity to fix.
  • Month-8 granularity change: any order model copied in months 1–7 will be rebuilt in month 8. Sequencing matters more than speed.

Recommended approach - tiered:

  1. Regulatory tier (240 tables): migrate with strict schema parity, parallel-run and reconcile to source before cutover. Change nothing structural.
  2. SLA dashboard tier (60 tables): re-architect for the 15-minute requirement - streaming or micro-batch ingestion with a serving layer designed for freshness, not a copied star schema.
  3. Long tail (~1,500 tables): usage-instrument the legacy warehouse for 60 days, then migrate only what has demonstrated consumers; retire the rest with owner sign-off.

Sequencing. Defer order-domain modeling until after month 8, or model to the new granularity now and back-fill. Building the old order model first wastes effort.

Risk to the savings target. Month-12 decommissioning is achievable only if the long-tail triage starts immediately; instrumenting for 60 days then migrating leaves a tight but feasible window. Flag to the CFO that decommissioning depends on retiring the tail, not on copying it.

2. How should the team model the customer domain given 14 overlapping dimensions and a hard regulatory dependency?

Read the worked answer

Do not attempt a big-bang single customer dimension. The regulatory tables depend on specific dimension keys; a wholesale replacement in one release would break filings and violate the parallel-run requirement.

Recommended pattern - conformed core with governed variants:

  1. Profile the 14 dimensions and classify each attribute as (a) shared and stable (legal name, tax ID), (b) shared but conflicting (address, status), or (c) context-specific (credit segment, marketing segment). Only (a) and (b) belong in a conformed core.
  2. Build a conformed customer hub with a survivorship ruleset for conflicting attributes, governed by data owners. Keep it additive - new keys mapped to legacy keys via a cross-reference table so regulatory tables continue to resolve their existing keys.
  3. Retain context-specific dimensions as satellites referencing the hub, rather than forcing every attribute into one table. This resolves overlap without destroying legitimate business distinctions.
  4. Migrate regulatory consumers last, after the hub and cross-reference are proven, and only with reconciliation evidence.

Why not a single dimension: collapsing 14 dimensions into one invites survivorship decisions that alter regulatory attributes (e.g., a corrected address changes a filing's entity record). The cross-reference approach gives one governed customer view while preserving the keys regulators and filings depend on. Document the survivorship rules - they are the governance artifact that makes the model auditable.