dbt Project Architecture and Modeling
The project-architecture domain tests whether you can structure a dbt repository that stays maintainable as models and contributors multiply: layer boundaries, DAG shape, materialization choice, incremental logic, snapshots, and the contract and versioning surface that protects downstream consumers.
Layering and DAG hygiene
Declare raw tables as sources in YAML and reference them with source(); reference every model with ref(). That discipline is what lets dbt build the DAG, run state:modified+, defer to another environment, and generate lineage. Staging models do one thing - rename, cast, deduplicate a single source. Put reusable joins in intermediate models, expose entity-grained marts that are documented and tested. In a mesh, groups plus access: private/protected/public stop another domain refing your internals; only public models are refable outside their group.
Materializations
Views cost no storage and keep logic close to source; tables pay compute once for fast repeated reads; incremental models pay per run for large facts. Ephemeral models are inlined as CTEs, so reserve them for small single-use logic - chained ephemeral models inflate compiled SQL and are hard to debug. Set defaults per folder in dbt_project.yml.
Incremental correctness
Guard with is_incremental() so first runs and full refreshes build everything. Pick a unique_key genuinely unique at the target grain, choose the adapter strategy deliberately (merge, delete+insert, insert_overwrite, append, microbatch), and add a lookback window so late records are recaptured. On BigQuery and Databricks, filter the source on the partition column and declare incremental_predicates so the warehouse prunes partitions instead of scanning.
Contracts, versions, snapshots
Enforced contracts require every column to declare name and data_type. Adding a column is non-breaking; renaming or retyping breaks consumers, so publish a new versions entry with latest_version and deprecation_date. Snapshots capture history: timestamp strategy needs an updated_at column, check strategy needs check_cols, and invalidate_hard_deletes handles deleted source rows.
Worked example
A 400M-row fct_orders needs daily refresh and a stable interface. Declare the source with freshness, build stg_orders as a view with casts, and make fct_orders incremental (unique_key='order_id', incremental_strategy='merge', on_schema_change='append_new_columns'), filtering the source on _loaded_at >= (select max(_loaded_at) - interval '3 days' from {{ this }}) inside the is_incremental() branch. Test order_id for uniqueness, enforce a contract, and let consumers use ref('fct_orders') so a rename fails at compile time.
Traps
- Refing raw tables by name, hiding lineage and skipping source freshness.
- Ephemeral chains or view-on-view stacks re-scanning the same table.
- A
unique_keynot unique at the model grain, producing duplicates inside a merge. - Changing a contracted column type without a version bump.