Course · Distributed processing
dbt for Data Engineers
dbt turns SQL select statements into tested, documented warehouse models. Learn models, materializations, incremental models, tests, snapshots and CI.
- Lessons
- 9
- Interview questions
- 0
- Projects & case studies
- 6
- Reading time
- ~4 h
Your progress
Saved in this browser onlyCourse structure
- 1 lessonStart hereThe complete overview of the course in one read.
- 2 lessonsBeginnerCore concepts you will use every day.
- 4 lessonsIntermediatePatterns used in production pipelines.
- 1 lessonAdvancedPerformance, internals and edge cases.
- 1 lessonInterview preparationThe top interview questions, with answers.
Practise
Lessons
Work through the lessons in order. Completed lessons show a tick; lessons you have opened are outlined.
Start here
The complete overview of the course in one read.
Beginner
Core concepts you will use every day.
- dbt Models, ref() and Sources: Staging, Intermediate and Marts Layersdbt models, ref() and sources explained: declare raw tables, build staging, intermediate and mart layers, name models well and use seeds for lookups.
- dbt Materializations: View, Table, Ephemeral, Incremental and Materialized Viewdbt materializations compared: view, table, ephemeral, incremental and materialized view, with the SQL dbt runs for each and when to choose which one.
Intermediate
Patterns used in production pipelines.
- dbt Incremental Models: Strategies, unique_key, Late Data and Microbatchdbt incremental models in practice: is_incremental(), unique_key, merge vs append vs delete+insert vs microbatch, lookback windows, schema changes and full refresh.
- dbt Tests and Data Quality: Generic, Singular and Unit Tests, Contracts and Freshnessdbt tests explained with real runs: generic and singular data tests, custom tests, severity and store_failures, unit tests, model contracts and source freshness.
- dbt Snapshots and SCD Type 2: Timestamp vs Check Strategy and Hard Deletesdbt snapshots for SCD Type 2 history: YAML snapshot config, timestamp vs check strategy, hard_deletes options, meta columns and point-in-time queries, run for real.
- dbt Jinja, Macros and Packages: dbt_utils, Custom Macros and When Not to Templatedbt Jinja and macros with real compiled output: loops, variables, whitespace control, a custom macro, generate_schema_name, dbt_utils and avoiding over-templating.
Advanced
Performance, internals and edge cases.
Interview preparation
The top interview questions, with answers.
Projects and case studies
Apply what you learned and prepare material to discuss in interviews.
Projects
System design case studies
- AdvancedDesign a Data Quality FrameworkBad data keeps reaching dashboards and models: duplicated orders after a retry, a source that silently sent half its rows, negative prices after an upstream change. Design a company-wide data quality framework that lets teams declare expectations on their datasets, enforces them at the right points in batch and streaming pipelines, stops bad data from being published, and routes problems to the people who can fix them.
- IntermediateDesign an ELT Pipeline with dbtA growing company loads data from its product database, Stripe, Salesforce and web events into Snowflake with hand-written scripts and runs hundreds of untested SQL views. Design an ELT pipeline built on dbt that loads raw data reliably, transforms it into tested, documented models, deploys changes safely, and finishes the daily build before the business day starts.
- AdvancedDesign a Metrics and KPI PlatformLeadership tracks about 80 company KPIs (revenue, active users, conversion, retention, delivery time) in dashboards, spreadsheets, board decks and product experiments, and the numbers rarely agree. Design a metrics platform where every KPI is defined once, computed consistently at any grain and dimension, versioned, monitored for anomalies, and served to every tool through one interface.
- AdvancedDesign an Analytics and BI PlatformDesign the analytics and BI platform for a mid-sized company where executives, finance, marketing and operations all need dashboards and self-service analysis, data comes from about 40 sources, and today different teams report different numbers for the same metric.
- AdvancedDesign a Slowly Changing Dimension FrameworkA warehouse has 60 dimensions (customers, products, stores, sales territories, employees) whose attributes change over time. Each team handles history differently, some overwrite values and lose history, others produce overlapping or duplicate versions. Design a reusable slowly changing dimension framework that applies the right history rules per attribute, loads changes idempotently from batch and CDC sources, handles late and out-of-order changes, and lets facts join to the version that was valid at the time.
Resources
Cheat sheets
Related courses
- SQLSQL is the core language of data work: querying, transforming and modelling data in warehouses, lakehouses and Spark. Start here before any other tool.
- ETL and ELTETL transforms data before loading it; ELT loads first and transforms inside the warehouse or lakehouse. Learn when each fits.
- Data modelingData modeling and warehousing: star schemas and grain, fact and dimension design, SCDs, Data Vault, dbt, semantic layers and incremental models.
- Apache AirflowApache Airflow schedules and orchestrates pipelines as DAGs. Learn scheduling, task dependencies, retries and idempotent task design.
