learning-it by example

Back to the video

dbt and DuckDB: A Full Project, Tests, and a Bad Row

Build a complete dbt project on DuckDB with models and tests, then watch a test catch a bad row that dbt run lets through.

Video #14 ยท Data engineering and pipelines

Steps in the video

  1. 0:00 A whole dbt project on a laptop
  2. 0:19 What dbt does
  3. 0:49 Setup
  4. 1:04 Two config files
  5. 1:52 Three models
  6. 2:29 Tests in YAML
  7. 3:10 Now break the data
  8. 4:05 What this does not show
  9. 4:38 Mistakes to avoid
  10. 5:22 My take and next steps
  11. 5:50 Recap

Files

  • add_duplicate.py (9 lines)
  • dbt_project.yml (7 lines)
  • models/raw_orders.sql (7 lines)
  • models/revenue_by_day.sql (7 lines)
  • models/schema.yml (12 lines)
  • models/stg_orders.sql (7 lines)
  • profiles.yml (6 lines)

Download the code (zip)

Tested with

Python 3.12.15 | dbt-core==1.12.6, dbt-duckdb==1.11.1, duckdb==2.0.0

What this does not show

A tiny made-up project with one planted duplicate row. A passing run only proves the tests you wrote, and a failing test reports bad data without fixing or blocking it. The wrong revenue figure is reasoned, not printed. One adapter and one version of each package.

Sources

Learning IT by Example.