01. Demo Overview and Configuration
Objective
Build an e-commerce ELT pipeline for Beems Techne using Snowflake and dbt. The project moves data through RAW, Bronze, Silver, Gold, and Mart layers.
By the end of the demo, students will understand how dbt uses source() for Snowflake source tables and ref() for dependencies between dbt models.
Project Flow
Database and Project Names
Snowflake database: BEEMS_TECHNE_ECOMM
dbt project: beems_techne_ecomm_dbt
Schemas: RAW, BRONZE, SILVER, GOLD, MART
source() and ref()
Use source() when the data already exists in Snowflake before dbt runs:
FROM {{ source('ecommerce_raw', 'orders') }}
Use ref() when reading another dbt model:
FROM {{ ref('stg_orders') }}
source() documents the external input. ref() creates the dependency graph, so dbt can run models in the correct order.
Sources and Models
Create models/sources.yml:
version: 2
sources:
- name: ecommerce_raw
database: BEEMS_TECHNE_ECOMM
schema: RAW
tables:
- name: customers
- name: customer_addresses
- name: categories
- name: products
- name: orders
- name: order_items
- name: payments
- name: shipments
- name: returns
- name: promotions
- name: inventory_snapshots
- name: product_reviews
Configure dbt_project.yml
name: beems_techne_ecomm_dbt
version: "1.0.0"
config-version: 2
profile: beems_techne_ecomm_dbt
model-paths: ["models"]
snapshot-paths: ["snapshots"]
seed-paths: ["seeds"]
test-paths: ["tests"]
macro-paths: ["macros"]
analysis-paths: ["analyses"]
models:
beems_techne_ecomm_dbt:
bronze:
+schema: BRONZE
+materialized: view
silver:
+schema: SILVER
+materialized: table
gold:
+schema: GOLD
+materialized: table
mart:
+schema: MART
+materialized: view
Run a configuration check:
dbt debug
Expected result:
Connection test: OK
All checks passed!
Continue with 02. Snowflake RAW Setup.