Model-Level Meta Fields
As our BEEMS TECHNE E-Commerce dbt project grows, we need more than SQL transformations.
We also need to capture information about the models themselves.
For example:
- Who owns this model?
- Which business domain does it belong to?
- How critical is it?
- What is the expected refresh frequency?
- Is it a core business model?
- Is it used by a dashboard?
- Is it confidential?
- Which team is responsible for it?
dbt provides the meta configuration for storing this type of custom metadata.
However, dbt also provides tags, which can appear to solve some of the same problems.
This lesson explains the difference between:
Tags
↓
Classification / Grouping / Selection
Meta
↓
Structured Custom Information
Documentation
↓
Human-readable Explanation
Simple rule:
Tags answer "Which group?"
Meta answers "What attributes or properties?"
Topics Covered
- What is Model-Level
meta? - Why do we need
meta? metavstags- Adding
metato models - Three real-world e-commerce examples
- Using
metafor ownership and governance - Using structured metadata
metavstagsvs documentation- Best practices
1. What is Model-Level meta?
meta allows us to attach custom structured metadata to a dbt resource.
For example:
meta:
business_domain: "Sales"
business_owner: "E-Commerce Business Team"
technical_owner: "Data Engineering"
criticality: "HIGH"
The metadata does not change the data returned by the model.
It does not change the transformation logic.
It provides additional information about the model.
Conceptually:
fct_orders
|
+--------------+--------------+
| | |
SQL Tags Meta
| | |
Transformation sales owner
gold domain
dashboard criticality
2. Why Do We Need meta?
Imagine our project initially contains 10 models.
It is easy to remember:
fct_orders
↓
Sales
↓
Owned by Data Engineering
But imagine the project eventually contains:
500+ models
50+ developers
10+ business domains
multiple dashboards
multiple data owners
Now questions such as these become difficult:
Who owns
fct_orders?
Which models are business critical?
Which models contain confidential information?
Which models have an hourly SLA?
Which team owns the customer domain?
This is where structured metadata becomes useful.
3. tags vs meta
This is the most important concept in this lesson.
Tags
Tags are primarily used for classification and selection.
Example:
tags:
- sales
- gold
- dashboard
We can then select models using:
dbt build --select tag:sales
Tags answer:
"Which models belong to this group?"
Meta
meta is used to store custom attributes or properties.
Example:
meta:
business_domain: "Sales"
business_owner: "E-Commerce Business Team"
criticality: "HIGH"
refresh_frequency: "HOURLY"
Meta answers:
"What additional information do we know about this model?"
4. The Easy Way to Remember
Think about an employee.
Employee: Ravi
Tags:
developer
full-time
engineering
Properties:
manager: John
joining_date: 2025-01-10
experience_years: 8
Tags tell us:
Which groups does Ravi belong to?
Properties tell us:
What are Ravi's attributes?
The same concept applies to dbt models.
DBT MODEL
Tags:
sales
gold
dashboard
Meta:
owner: Sales Analytics
criticality: HIGH
SLA: 60 minutes
5. Model 1 — stg_customers
Let's start with our Bronze layer.
models/bronze/stg_customers.sql
We can define metadata in our model YAML.
version: 2
models:
- name: stg_customers
description: >
Staging model containing cleaned customer data
from the RAW customer source.
config:
tags:
- customer
- bronze
- hourly
meta:
business_domain: "Customer"
business_owner: "Customer Analytics Team"
technical_owner: "Data Engineering"
criticality: "MEDIUM"
refresh_frequency: "HOURLY"
Now let's separate what each section means.
Tags
tags:
- customer
- bronze
- hourly
These classify the model.
We can run:
dbt build --select tag:customer
or:
dbt build --select tag:hourly
Meta
meta:
business_domain: "Customer"
business_owner: "Customer Analytics Team"
technical_owner: "Data Engineering"
criticality: "MEDIUM"
refresh_frequency: "HOURLY"
These are properties describing the model.
For example:
Business Domain:
Customer
Business Owner:
Customer Analytics Team
Technical Owner:
Data Engineering
Criticality:
MEDIUM
Refresh:
HOURLY
We are not trying to select models using:
criticality: MEDIUM
We are storing this as information about the model.
6. Model 2 — fct_orders
Now let's move to the Gold layer.
models/gold/fct_orders.sql
This is an important analytical model for our e-commerce platform.
version: 2
models:
- name: fct_orders
description: >
Order-level fact table containing customer, order,
product and payment metrics for the BEEMS TECHNE
E-Commerce platform.
config:
tags:
- sales
- gold
- core
- dashboard
meta:
business_domain: "Sales"
business_owner: "E-Commerce Business Team"
technical_owner: "Data Engineering"
criticality: "HIGH"
refresh_frequency: "HOURLY"
source_system: "BEEMS_TECHNE_ECOMM"
Now look at the tags:
tags:
- sales
- gold
- core
- dashboard
These tell us:
This model is:
Sales related
Gold layer
Core model
Used by dashboards
We can use them for selection.
For example:
dbt build --select tag:sales
Now look at the metadata:
meta:
business_domain: "Sales"
business_owner: "E-Commerce Business Team"
technical_owner: "Data Engineering"
criticality: "HIGH"
refresh_frequency: "HOURLY"
source_system: "BEEMS_TECHNE_ECOMM"
This tells us additional information about the model.
For example:
Business Domain:
Sales
Business Owner:
E-Commerce Business Team
Technical Owner:
Data Engineering
Criticality:
HIGH
Refresh Frequency:
HOURLY
Source System:
BEEMS_TECHNE_ECOMM
These are attributes, not groups.
7. Model 3 — customer_360
Now consider a model used by multiple dashboards.
models/mart/customer_360.sql
We could define:
version: 2
models:
- name: customer_360
description: >
Customer 360 analytical model combining customer,
order, payment and product information.
config:
tags:
- customer
- mart
- dashboard
- executive
meta:
business_domain: "Customer"
business_owner: "Customer Analytics Team"
technical_owner: "Data Platform Team"
criticality: "CRITICAL"
refresh_frequency: "HOURLY"
consumption: "EXECUTIVE_DASHBOARD"
data_classification: "INTERNAL"
Now we have:
Tags
customer
mart
dashboard
executive
These allow us to group/select the model.
Meta
business_domain
business_owner
technical_owner
criticality
refresh_frequency
consumption
data_classification
These describe the model.
8. Comparing Our Three Models
We now have three different models.
| Model | Tags | Example Meta |
|---|---|---|
stg_customers | customer, bronze, hourly | domain, owner, criticality |
fct_orders | sales, gold, core, dashboard | domain, owner, SLA, source |
customer_360 | customer, mart, dashboard, executive | domain, owner, criticality, classification |
Notice the difference.
Tags:
customer
sales
gold
dashboard
executive
hourly
These are useful for grouping and selection.
Meta:
business_owner
technical_owner
criticality
refresh_frequency
source_system
data_classification
These are attributes.
9. Why Not Put Everything into Tags?
A common beginner question is:
"Why don't we just use tags for everything?"
For example:
tags:
- sales
- high_criticality
- hourly
- data_engineering
- ecommerce_business_team
This can work technically, but it is not a good way to represent structured information.
Consider:
criticality = HIGH
This is a property with a value.
Compare that with:
sales
which is a classification.
Therefore:
tags:
- sales
- dashboard
is appropriate.
While:
meta:
criticality: "HIGH"
business_owner: "E-Commerce Business Team"
is more appropriate.
10. Tags and Meta Together
In real projects, we often use both.
For example:
models:
- name: fct_orders
config:
tags:
- sales
- gold
- dashboard
meta:
business_domain: "Sales"
business_owner: "E-Commerce Business Team"
technical_owner: "Data Engineering"
criticality: "HIGH"
refresh_frequency: "HOURLY"
Think of it as:
FCT_ORDERS
|
+-----------+-----------+
| |
TAGS META
| |
"How do we group it?" "What do we know about it?"
| |
sales / gold owner
dashboard domain
criticality
refresh
11. Model-Level meta in SQL
Metadata can also be configured directly inside the model's config() block.
For example:
{{ config(
materialized='table',
tags=['sales', 'gold'],
meta={
"business_domain": "Sales",
"criticality": "HIGH",
"business_owner": "E-Commerce Business Team"
}
) }}
SELECT
...
FROM {{ ref('int_order_details') }}
This is valid.
However, for our training project, we will generally keep model documentation and metadata in schema.yml.
This keeps:
SQL
focused on transformation logic and:
schema.yml
focused on documentation, tests and model configuration.
12. Structured Metadata
One of the advantages of meta is that it can contain structured information.
For example:
meta:
ownership:
business_owner: "E-Commerce Business Team"
technical_owner: "Data Engineering"
governance:
criticality: "HIGH"
data_classification: "INTERNAL"
operations:
refresh_frequency: "HOURLY"
sla_minutes: 60
This gives us a structure like:
fct_orders
│
├── ownership
│ ├── business_owner
│ └── technical_owner
│
├── governance
│ ├── criticality
│ └── data_classification
│
└── operations
├── refresh_frequency
└── sla_minutes
This is much richer than simply having a list of tags.
13. meta Can Support Automation
Because metadata is structured, advanced dbt projects can use it programmatically.
For example:
meta:
criticality: "HIGH"
A macro or external tool could potentially use this information to:
- identify critical models
- generate governance reports
- identify model owners
- apply custom processing
- integrate with data catalog tools
- enforce organizational standards
This is one reason meta is more than just another form of tagging.
14. meta vs tags vs Documentation
We now have three different mechanisms.
Tags
tags:
- sales
- gold
- dashboard
Purpose:
Classification and selection
Example:
dbt build --select tag:sales
Meta
meta:
business_owner: "E-Commerce Business Team"
criticality: "HIGH"
refresh_frequency: "HOURLY"
Purpose:
Structured custom attributes
Documentation
For detailed explanations, use descriptions or dbt documentation.
For example:
description: >
FCT_ORDERS contains order-level analytical information
used for sales reporting and dashboard analytics.
Or use a documentation block:
{% docs fct_orders_business_definition %}
FCT_ORDERS represents the analytical view of customer orders.
The model combines order, customer, product and payment
information and is used by sales dashboards.
{% enddocs %}
Then:
description: "{{ doc('fct_orders_business_definition') }}"
Documentation answers:
"What does this model mean?"
15. Complete Comparison
| Feature | Tags | Meta | Documentation |
|---|---|---|---|
| Primary purpose | Grouping | Custom attributes | Explanation |
Select with --select | Yes | No | No |
| Store owner | Not ideal | Yes | Possible, but not ideal |
| Store criticality | Not ideal | Yes | Possible, but not ideal |
| Store business domain | Yes | Yes | Not its primary purpose |
| Store refresh frequency | Not ideal | Yes | No |
| Explain business logic | No | No | Yes |
| Store structured values | Limited | Yes | No |
| Useful for orchestration | Yes | Can support automation | No |
| Human-readable explanation | No | Limited | Yes |
16. Practical Example
Let's put everything together for fct_orders.
version: 2
models:
- name: fct_orders
description: >
Order-level fact table containing customer, product,
payment and order information.
config:
tags:
- sales
- gold
- core
- dashboard
meta:
ownership:
business_owner: "E-Commerce Business Team"
technical_owner: "Data Engineering"
governance:
criticality: "HIGH"
data_classification: "INTERNAL"
operations:
refresh_frequency: "HOURLY"
sla_minutes: 60
Now we can think of the model as:
FCT_ORDERS
│
├── TAGS
│ ├── sales
│ ├── gold
│ ├── core
│ └── dashboard
│
├── META
│ ├── ownership
│ │ ├── business_owner
│ │ └── technical_owner
│ │
│ ├── governance
│ │ ├── criticality
│ │ └── data_classification
│ │
│ └── operations
│ ├── refresh_frequency
│ └── sla_minutes
│
└── DESCRIPTION
└── Business definition of FCT_ORDERS
17. Hands-On Exercise
Add the following configuration to your three models.
stg_customers
config:
tags:
- customer
- bronze
- hourly
meta:
business_domain: "Customer"
business_owner: "Customer Analytics Team"
criticality: "MEDIUM"
fct_orders
config:
tags:
- sales
- gold
- core
- dashboard
meta:
business_domain: "Sales"
business_owner: "E-Commerce Business Team"
technical_owner: "Data Engineering"
criticality: "HIGH"
refresh_frequency: "HOURLY"
customer_360
config:
tags:
- customer
- mart
- dashboard
- executive
meta:
business_domain: "Customer"
business_owner: "Customer Analytics Team"
criticality: "CRITICAL"
data_classification: "INTERNAL"
18. What Should We Use?
When designing metadata for a dbt model, ask these questions:
Question 1
"Do I want to group or select models?"
Use:
tags:
Question 2
"Do I want to store a property or attribute about the model?"
Use:
meta:
Question 3
"Do I want to explain the meaning or business logic?"
Use:
description:
or:
{% docs ... %}
Summary
The three concepts have different responsibilities.
DBT MODEL
|
+-----------+-----------+
| | |
TAG META DOCS
| | |
▼ ▼ ▼
CLASSIFY ATTRIBUTES EXPLAIN
| | |
▼ ▼ ▼
"Which?" "What?" "Why/How?"
Tags
tags:
- sales
- gold
- dashboard
Used primarily for grouping and selection.
Meta
meta:
criticality: "HIGH"
business_owner: "E-Commerce Business Team"
Used for custom structured metadata and attributes.
Documentation
description: >
Order-level analytical fact table...
Used to explain the model and its business meaning.
Key Takeaway
Don't think of
metaas a replacement for tags.Tags classify models. Meta stores attributes about models. Documentation explains models.
For our BEEMS TECHNE E-Commerce project:
stg_customers
│
├── Tags → customer, bronze, hourly
├── Meta → owner, domain, criticality
└── Docs → What is customer staging data?
fct_orders
│
├── Tags → sales, gold, core, dashboard
├── Meta → owner, SLA, criticality, classification
└── Docs → What does an order fact represent?
customer_360
│
├── Tags → customer, mart, dashboard, executive
├── Meta → owner, criticality, classification
└── Docs → What does Customer 360 mean?