Skip to main content

02. Creating a DBT Project

Creating Your First DBT Project​

Installing DBT Core is only the prerequisite. To actually start writing data transformations, you must formally initialize a project. A DBT project is a structured directory of SQL files and YAML configurations that tells the DBT Compiler exactly what to build.


❄️ 1. Preparing Your Snowflake Source Data​

Before initializing your DBT project, you need a target database and schema to exist in Snowflake. You can copy and paste the following DDL script into a Snowflake Worksheet to create a database, schema, and sample table.

This gives you a ready-to-use dataset for practicing source() and ref() functions later on!

-- 1. Set the context
USE ROLE ACCOUNTADMIN;
USE WAREHOUSE COMPUTE_WH;

-- 2. Create Database and Schemas
CREATE DATABASE IF NOT EXISTS beems_dbt_project_demo;
CREATE SCHEMA IF NOT EXISTS beems_dbt_project_demo.raw; -- For raw source data
CREATE SCHEMA IF NOT EXISTS beems_dbt_project_demo.dev; -- Default schema for dbt to build into

-- 3. Set context to the source schema
USE DATABASE beems_dbt_project_demo;
USE SCHEMA raw;

-- 4. Create sample raw source tables (5 columns each)
CREATE OR REPLACE TABLE raw_customers (
customer_id INT,
first_name STRING,
last_name STRING,
email STRING,
signup_date DATE
);

CREATE OR REPLACE TABLE raw_orders (
order_id INT,
customer_id INT,
order_date DATE,
status STRING,
amount FLOAT
);

-- 5. Insert sample records to practice with
INSERT INTO raw_customers (customer_id, first_name, last_name, email, signup_date)
VALUES
(1, 'Alice', 'Smith', 'alice@example.com', '2023-01-15'),
(2, 'Bob', 'Jones', 'bob@example.com', '2023-02-20'),
(3, 'Charlie', 'Brown', 'charlie@example.com', '2023-03-10');

INSERT INTO raw_orders (order_id, customer_id, order_date, status, amount)
VALUES
(101, 1, '2023-01-20', 'completed', 150.50),
(102, 2, '2023-02-22', 'completed', 200.00),
(103, 1, '2023-03-01', 'pending', 50.25);

2. Running the Initialization Command​

Instead of creating folders manually, DBT provides a powerful scaffolding command. Open your terminal, navigate to your desired workspace (e.g., ~/Desktop/), and run:

dbt init my_first_dbt_project

DBT will immediately trigger an interactive wizard. Because DBT acts as a translator for your specific data warehouse, it must first ask for your connection credentials to build your profile behind the scenes.

The Interactive Terminal Flow​

When using the Snowflake adapter, the wizard will ask you standard connection questions:

13:35:09  Setting up your profile.
Which database would you like to use?
[1] snowflake

Enter a number: 1
account (https://<this_value>.snowflakecomputing.com): ZTVPZHE-RD14678
user (dev username): SFTRAINING
[1] password
[2] keypair
[3] sso
Desired authentication type option (enter a number): 1
password (dev password):
role (dev role): ACCOUNTADMIN
warehouse (warehouse name): COMPUTE_WH
database (default database that dbt will build objects in): beems_dbt_project_demo
schema (default schema that dbt will build objects in): dev
threads (1 or more) [1]: 1
13:39:15 Profile my_first_dbt_project written to ~/.dbt/profiles.yml using target's profile_template.yml and your supplied values.

Your new project directory is now successfully created with all the required folders!


3. Understanding Profiles (profiles.yml)​

You might have noticed something extremely important in that terminal output: Profile my_first_dbt_project written to ~/.dbt/profiles.yml

Why did DBT save your database connection details to a hidden ~/.dbt folder on your laptop, instead of saving it inside your newly created project folder?

Security Best Practice

Your source code (models, macros) will be committed to a Git repository (like GitHub) and shared with your entire team. Your database password should never be committed to version control.

By hardcoding the profiles.yml file into a hidden directory on your personal computer (~/.dbt/), your database credentials are securely isolated from your codebase.

Inside the Profile File​

If you opened that ~/.dbt/profiles.yml file using a text editor, it would look like this:

my_first_dbt_project:    # ️ This name binds the profile to your project
target: dev
outputs:
dev:
type: snowflake
account: ZTVPZHE-RD14678
user: SFTRAINING
password: "YOUR_PASSWORD_HERE"
role: ACCOUNTADMIN
database: beems_dbt_project_demo
warehouse: COMPUTE_WH
schema: dev
threads: 4
client_session_keep_alive: False

Whenever you run a command inside your project, DBT reads the local dbt_project.yml file, sees the project name is my_first_dbt_project, and automatically secretively links to ~/.dbt/profiles.yml to pull those credentials into memory.


‍ 4. Validating the Connection (dbt debug)​

Before you write a single line of SQL, you must verify that DBT can successfully communicate with your Cloud Data Warehouse. You do this using the debug command.

Critical Step: Navigate into the Project

If you try to run a DBT command directly in your Home folder, you will encounter the dreaded "Profile should not be None" error. You must change directories into your newly created project folder first so DBT has context of what project to run for!

cd my_first_dbt_project
dbt debug

If your configuration in profiles.yml is flawless, DBT will ping Snowflake and return all green checks:

13:58:36  Running with dbt=1.10.7
13:58:36 dbt version: 1.10.7
...
13:58:37 Configuration:
13:58:37 profiles.yml file [OK found and valid]
13:58:37 dbt_project.yml file [OK found and valid]
13:58:37
13:58:37 Required dependencies:
13:58:37 - git [OK found]
13:58:37
13:58:37 Connection test: [OK connection ok]
13:58:37
13:58:37 All checks passed!

Congratulations! Your local machine is perfectly bridged to Snowflake, and you are officially ready to start engineering data.