Skip to main content

Reading a GitHub Actions YAML

If you are using dbt Cloud, it handles all Continuous Integration (CI) and Deployment for you via its web interface.

However, many data teams (especially those using dbt Core) prefer to use GitHub Actions to orchestrate their CI/CD pipelines. This allows Data Engineers to define their deployment infrastructure entirely as code using YAML files.

Topics Covered​

  • What is a GitHub Action?
  • Triggers (Push vs Pull Request)
  • Managing Credentials securely (GitHub Secrets)
  • Anatomy of a dbt CI Workflow

1. What is a GitHub Action?​

A GitHub Action is simply a script that runs on a temporary GitHub server (called a "Runner") every time a specific event happens in your repository.

For dbt, the most common use case is a CI Check: When Arjun opens a Pull Request to merge his changes into the main branch, GitHub Actions spins up a server, installs dbt, connects to Snowflake, and runs dbt build to ensure his code doesn't break anything. If the build fails, the PR cannot be merged.

2. Anatomy of a GitHub Actions YAML​

Workflow files must be placed in a very specific directory in your repository: .github/workflows/.

Let's look at a complete, production-ready YAML file designed to test dbt Pull Requests for an Indian e-commerce company.

name: dbt Slim CI Pipeline

# ---------------------------------------------------------
# 1. TRIGGERS: When should this run?
# ---------------------------------------------------------
on:
pull_request:
branches:
- main # Only run when someone opens a PR targeting the main branch

# ---------------------------------------------------------
# 2. ENVIRONMENT VARIABLES & SECRETS
# ---------------------------------------------------------
env:
# Do NOT hardcode passwords here. Use GitHub Secrets.
DBT_SNOWFLAKE_ACCOUNT: ${{ secrets.SNOWFLAKE_ACCOUNT }}
DBT_USER: ${{ secrets.SNOWFLAKE_CI_USER }}
DBT_PASSWORD: ${{ secrets.SNOWFLAKE_CI_PASSWORD }}
DBT_ROLE: CI_ROLE
DBT_WAREHOUSE: COMPUTE_WH
DBT_DATABASE: BEEMS_PROD
DBT_SCHEMA: PR_TESTING_${{ github.event.pull_request.number }}

# ---------------------------------------------------------
# 3. JOBS: What exactly should the server do?
# ---------------------------------------------------------
jobs:
run-dbt-ci:
name: Run dbt models
runs-on: ubuntu-latest # The operating system for the runner

steps:
# Step A: Download the repository code onto the runner
- name: Checkout code
uses: actions/checkout@v3

# Step B: Install Python
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.10"

# Step C: Install dbt Core and the Snowflake adapter
- name: Install dependencies
run: |
pip install dbt-core dbt-snowflake

# Step D: Install dbt packages (from packages.yml)
- name: dbt deps
run: dbt deps

# Step E: Run the actual dbt models
# (In a real setup, you would use Slim CI / state:modified here)
- name: dbt build
run: dbt build --target ci

3. Understanding the Components​

Triggers (on:)​

This tells GitHub when to run the script. on: push runs every time a commit is made. on: pull_request is standard for CI testing. You can also use on: schedule (with a CRON expression) to run nightly batch jobs.

Secrets (${{ secrets.XYZ }})​

You never commit passwords to a Git repository. Instead, a repository admin goes to the GitHub UI -> Settings -> Secrets and Variables -> Actions, and adds the Snowflake password there. The YAML file references it dynamically at runtime.

Dynamic Schemas (${{ github.event.pull_request.number }})​

Notice this line: DBT_SCHEMA: PR_TESTING_${{ github.event.pull_request.number }}. If Arjun opens PR #42, dbt will build his test tables in a schema named PR_TESTING_42. This ensures that if Priya opens PR #43 at the exact same time, their tests do not overwrite each other!

Steps (steps:)​

The steps are executed sequentially. If Step B fails, the workflow immediately halts and marks the PR with a red "X".

Summary​

  • GitHub Actions automate dbt CI/CD workflows using YAML files placed in .github/workflows/.
  • Use on: pull_request to trigger test runs before code is merged.
  • Never hardcode credentials; always use GitHub Secrets (${{ secrets.VAR }}).
  • Use GitHub variables to create Dynamic Schemas (e.g., PR_TESTING_123) so multiple developers can run CI tests simultaneously without conflict.