Skip to main content

04. Debugging with Logs

Debugging with Logs in dbt​

Debugging with logs is essential to identify failures, performance bottlenecks, and unexpected behavior in dbt models.
Proper log analysis ensures efficient troubleshooting and improves pipeline reliability.


Key Log Types​

dbt run logs – Track model execution and SQL statements
dbt test logs – Track results of tests (uniqueness, not null, relationships)
Snapshot logs – Track historical versioning and changes
CLI debug mode – Provides detailed execution information


1. Enabling Debug Mode​

  • Use --debug to see detailed logs during dbt runs:
dbt run --profiles-dir ./profiles --target prod --debug
  • Provides information on:
    • SQL compilation
    • Model execution time
    • Errors and stack traces

2. Reviewing dbt Logs​

  • Logs are stored in logs/dbt.log by default
  • Contains:
    • Timestamps for execution steps
    • SQL queries executed
    • Error messages and warnings
tail -f logs/dbt.log
  • Use grep or search tools to filter relevant logs

3. Debugging Common Issues​

Common scenarios:

  1. Compilation errors – Check dbt compile output for SQL issues
  2. Test failures – Review dbt test logs for failing assertions
  3. Performance bottlenecks – Identify long-running models in logs
  4. Snapshot issues – Ensure correct configuration of unique_key and strategy

4. Best Practices for Log Debugging​

Enable --debug for troubleshooting complex issues
Review dbt logs after every production run
Centralize logs if running in orchestration tools (Airflow, Prefect)
Monitor execution times and identify slow models
Combine logging with notifications for proactive alerts