
using-dbt-state
โ 608by dbt-labs ยท part of dbt-labs/dbt-agent-skills
Use when a user is enabling, configuring, optimizing, or debugging dbt State (the server-backed reuse mechanism that clones or skips nodes instead of rebuilding them). Use when they conflate dbt State with the `state:modified` selector or `--state` deferral. Use when asked about models rebuilding unexpectedly, views with `select *` rebuilding, volatile SQL (`current_timestamp()`, `random()`) rebuilding or not, cross-developer cloning, lag_tolerance.
This is the playbook your agent receives when the skill activates โ you don't need to read it to use the skill, but it's here to audit before installing.
Using dbt State
dbt State is a server-backed reuse mechanism. It should not be conflated with dbt's state:modified selector or --state deferral.
Before building each selected node, dbt asks the dbt State server whether the object can be skipped (reuse from the target schema), cloned (reuse from another schema), or must be built. It is the successor to State-Aware Orchestration, but works in dbt Core, in development, and in CI โ not just Fusion in production.
dbt State is a paid product, but it does not require a dbt platform (fka dbt Cloud) subscription.
Common Misconceptions
| Misconception | Reality |
|---|---|
"dbt State is just state:modified / --state" | No. state:modified hashes file contents against a manifest you manage and rebuilds state:modified+ (all descendants). dbt State manages state automatically on a server, parses SQL into a syntax tree and compares semantic hashes, considers upstream data freshness, and rebuilds a descendant only if it actually depends on the change (not the whole + subtree). |
| "It's Fusion-only / production-only" | Works in dbt Core, the dbt platform, and Fusion, across dev, CI, and production, with any orchestrator. |
| "dbt Core users can't use it" | They can. dbt Core 1.7โ1.11 require pip install dbt-state. It's baked into dbt Core 1.12 / v2.0 and Fusion. |
| "It's free / it's local" | It calls the dbt State server and requires authentication via a dbt platform account or a standalone dbt State account (app.state.dbt.com). Reuse is metered in DATTs โ daily active target tables (see Billing below). |
| "It sends my data to dbt Labs" | It sends last-modified timestamps and SQL text. The SQL is hashed then discarded โ dbt Labs cannot read query contents after hashing, and can never access raw data. |
How the reuse decision works
For each selected node, dbt State picks the cheapest valid option:
- Skip โ object exists in the target schema, its semantic hash is unchanged, and no parent has fresher data beyond
lag_tolerance. Does nothing. - Clone โ a matching object (same hash, fresh data) exists in another schema (e.g. production, or a teammate's dev schema). Clones it, marked Reused. Uses zero-copy clone if supported by the warehouse, or runs a CTAS statement to copy the transformed data from elsewhere if not. Test results are reused too โ a failing test still surfaces even though it wasn't re-executed.
- Build โ no valid reuse. Builds normally, auto-deferring unselected upstream nodes.
If a node is selected for execution but its inputs do not exist in the target schema, dbt State uses deferral as normal. If a manifest.json is present it will use that, otherwise it will make a best-effort guess at the correct FQN based on the generate_*_name macros. Deferral does not consume DATTs. The defer_to_target config in profiles.yml can be used to specify which schema to defer to for self-managed users. It is not necessary for dbt platform users.
To get freshness, dbt fetches warehouse metadata (or loaded_at_field/loaded_at_query) for each input relation. For views without a loaded_at config, it traverses upstream until it finds a real table.
Query normalization & why models rebuild
dbt State hashes a parsed syntax tree, so it ignores cosmetic changes โ whitespace, comments, table aliases, dbt lint --fix reformatting. A model rebuilds only when its logic or data changes.
Volatile SQL (current_timestamp(), getdate(), random()): by default treated as logic โ the hash uses the function name, not its runtime value, so it does not invalidate the model every run (otherwise nothing downstream of getdate() could ever be reused). To make a model rebuild when the value changes:
- Set
evaluate_volatile_sql: true(preferred โ covers all functions in the model, inheritable like any config). dbt State emulates the function's value into the hash. - Or use a Jinja equivalent (e.g.
{{ run_started_at }}) โ Jinja renders before parsing, so it changes the compiled SQL each run.
Non-deterministic Jinja (e.g. dbt_utils.get_relations_by_pattern returning relations in varying order) produces a different compiled hash and triggers rebuilds even when logic is unchanged.
Config changes: only build-relevant configs affect the hash (materialized, on_schema_change, severity, โฆ). Cosmetic configs (meta, tags) are ignored. If a post-hook mutates tables based on ignored fields (e.g. applying meta as warehouse tags), set execute_hooks_on_any_reuse: true so hooks run on reuse.
Configs quick reference
Set under models: +state: in dbt_project.yml, in schema.yml config.state, or in {{ config(state={...}) }}.
| Config | Default | Purpose |
|---|---|---|
lag_tolerance | 45m | How stale data may be before a node is eligible to rebuild. Data freshness only โ SQL changes rebuild regardless. |
require_fresh_data_from | any | Whether any or all direct parents need fresh data to trigger a rebuild. |
evaluate_volatile_sql | false | Hash the runtime value of volatile functions instead of the name. |
pre_clone | if_missing | Pre-populate incremental models/snapshots by cloning prod before a run (never / if_missing / always). |
execute_hooks_on_any_reuse | false | Run pre/post-hooks even when a node is reused. |
defer_to_target | prod | (Self-managed only, profile) Which profile target to defer/clone from. |
metadata_warehouse | profile warehouse | (Snowflake only, profile) Separate warehouse for metadata lookups. |
Supported warehouses: Snowflake, Databricks, BigQuery, Redshift.
Billing: daily active target tables (DATT)
dbt State usage is metered in DATTs (daily active target tables), not by "models built".
- A target table is a database object managed by your project (per database + schema): seeds, snapshots, models (incl. incremental), and each distinct test โ even tests not stored in the database (
store_failuresoff). Example:dim_customerswithnot_nullanduniqueonid= 3 target tables (the model + 2 tests). - A target table becomes a DATT when dbt State performs at least one skip, clone, or test reuse on it on a given day (UTC). All reuses of the same target table in one day count as a single DATT. A full build is not a reuse.
- Views are never billed as DATTs, even if reused or cloned. Tests attached to a view will be billed as normal.
If asked about pricing details, refer the user to https://www.getdbt.com/product/dbt-state.
Optimizations for best results
lag_toleranceper environment โ in dev, set it high (e.g. a week) so dbt does nothing when data is only slightly stale; cloning is cheap but doing nothing is cheaper. Example:# dbt_project.yml models: +state: lag_tolerance: "{{ '4h' if target.name == 'prod' else '7d' }}"- Keep using selectors in development. Any target table dbt State reuses counts as a DATT for that day (even one inside its lag-tolerance window). Select only the nodes you're working on so plain deferral handles the rest โ untouched, unselected nodes incur no dbt State usage.
- Reduce complex selector usage in production. dbt State makes most jobs collapse toward plain
dbt build; let it decide what to rebuild instead of hand-tuning per-job selection. Specify lag_tolerance to prevent overbuilding. - Specify columns instead of
select *to increase likelihood of reuse. If dbt State can't prove atable.*or similar has the same column set, it will rebuild to be sure. This is particularly relevant for views. Fusion's static analysis is not currently used for this.
Diagnosing confusing behavior
| Symptom | Cause / fix |
|---|---|
A model with current_timestamp() keeps rebuilding | Likely evaluate_volatile_sql: true somewhere, or a Jinja value (e.g. run_started_at) changing the compiled SQL. If you want reuse, leave volatile SQL as default (logic). |
| Model rebuilds despite "no change" | Cosmetic change isn't the cause (those are normalized away). Look for non-deterministic Jinja (unordered macro output), a build-relevant config change, or fresher upstream data past lag_tolerance. Metadata tables can consider a table modified by an insert command even if no new rows were added. Consider using loaded_at_field, but this may be more costly in the warehouse - metadata queries are often free but loaded_at_field will be a standard paid query. |
| Post-hooks didn't run on a reused model | Hooks don't run on reuse by default โ set execute_hooks_on_any_reuse: true. |
| Want to know why a node was reused/rebuilt | Use the dbt-state explain command (dbt v1.7โ1.12) to inspect the decision. |
| Need authentication / access | Log in via your dbt platform account or a standalone dbt State account. For an org, set state-org-id under dbt-cloud: in dbt_project.yml. |
v1 (Python) vs v2 (Rust/Fusion)
| dbt Core 1.7โ1.11 | dbt Core 1.12 / v2.0 | Fusion | |
|---|---|---|---|
| Install | pip install dbt-state required | Built in | Built in |
- dbt v1.7-1.11 users must install the separate
dbt-statepackage to use dbt State. - dbt v1.12+ users have the
dbt-statepackage included automatically. - dbt v2.0+ (either Core or Fusion distributions) have the Rust implementation of the client logic built in, so no separate install is needed.
The reuse behavior, configs, and query normalization are server-side and behave consistently across all engines. The main v1 difference is the separate dbt-state install for 1.7โ1.11. The dbt-state explain diagnostic is not available in dbt v2.
Related docs
- Overview:
/docs/deploy/dbt-state-about - Setup:
/docs/deploy/dbt-state-setupยท Examples:/docs/deploy/dbt-state-examples - Monitor activity:
/docs/deploy/dbt-state-interfaceยท Deferral:/docs/deploy/dbt-state-deferralยท CI/CD:/docs/deploy/dbt-state-cicd - Configs:
/reference/resource-configs/dbt-state-configsยทlag_toleranceยทdefer_to_target
npx skills add https://github.com/dbt-labs/dbt-agent-skills --skill using-dbt-stateRun this in your project โ your agent picks the skill up automatically.
No common issues documented yet. If you hit a problem, the repository's GitHub Issues page is the best place to look.
Licensed under Apache-2.0โ you can use, modify, and redistribute it under that license's terms.
View the full license file on GitHub โ