Performance & Cost

Your model is documented and secured — the last stage before publishing is optimizing how your modeled data is stored and served, for performance and cost. Credible maintains managed, derived copies of your data, and you opt parts of your model into them with a single annotation. The platform builds the copy, keeps it fresh, reuses it where it can, and wires it into serving — so your model gets faster, cheaper, and more searchable without you managing any of the machinery.

There are three kinds of derived copy, each opted into at its natural grain:

Each is derived from your published model, kept fresh by Credible, and reused automatically. You choose what to persist, what to make searchable, and what to roll up; Credible owns everything else.

Serving Behavior

A query on a persisted source serves from the derived copy when one exists, and otherwise runs live against your warehouse. The result is always correct — if a derived copy isn't available yet, the query is simply slower, not wrong.

• A materialized table persists a source's data as a physical table and routes queries to it.
• A search index embeds a dimension's values so they are findable by value search, filter suggestions, and the AI agent.

How Materialization and Indexing Compose

The two features work together automatically, with no extra configuration. If you index a dimension on a source you have also materialized, Credible builds the search index from the materialized table instead of re-scanning the warehouse — and reverts to the warehouse if you un-materialize the source.

You can rely on three properties:

• Consistent — the index reflects the materialized snapshot, so searchable values match what queries return.
• Stable — a table-backed index refreshes exactly when its source table refreshes. An index on a source you have not materialized refreshes on publish and on demand, and — if you declare a freshness window — on that window.
• Cheap — indexing reuses the table you already built instead of paying to re-scan the warehouse.

The recommended pattern for an expensive, frequently-searched source is "materialize the source, then index its dimensions." Credible sequences the work for you so the index is always built after the table it derives from.

Deciding What to Persist

The annotation alone is the intended usage. Defaults are chosen so the platform can optimize on your behalf — deduplicating copies across model versions and scheduling refreshes to meet a freshness objective. At most, add a freshness window for data with a real staleness requirement.

Configuration

Annotations

Add the annotation to the source or dimension you want to persist. Options are optional key="value" pairs; omit them to accept the platform default.

#@ persist name="orders_fast" refresh="incremental" watermark="order_date" freshness.window="24h"
source: orders is conn.table('sales.orders') extend {
  #(index)
  dimension: status is order_status
}

• name — choose where the materialized table lands (optional; container-qualifiable).
• refresh — "full" (the default) rebuilds the whole copy; "incremental" applies only what changed and requires a watermark (see Incremental Refresh).
• watermark / merge_key — how an incremental table finds and applies new rows (see Incremental Refresh).
• freshness.window — the staleness objective the platform schedules against (see Freshness).

Package Manifest

Reuse scope and the refresh cadence are declared once for the whole package in publisher.json (the same manifest described in Publishing):

{
  "scope": "package",
  "materialization": { "freshness": { "window": "24h", "fallback": "live" } }
}

• scope: package (the default) — a derived copy is reused across the package's versions whenever they define the same thing. Maximal reuse, lowest cost.
• scope: version — each version keeps its own copies, with no cross-version reuse. Choose this when you want to own an exact rebuild schedule for a version.

Declare either a freshness objective or an explicit materialization.schedule, never both. A fixed schedule is the power-tier option and is only valid under scope: version.

Incremental Refresh

By default (refresh="full") every refresh recomputes the whole table. For a large, append-mostly source — a fact table that grows daily — that means re-reading years of data to add one day. Declare refresh="incremental" instead, and each refresh reads only the rows that are new since the last build and applies them to the existing table:

#@ persist refresh="incremental" watermark="order_date"
source: daily_revenue is orders -> {
  group_by: order_date
  aggregate: revenue is amount.sum()
}

The whole declaration lives on the #@ persist tag — the source body is exactly what you would write with no persistence at all, queries against the source are unchanged, and search indexes are already incremental with no declaration needed.

The three keys form a chain — merge_key requires watermark, and watermark requires refresh="incremental" — and publishing fails with a targeted error if any link is missing, if a named dimension doesn't resolve to an output column, or if it names a measure. You find out where you declared it, not by watching a table that never advances.

Which shape is yours

A rollup or an append-only fact — no merge_key. A row's order_date or ingested_at never changes, so each refresh replaces its date range outright. This also picks up rows deleted upstream within the refreshed range:

#@ persist refresh="incremental" watermark="ingested_at"
source: events is conn.table('raw.events') -> {
  select: ingested_at, event_id, user_id, payload
}

A mutable table — watermark plus merge_key. updated_at moves when a row changes, so the platform needs id to find and replace the stale copy:

#@ persist refresh="incremental" watermark="updated_at" merge_key="id"
source: accounts is conn.table('raw.accounts')

No monotone dimension — leave refresh unset. A small lookup table overwritten wholesale upstream has nothing to order rows by; full-copy is a supported, cheap answer.

Limits and repairs

Incremental trades completeness for cost, and two gaps are disclosed at publish rather than solved:

• Late data. A row that arrives with a watermark value below the range already covered is never picked up automatically.
• Hard deletes. A row deleted upstream can't appear in any delta, so a merge_key source retains it. Prefer soft deletes — a tombstone flag arrives as an ordinary update — and keep the flag in the persisted source's output, filtering it in consuming views instead.

Both are repaired the same two ways: correct the row upstream and advance its watermark so the next refresh applies it, or force a full rebuild with a Rerun from the package page (or forceFullRebuild on the runs API). Changing the model always triggers a full rebuild automatically — a delta is never applied across a logic change.

Pre-Aggregations

A pre-aggregation makes a measure fast and cheap at coarse grains. You mark a hot measure and its rollup grain; Credible maintains a rolled-up copy and silently answers coarse queries from it. You never hand-write a rollup source, and no query ever names one — routing is a property of the platform, not a judgment the caller (or the AI agent) makes per query:

#@ preaggregate grain="order_time.day, category"
measure: total_revenue is amount.sum()

• grain — required: the dimensions the rollup stores. A query is served from the rollup when everything it groups by and filters on is covered by the grain — a coarser truncation of a stored time dimension (order_time.month over a day grain) counts. Anything else falls back to the base source and runs live: "unsupported" and "unaccelerated" are the same, correct outcome.
• #@ -preaggregate pins a measure to the base even when a covering rollup exists — the escape hatch for a consumer that can't tolerate the rollup's freshness.

Routing is correctness-aware — the platform never serves a silently wrong number from a rollup:

• Additive measures (sum, count, min, max) and avg are re-aggregated from the rollup at any covered grain.
• Non-additive measures (count(distinct), median, percentiles) can't be correctly re-aggregated to a coarser grain, so a rollup answers them only at exactly its declared grain — other grains run live. You're told this once, as a publish-time warning on the measure.

Everything else on this page applies unchanged. Measures declared at the same grain pack into a single rollup table, built by the same runs — from the materialized table when the base source is also #@ persist-ed, straight from the warehouse when it isn't (often the right choice: a rollup is frequently worth maintaining when a full copy of the base is not). Rollups share their base's freshness window — a rollup is never fresher than the table it was built from, and a stale rollup is skipped in favor of the base, never served — and they're reused and garbage-collected like any other derived copy.

For an expensive source with hot measures, the recommended pattern extends to a three-part bundle: persist the source, index its searchable dimensions, and pre-aggregate its hot measures — three annotations, with Credible sequencing all of it.

Freshness

freshness.window is an objective, not a fixed refresh time: it tells Credible how stale the derived copy is allowed to get, and the platform schedules refreshes to meet it. This lets Credible batch work, run off-peak, and skip a refresh any recent publish or on-demand run already covered.

The fallback setting controls what a query does when a materialized table is older than its window — live runs the query against the warehouse instead of serving stale data.

Search indexes surface their staleness on the version page and in search and retrieval responses, so the agent can tell when suggestions come from an older snapshot.

Builds and Refreshes

• On publish — Credible builds every persisted source and search index for the new version automatically.
• On demand — trigger a Rerun from the package page (or the runs API) to force a rebuild. You can rerun a whole version, or a single source or dimension — optionally including its upstream persisted sources.
• On a schedule — the platform refreshes derived copies to meet their freshness objectives.

The version page shows Materialized sources and Indexed dimensions side by side, each with a simple status and its build and refresh history, so you can see at a glance whether a version is fully built.

Storage Reclamation

Credible garbage-collects every unused derived copy — a materialized table or index is kept only while an unarchived package version references it. Archiving a version releases its references, and any copies no longer referenced by another version are reclaimed automatically. So the way to keep storage costs down is to archive package versions you no longer use — auto-archive (on by default) does this for you on a retention window you control.

Next Steps