Inside the AI Analytics Engine

The modern data stack was never designed. It piled up, one workaround at a time.

The market has two answers to AI analytics so far. One camp sells a smarter workspace: an in-app agent, grounded by a proprietary semantic layer, inside a destination you log into. The other sells a faster warehouse: compute tuned for agent traffic. Both are real engineering. Both accept the layers in the modern data stack and upgrade one of them.

We think the layers are the problem. So we started over, from scratch, and built something without any: Credible, the AI Analytics Engine. You don't layer on an engine -- you build your platform around it.

One claim up front: this is not another layer, and not another data platform. It is a paradigm shift -- the same one software made thirty years ago, when hand-written assembly gave way to compilers, and compilers to managed runtimes.

This is a technical read on purpose -- a claim this big has to show its insides. But what the engine buys you is simple enough to say up front:

• You write your data's meaning down once, and the model appreciates -- the engine gets faster, cheaper, and more accurate on its own.
• Every query from every surface passes through one gateway, so the engine sees your entire workload -- a view no tool in the old stack ever had.
• Your model runs on any warehouse and is locked into none -- the same deal high-level languages gave programmers.

The rest of this post shows the engine behind those three -- what it is, how it's built, and what it replaces.

What the engine does

The AI Analytics Engine does one job: it turns raw data into answers you can trust -- for every person, agent, and application that asks.

You give it two things.

• Your data, in whatever shape your systems produce it: transaction tables, logs, spreadsheets, JSON.
• What your data means: what counts as a sale, how revenue is calculated, who's allowed to see what -- written down once as a high-level data model.

That might sound hard, but don't worry -- our agent skills help you write your data model.

The engine does the rest. It decides how to reshape the data -- what to precompute, index, denormalize, and materialize -- so the context each question needs arrives fast and cheap. It enforces who sees what. And it serves answers reliable enough for others to build on.

You own "the what". The engine owns "the how".

The AI Analytics Engine runs in a cycle that has four strokes.

• Intake: your data, as your systems produce it; your data's meaning -- scattered across docs, dashboards, SQL, and the heads of a few experts; and the questions people and agents bring to it.
• Compression: the modeling. The data's meaning is written down once as a data model, and raw events squeeze into named concepts: indexed, materialized, pre-aggregated. A question should only touch what it needs.
• Power: deliver the data and its meaning that propels your organization forward -- to AI agents, dashboards, data apps embedded in your product, and whoever asks next. One data model, delivered to every surface, produces consistent answers everywhere.
• Exhaust: the telemetry every answer leaves behind, fed back into the next intake. The more you query, the faster the engine turns.

Two domains, one transform

Your systems speak in events: order #4482 updated at 3:14 PM. Your business speaks in outcomes: September revenue in the Midwest. Same information, two shapes. The operational domain is transactional and normalized, shaped by the application that wrote it. The business domain is analytical and dimensional, shaped by the questions people ask of it.

A data model is just the transform between those two domains.

The business domain also comes with rules of its own: what counts as net revenue (recognized at completion, after returns), who sees which rows (the West team sees West), what compliance requires. Those rules are part of the business domain -- so they belong in the model, not scattered across warehouse policies, BI settings, and institutional memory.

The stack that piled up

For fifty years, that transform function was too complicated to write down and too expensive to run. Databases were slow, so you couldn't just declare the business domain and compute it on demand -- you had to precompute it, in stages, and store every intermediate shape. So the industry built transform functions by piecing together infrastructure: pipelines to run it, a warehouse to hold the intermediate results, a metrics layer to re-declare pieces of it, a catalog to document it, an orchestrator to sequence it, caches to make it bearable. Each piece made sense when it was built. Piled together, they became the modern data stack.

The clearest example is the pipeline layer. A dbt project encodes the transform the way programmers once wrote programs: by hand, one low-level step at a time, with a human sequencing the steps. That was not a mistake -- it's what you build before a compiler exists. Very few people have programmed a processor that way in decades. Sadly, data teams program their stacks that way to this day.

Plate · TSC 6802 monitor listing, 1980

Software left that world behind for two reasons. The first is speed: a human can polish one step at a time, but a compiler that holds the whole program can rethink all of it at once -- which is why compiled code eventually beat hand-tuned code. The second is bigger: people could finally read the code. By the 1990s, codebases had grown too big for any one person to hold in their head, and the industry's answer was better languages -- structure, types, interfaces -- that made a million lines readable, reviewable, and safe to change by people who didn't write them. Software made that transition thirty years ago. Data never did. A mature pipeline project is thousands of hand-written steps, and the only way to really learn what revenue means is to read them all, bottom-up, one step at a time.

AI makes this ten times worse. The newest reader of your data code is an agent, and agents read and query at machine volume -- warehouse vendors now credit agentic workloads for their growth on earnings calls. Agents also fail differently than people do. A human analyst reading a cryptic schema fills the gaps with judgment; an agent fills them with unfounded confidence. Leaving the data's meaning for the analyst to work out barely held up when the analyst was a patient person. It fails disgracefully, at volume, when the analyst is a machine no one ever taught the institutional knowledge.

Your data's meaning, written down

A transform, like any program, has a "what" and a "how". The modern data stack hand-writes both, one pipeline stage at a time. The engine only needs "the what": your data's meaning, written down. That is the data model.

That data's meaning often already exists -- in docs and decks, in dashboards and old SQL, and in the heads of a few experts. Credible connects to wherever it lives, over MCP (the open protocol that connects agents to tools), and captures it. Our modeling skills do most of the writing; your team supplies the judgment.

Here is what that looks like (see Figure 4 below). It is in Malloy, the open-source language the engine speaks. This is condensed from a production model we run on our own authorization logs. The raw input is a log table of JSON blobs, one row per event -- about as operational as data gets.

Read it in the order the data flows. The base of the ladder is the rawest shape there is: JSON strings in a log table. Private dimensions parse it. Look at what raw_user is: not a staging table, not a storage location -- a typed, named expression. In a pipeline, every rung of this ladder would be a table you schedule, store, and re-read. Here, every rung is just a name, and the compiler decides what, if anything, becomes storage.

Climb the ladder and the names get more meaningful. is_engaged_action is the company's definition of an active user -- the one sentence of business judgment that weekly and monthly active users (WAU, MAU), sessions, and retention all inherit. It is written exactly once. One rung up, external_users is WAU or MAU, depending only on the time filter. And the top of the ladder is not a metric but a full business analysis: monthly cohort retention, with the retention matrix as a view -- the kind of thing that takes a specialist a day of gnarly SQL, written in the same language as the parsing at the bottom.

The file also says what to hide. The private and public modifiers are the encapsulation software got in the 1990s and data never did. The parsing machinery is invisible to analysts, agents, and downstream sources; the model shows an interface, not its guts. The #(authorize) gates say who can query each source -- access control written next to the data it protects, not in a separate admin console. And the #(persist) and #(preaggregate) annotations are the only mention of storage in the entire file. They don't change what anything means -- they just mark which expressions are worth keeping warm: which sources become tables, which measures stay rolled up. The engine works out the build order itself: the parsed logs first, then the rollup and the retention table built from them, refreshed incrementally.

The build order still exists. Nobody had to write it.

Correct by construction, not bolted on. There is exactly one definition of is_engaged_action, so every dashboard, agent answer, session count, and cohort agrees with every other one. Malloy understands how the tables relate, so joined totals never double-count -- the fanout bugs that silently corrupt SQL results can't even be written. And every field reference is checked against the live schema at compile time, so a renamed column is a build error, not a quietly wrong dashboard.

And all of it is plain text, in Git. What the old stack spread across a pipeline repo, a metrics layer, a catalog, and a BI permission screen -- each with its own release cycle, each free to drift from the others -- is here in one file. A definition change is one diff, one review, one test run. And when it publishes, everything derived from it -- the tables, the docs, the index, the access enforcement -- moves together, as one versioned package. There is nothing to keep in sync, because there is only one thing.

Engine design

That model file is the input. Here is the engine built around it.

Every question -- from a person in a workspace, an agent calling in over MCP, an API call from your application -- enters through one gateway, where it is checked against the model's access rules and logged to a permanent audit trail. Behind the gateway, a control plane manages the assets and a stateless data plane serves the queries. Each scales and fails on its own, so administration never slows down answers.

The versioned package is the unit of deployment: read-only, copied across many workers, promoted with zero downtime -- rolling back is just pointing at the previous version. Everything the model declares ships inside it, so it tests as one unit, releases as one unit, and rolls back as one unit. This is the shape the hyperscalers taught me and the industry -- the same design lineage as Google's storage and serving systems -- applied to data's meaning instead of rows or files.

Two parts of the engine matter for the story ahead.

• The concept index is the engine's memory: the model's definitions and the data's indexed values, searchable by meaning rather than by exact name.
• The lakehouse tier is the engine's internal storage -- a SQL catalog over Parquet on object storage -- where it keeps the tables, rollups, and caches it builds for itself.

Because the engine has its own analytics query engine and columnar storage, the warehouse is optional, not required: connect data where it already lives -- transactional databases, spreadsheets, flat files -- and the engine brings the query engine and the storage. The months-of-ETL tax in front of an analytics project is gone.

The engine chooses how

The model tells the engine what the answer must be. That leaves the engine free to choose how to compute it -- the answer never changes, only the route to it. This is the real advantage of high-level languages -- and the old stack gave it up when people started writing "the how" by hand.

The engine compiles each query to optimized SQL for whatever database holds the data -- BigQuery, Snowflake, Postgres, Databricks, DuckDB -- so your model is locked into none of them. Programmers won that independence a long time ago: a program in a high-level language runs on any processor, not one vendor's chip, and that portability helped make software ubiquitous. Data teams finally get that same deal.

One annotation turns an expensive source into a real, managed table -- built safely behind the scenes, refreshed incrementally, shared across model versions instead of copied, with a reference count on every table so the engine knows exactly who can still reach it. Hot rollups are pre-computed, and any coarser question is answered straight from them. Results are cached, so repeat questions -- and agents ask a lot of repeat questions -- come back in tens of milliseconds. Search indexes are built from the engine's own copies instead of re-scanning your warehouse.

Then, on every question, the engine chooses what to serve from. If a stored copy covers your query, you get it fast and cheap; if not, the query runs live and is just slower -- never wrong. There is no orchestrator to configure and no pipeline to babysit: the annotation is the intent, the engine is the execution.

This is what managed runtimes did for programs -- nobody schedules their own garbage collection. Here the analogy is literal. Remember what a package holds: everything the model declares, versioned together, with a reference count on every table the engine built. Archive a package, and if the tables it depended on aren't referenced by any other versions -- the engine garbage collects them.

Now think about what that replaces. A data stack migration is months of detective work: which tables are still used, which jobs feed them, which dashboards break if you turn something off. In the engine, that whole project becomes a garbage collection problem -- and the engine runs the collector. You don't clean up old pipelines, tables, dashboards, or catalogs for the same reason you don't free your own memory.

The engine even picks its own database. Under the hood it runs on a fast analytics query engine over an open lakehouse format -- the lakehouse tier from the architecture diagram. Fast storage and fast execution are in there, picked by the runtime, invisible to you. In this paradigm, the fast query engine is not a product you build a stack around. It's an implementation detail.

One last choice is quiet but the most important: the engine watches every query from every consumer, and uses what it sees to decide what to optimize next.

AI on both ends

AI sits at both ends of this engine: an agent helps you write the model, and every agent that asks a question draws on it. The question is the spark -- what follows is fire: the right context, delivered to the surface that requested it.

The same annotations that drive optimization drive discovery. When you publish a package, the engine compresses the model into the concept index: every source, dimension, measure, and view; every #(doc) definition; and the actual data values of every #(index) dimension -- searchable by meaning, walled off per organization and by row-level access controls.

When a question arrives, the engine matches it by meaning, not by string -- "sports gear" finds Running Shoes and Athletic Apparel -- and returns the slice of the model the question needs, plus suggested queries grounded in governed views. Not the whole model stuffed into every prompt: the right context, retrieved.

And it's measurable -- every question is a test of whether the model surfaced the right concepts, which turns model quality into a feedback loop.

There is also the question of who keeps what the AI learns. When a platform's AI "gets better with every use," your company's knowledge piles up in the vendor's format, behind the vendor's login. In the engine, it lands as #(doc) and #(index) lines in open, versioned text files you own. We open-sourced our agent skills and MCP tools because the models got good enough that anyone can build an in-app agent -- it stopped being a moat, for us or for anyone. A platform whose main value is its in-app agent isn't wrong; it's selling something that is no longer scarce.

The engine tunes itself

Remember the gateway -- the one point every query passes through? It does more than governance. It is where the managed-runtime story finishes.

Three strokes are behind us: the data came in, the model compressed it into concepts and warm tables, and an answer went out. The old stack does a rough version of all three. What it lacks is the fourth: it learns nothing from the questions it answers, so nothing the last one taught makes the next one cheaper. Harnessing the energy in the exhaust is where this engine is fundamentally different.

A normal compiler optimizes what it can see in the source. A just-in-time compiler goes further: it watches the program run, finds the hot spots, and re-optimizes them. That is how managed runtimes eventually beat code humans tuned by hand. The engine has the same advantage, built in: because every query from every surface flows through that one gateway, the engine sees the entire workload. No tool in the old stack ever had that view. The pipeline never met the dashboard; the warehouse saw queries but not the data's meaning; the catalog saw neither.

So the engine watches itself run, and optimizes at two levels. The first is serving: query patterns decide what the engine precomputes, indexes, denormalizes, and materializes. A view a dashboard starts hammering gets kept warm; a rollup nobody hits loses its refresh budget and, eventually, its storage. Hot paths get compiled down toward storage, the way a JIT compiles hot functions down to machine code.

The second level is one no runtime ever had: the engine watches the model. It sees which questions found the right concepts and which missed; evals -- automated tests of answer quality -- score whether answers were grounded and correct. Misses become proposed fixes -- a better #(doc) here, a missing #(index) there, a dimension or view worth declaring -- surfaced as reviewable changes to the model, in Git, where a human approves them. The program improves its own source, with the pull request as the safety rail.

The gateway is the center the cycle turns around, and every query from every surface is pulled through it. That is the centripetal force -- it pulls the work together instead of letting it fly apart into separate tools. And the cycle builds with use: more queries mean more telemetry to learn from, so the model sharpens and the storage stays warm the more the engine is asked. Every turn leaves the model a little more correct, the storage a little warmer, the next answer cheaper and more accurate. Past a certain point the cycle carries itself -- and what it drives forward is the business.

This is why the paradigm compounds. A stack of hand-written steps decays: every schema change, every departure, every vendor swap chips away at your data's meaning. A declared data model appreciates: write the data's meaning down once, and the engine gets faster, cheaper, and more accurate on its own.

A stack of hand-written steps decays.
A declared data model appreciates.

What it costs

An engine has to be efficient and reliable -- that's the whole job. And agentic analytics bills in tokens and compute. The brute-force architecture -- carry the entire model in every request -- runs up both. Prompts bloat, accuracy drops, retries pile up, and you pay for all of it.

The engine attacks both bills. Concept retrieval returns only the slice of the model the question needs, so a correct answer costs a fraction of the tokens -- and correctness is itself the biggest saving, because wrong answers are what multiply retries. Queries run against the engine's internal storage -- the tables, rollups, and caches in its lakehouse tier -- instead of scanning a warehouse, so the compute bill falls too. Storage is cheap, and spending a little of it is what shrinks the other two. Spend less on tokens, burn less compute, keep the difference.

What's left of the modern data stack

Here is the scorecard: each layer of the modern data stack, and where its job went.

To be honest about the other column: the engine does not replace your operational systems, and it does not ask you to abandon what works. If you run dbt today, the on-ramp is a governed model over your existing marts -- your project is the input, not a rewrite -- and you climb from there at your own pace.

Stop stacking, start your engine

A smarter workspace or a faster warehouse: both keep the old layers and upgrade one. We took the layers out. What is left is an engine -- one language for what your data means, one runtime that owns everything else.

Write the data's meaning down once and the cycle starts turning. Your data, its meaning, and your questions come in. The model compresses them into named concepts. Answers go out to every surface -- and the engine learns from each one, warming the storage and sharpening the model for the next. There is no assembly to write, no pipeline to babysit, no drift to chase. The more you ask, the better it gets.

That is not a better data platform. It is the shift software made thirty years ago -- hand-written steps giving way to declared meaning, with a compiler and query engine owning the rest -- finally happening in data. Nobody went back to assembly. Nobody will go back to the stack.

You own "the what". The engine owns "the how". We build the engine. You build what comes next.