> ## Site Index
> Fetch the site index at: https://www.credibledata.com/llms.txt
> Use this file to discover all available pages before exploring further.

# The Semantic Layer Is an Interface, Not a Compiler

> Agents didn't make the interface unnecessary. They made it affordable.

[Paul Blankley](https://www.linkedin.com/in/paulblankley/), CTO of Zenlytic, gave
a talk at Data Day Texas called "Agents Eating The Semantic Layer," and made the
argument again on
[It's About Data](https://youtu.be/NGxcwl6D4tQ) with Matt Housley and Tony Baer.
His claim: a semantic layer is two things -- a store of business context and a
compiler that turns measures and dimensions into SQL. Agents have made the
compiler unnecessary, and keeping it now costs more than it buys.

We disagree about the conclusion, but agree on almost everything else.
"Semantic layer" has become a phrase people use to mean four different things,
and we are both tired of the noise. Context is the thing that matters most.
Agents write excellent SQL and will keep getting better, and betting against
that is a bad bet. LookML is genuinely a bad interface: rigid, hand-authored,
and unable to express questions people actually ask.

Here is where we part, and it comes down to what a semantic layer actually cost
you. Writing it by hand was the whole cost: slow, expensive, and the one person
who could write it had a queue of requests. Nearly every complaint about
semantic layers traces back to that.

Agents took that cost to zero.

So the best reason to delete the layer is the reason that just disappeared.
Paul's answer is to remove the semantic layer. Mine is that we can finally
afford it.

"Don't bet against the agents" is an argument for raising the interface, not
flattening it.

## Implementation, or contract

Here's how Paul defines it: "the thing that makes a semantic layer uniquely a
semantic layer is that query compiling step. That's the thing that's different
from a knowledge graph, or different from an ontology, or different from a Git
repo full of markdown files."

That defines a semantic layer by how it is built. He then shows that the way it
is built is limiting, correctly, and concludes the part should go.

I'd define it by what it promises the people using it. **A semantic layer is
your data's public API.** Every dashboard, notebook, app and agent that reads
your data goes through it. Define it that way and every criticism Paul makes
still lands. But none of them reach his conclusion.

I've spent my career in system design: computer architecture first, then
large-scale distributed systems. Three ideas do nearly all the work:
decomposition, interfaces, encapsulation. Split a system into parts, define the
contract between them, make each part's internals nobody else's business. First
month of a CS degree.

Data never went through any of that. Software spent fifty years building up
layers of abstraction. Data got fragmented products instead, and when the only
things you're handed are SQL and YAML files, you end up describing your
architecture in vendor names.

## Try the same argument on an API

Software engineering settled this a long time ago, so point Paul's argument at
an API instead.

Agents write excellent HTTP clients, better than most humans. So: delete your
public API. Hand every client a connection string to the production database and
let it write whatever queries it likes. To keep things honest, parse the SQL
afterward and tag whether it used the right definition of "active user."

Nobody would ship that. Not because the agent can't write the query -- it can,
first try, every time. **The API was never there because the callers were bad at writing queries.**
It's there to do five other things:

- **Encapsulate.** Callers don't depend on your physical schema, so you can
  change storage without rewriting every consumer.
- **Enforce.** Authorization happens at one gate instead of N.
- **Observe.** You get an audit trail in the language of the business rather
  than the language of the tables.
- **Optimize.** There's one place to cache, pre-aggregate, materialize, and rate
  limit -- a stable surface the callers build against.
- **Evolve.** The contract can be versioned and deprecated deliberately, rather
  than discovered in production.

Those are all jobs for the interface, not the compiler. Better agents write
better SQL. They don't do the job of a good API.

## Expressiveness was never the issue

The generality argument went like this. If you keep extending the modeling
language to cover more cases, "you're going to asymptotically approach the
flexibility of SQL itself, in which case you should just be using SQL."

I'll go further than that. On the questions this argument is about --
analytical ones -- Malloy is as expressive as SQL. It isn't trying to be the
whole of SQL: fifty years of accumulation have left SQL with a lot of cruft you
don't need in order for the language to be complete.

But expressiveness misses what languages are for. By that argument every
programming language after 1936 was unnecessary, because
[lambda calculus](https://en.wikipedia.org/wiki/Lambda_calculus) could already
compute anything that can be computed. We built them anyway for seventy
years, assembly through managed languages, and not one of them let you compute
something new. Every one was still worth building, because each one made
complexity easier to handle. That is what took software from batch payroll to
building artificial intelligence. Alan Perlis had the short version in 1982:
*"Beware of the Turing tar-pit in which everything is possible but nothing of
interest is easy."* He was writing about programming languages, forty years
before the "modern data stack," and the line fits it exactly.

And the tar-pit is getting deeper. AI is generating complexity faster than
anything before it -- more sources, more consumers, more rogue agents, more
systems that all need the same numbers to mean the same thing. The proposal is
to remove a layer of abstraction at the moment the complexity curve is hitting
an inflection point.

To be fair, "layer" should be plural. One flat abstraction over a whole business
is constraining, but the answer is to compose abstractions, not pull them out.
In Malloy, sources extend other sources, so sales, finance, product and support
can each have their own, in their own words.

What makes that work is that Malloy reads: a measure is a named line with its
definition beside it. Someone with basic data literacy can say whether it's
right without being able to write it. That is what makes a layer ownable: if
sales owns the sales layer, somebody in sales has to be able to tell you it's
wrong.

## So which language, then

If not SQL as the interface, what, and why? Not expressiveness, which the last
section just established.

**The first reason is that SQL mixes the model and the query together.** The join path, the
grain, the filter that defines "active," which revenue column is authoritative --
inlined, every time, by everyone. There is
[no way to save a *calculation* in SQL](/docs/concepts/malloy-vs-sql), only a
query or a view, and a view is the wrong shape because it freezes grain and
filters into the thing it saves. Nothing holds the model in one place, so
there's nothing to review, version or change.

"Just use SQL" therefore doesn't mean *no modeling language*. It means the model
is never written down anywhere. Every caller restates it, the copies drift apart,
and nothing tells you when they have. That is a convention nobody enforces,
which is the thing an interface exists to replace.

Paul's best example is the one I'd point at too, though we read it differently.
One of Zenlytic's customers counts active organizations with an 800-line SQL
query. It's a business question pushed all the way down into the operational
domain, rebuilding joins, definitions and business rules by hand against raw
tables. He reads it as proof that business logic doesn't fit in a model. I read
it as what happens when there's no interface to put the logic in. That's why
it's 800 lines, and why nobody can review it.

**The second reason is domain.** The relational model describes how data arrives
and is stored: customer records, transactions, foreign keys. The dimensional
model describes how a business reasons: revenue, region, quarter, grain. Those
are different domains, and the business operates in the second one.

Make SQL your interface and the names your tables happen to use become your
public contract. Your internals become the thing everyone writes code against.

It's also the real answer to Paul's best line -- "you need to give them the same
interface that humans work with." Humans don't reason in the relational domain.
They think dimensionally and then translate down to tables by hand, badly, which
is *why* the translation runs to 800 lines nobody can review. A dimensional
modeling language isn't giving the agent less than the human gets. It gives it
the vocabulary the human was already thinking in, with the translation to tables
already written.

And that translation is right for ordinary engineering reasons: it lives in a
file, gets reviewed in a pull request, and the compiler checks every field
against your warehouse schema, so a rename breaks the build instead of quietly
returning nulls.

Which is where the cost lands. **Every argument for a higher-level interface
still stands**: one definition, code you can review, internals you can hide,
one place to enforce access. Nobody argues that because agents write excellent
assembly we should get rid of compilers.

### The limit was LookML's, not the compiler's

Paul's concrete example:

> If you want to ask a question about users who took some action last year,
> what's their number of orders this year? You can't do that. That requires a
> CTE, and that just doesn't fit in LookML's query compiler model.

That is true of LookML. So let's write it in Malloy, with the two things kept
apart. **This is the model** -- the cohort, and the orders that belong to it:

```malloy
given:
  cohort_period :: filter<date> is f''

// The cohort: who used advanced search in the period asked for.
source: used_advanced_search_in_period is duckdb.table('events.parquet') -> {
  where: event_name = 'used_advanced_search' and event_at ~ $cohort_period
  group_by: user_id
} extend {
  primary_key: user_id  // one row per user
}

// Their orders, with membership already applied.
source: cohort_orders is orders extend {
  join_one: cohort is used_advanced_search_in_period on customer_id = cohort.user_id
  where: cohort.user_id is not null
}
```

**And this is the query**, with the period supplied by the caller:

```malloy
// Sent with the request: "givens": { "cohort_period": "2025" }
run: cohort_orders -> {
  where: created_at.year = @2026
  group_by: customers.region
  aggregate: order_count, total_revenue
}
```

The model fixes what things mean. The caller supplies only the knobs.

That runs, and it returns the right answer. Here is the SQL the compiler emitted
for it:

```sql
WITH __stage0 AS (
  SELECT
     base."user_id" as "user_id"
  FROM events.parquet as base
  WHERE (base."event_name"='used_advanced_search') and (base."event_at" >= CAST(TIMESTAMP '2025-01-01 00:00:00' AS date) AND base."event_at" < CAST(TIMESTAMP '2026-01-01 00:00:00' AS date))
  GROUP BY 1
)
SELECT
   customers_0."region" as "region",
   COUNT(1) as "order_count",
   COALESCE(SUM(base."amount"),0) as "total_revenue"
FROM orders.parquet as base
 LEFT JOIN customers.parquet AS customers_0
  ON customers_0."customer_id"=base."customer_id"
 LEFT JOIN __stage0 AS cohort_0
  ON base."customer_id"=cohort_0."user_id"
WHERE ((DATE_TRUNC('year', base."created_at")>=DATE '2026-01-01') and (DATE_TRUNC('year', base."created_at")<DATE '2027-01-01'))
AND (cohort_0."user_id" IS NOT NULL)
GROUP BY 1
ORDER BY 2 desc NULLS LAST
```

The CTE he says a query compiler can't produce is the first thing in the output.

And the better half: the cohort and the path to its orders are **named,
reusable sources**, not inlined subqueries. The period is a given, not a
hardcoded year, so the same cohort answers next year's question too. In
SQL that CTE lives only inside the one statement that contains it, the same
conflation problem.

Raw SQL gives you [freedom without
safety](/docs/concepts/why-malloy#freedom-and-safety). LookML gives you safety
without the freedom to ask much. You have to choose between them because LookML
is a configuration format, not because modeling forces the choice. When the model can't say something, `sql()` blocks
are the documented exit, type safety intact.

Malloy is the work of the team whose earlier language, LookML, defined the
modern semantic layer. It's their second attempt, maintained in the open by the
Linux Foundation. Paul's critique of LookML is a fair reading of why they went
and built something else.

## Why checking the SQL afterward isn't a contract

Matt asked the right question on the episode: how do you check what the agent
wrote? Paul's answer is to let it write whatever SQL it wants, then parse the
syntax tree and tag whether it used the company's official definition of gross
revenue. "Give the agent full flexibility, you parse its answers, and make that
understandable for someone who's not a SQL jockey."

That's a clever mechanism and I'd rather have it than not. But there's a kind of
error no check of this shape can catch, because of where it sits rather than how
well it's built. Zenlytic may handle this case some other way; I'm arguing about
the mechanism Paul described on the show, not their product.

Take six orders totalling **$955**, spread across twelve line items. Ask for
total revenue. An agent writes this, using the official definition correctly:

```sql
SELECT SUM(o.amount) AS total_revenue
FROM orders o
JOIN order_items i ON o.order_id = i.order_id
```

It returns **$1,445** -- overstated by 51%, because each order's amount repeats
once per line item. Run the validator over it and the syntax tree confirms the
agent used your official `SUM(amount)` definition. It **reports green.** The
error is in the grain, not the formula, and the formula is all it inspects.

The same question against a model, with the fan-out join present:

```malloy
run: orders -> { aggregate: total_revenue, item_count }
```

Returns **$955** and **12**. Correct revenue and correct item count, in one
query, across a `join_many`. Malloy computes
[symmetric aggregates](/docs/concepts/why-malloy#correct-by-construction): it
knows the join graph and the grain each measure belongs to. It doesn't catch the
fan-out and warn you. You can't write it in the first place. Catching a bad
answer and making it impossible are different things.

You keep the audit trail either way. Credible returns the compiled SQL for any
query, so a DBA can read exactly what ran. The difference is that the
definitions were fixed *before* the query ran rather than inspected after.

## Who is allowed to see this?

A syntax-tree check can confirm the agent used the company's official definition
of gross revenue. It cannot tell you **whether the person who asked was allowed
to see the data that came back.** That answer is not in the SQL, however good
the parser gets.

Put the gate on the interface and it evaluates against whoever is asking. An
agent asking on your behalf is you: your groups, your region, your rows. Hand
the same agent to a colleague elsewhere and it hands back theirs, and neither of
them can tell a gate ran. Nobody had to decide what the agent may see: you
decided what each person may see long before anyone mentioned agents.

Delete the interface and enforcement doesn't move somewhere else, it stops. Your
warehouse can still filter rows, but only in table and column names, and "reps
see only accounts in their own territory" isn't a sentence you can say in those.
It needs the joins and the business names, which is the model. A better agent
doesn't give you one.

[Adam Ribaudo](https://www.linkedin.com/in/adamribaudo/) names the same
boundary from the other side in
[The 4 Questions to Ask Before Deploying Semantic Layers in Production](/blog/posts/semantic-layers-in-production),
with the right word for it: *information hiding*. The mechanisms are ordinary
language constructs -- `public:`, `internal:` and `private:`, and
[`#(authorize)` gates](/docs/how-to/modeling/fine-grained-acls) on the sources --
the way Spring Security puts authorization on the interface. All of it rests on
one condition: the identity a gate reads has to come from somewhere the caller
can't set.

Which is the real problem with an agent assembling its own context out of
Confluence pages and old dashboards: every caller writes its own copy of the
contract, the copies drift, and nothing tells you when they have. That is how a
company ends up with two revenue numbers.

## Where this leaves us

Data teams are the bottleneck, and the people are not the reason. The stack
they were handed doesn't scale with complexity, and complexity is arriving
faster than ever. Every abstraction that would help has to be written by hand,
by the few people who can write it, which is what creates a queue.

Paul and I are not far apart on this. He thinks the compiling step is what
holds you back; I think LookML was what held him back. The answer isn't no
interface. It's an interface nobody has to write by hand.

There is a moment on the episode where we end up saying the same thing. Paul is
asked how you stay consistent without a compiler, and his answer is that you do
what a good analyst does -- go back to the canonical query. "I know how to modify
that query to make sure that it still counts what I want it to count while
grouping by something new or filtering different fiscal period."

Read that twice. "Still counts what I want it to count" is the part that must
never change. "Grouping by something new or filtering a different fiscal period"
is the part that changes with every question. That is the model and the query,
named and separated, by someone arguing you don't need a layer to separate them.
The disagreement was never whether the separation exists. It's whether it lives
in a file the compiler checks every query against, or gets re-derived out of
eight hundred lines of SQL every time somebody asks, with nothing checking the
derivation.

That's the bet we've made with the AI Analytics Engine. You, with your agent's
help, write the model -- entities, metrics, relationships, business rules,
access controls -- and the engine generates the layer that serves it, along
with the pipelines, materialized storage and indexes underneath. The model is
the input you own, in files in your repo. Everything else is generated, which
is exactly the part Paul is right to want to stop maintaining by hand. How that
works is
[Inside the AI Analytics Engine](/blog/posts/inside-the-ai-analytics-engine).

On one point we agree completely: the agent should manage its own context. The
difference is where that context ends up. In his version it accumulates inside
the agent. In ours it arrives as a diff on a model you own, and somebody has to
approve it.

In the AI Analytics Engine, the consumption layer is disposable. So is the
query engine. What your data means is the one thing you shouldn't throw away,
the argument I made in June, in
[Charting is Commoditized. Meaning Is the Product.](/blog/posts/meaning-is-the-product).

Agents didn't make the interface unnecessary. They made it affordable.

Paul, if I've read any of your argument uncharitably, tell me and I'll fix it
here. I'd rather get this right than win it.