> ## Site Index
> Fetch the site index at: https://www.credibledata.com/llms.txt
> Use this file to discover all available pages before exploring further.

# A governed dataset end to end: Claude Code and Malloy on real CVE data

> Every piece from the series — model, lock, ask, ship — applied step by step to real security vulnerability data, with the repo to reproduce it.

**TL;DR** — Our recent series of blog posts explains how to use open-source Malloy with AI agents.
We've been building the pieces one at a time: what are the tools and skills, how to build the Malloy
model, getting a trustworthy answer, and building a data app. 
In this post we use them all step-by-step on one real non-trivial dataset — public security advisories.

## Introduction

In our recent series of blog posts, we explained how to use Malloy Publisher with AI agents. First we saw [what skills and tools are available](/blog/posts/open-sourcing-malloy-agents); then we explained how to use an agent to [build a data model](/blog/posts/how-an-agent-builds-a-semantic-model), the mechanics of how an agent [turns a question into a trustworthy answer](/blog/posts/how-an-agent-turns-a-question-into-a-trustworthy-answer), and how Publisher data apps [democratize building dashboards](/blog/posts/dashboards-arent-dead).

This post runs all four stages against a complex (but important) dataset: security
vulnerabilities. This data is ambiguous in ways that change the answer, which makes it a
good demonstration of what a governed data model buys you. We've put all relevant code in
[one repo](https://github.com/credibledata/cve-malloy-demo), so you can follow along and reproduce the results.

Claude Code, Codex (and others) are in an arms race for the best agentic harness, and the features land weekly. The
agents bundled into data warehouse products aren't in that race and likely won't be able to keep pace.
More of the work already happens inside these general-purpose harnesses — writing code, reviewing it, shipping
it — and analytics is moving in with it. That's what makes governed data a requirement rather
than a nicety: the harness has no idea what your data *means*, and it will answer anyway.
Open-source Malloy Publisher supplies that meaning over MCP, via files that you own and control:

- **The Malloy model** — version controlled, reviewable, easy to share and evolve.
- **The method** — the agentic skills in plain markdown, that you can understand and rewrite if needed to fit your needs.

New to Malloy Publisher? [Start here](/blog/posts/query-any-data-with-an-agent). New to Malloy? [Start one step earlier](/malloy)
or [here](https://docs.malloydata.dev/documentation/user_guides/publishing/publishing) to learn the foundations, then come back.

## The Data

For this blog post, we use two public datasets:

- [CIRCL/vulnerability-scores](https://huggingface.co/datasets/CIRCL/vulnerability-scores)
  (Hugging Face, CC-BY-4.0), from CIRCL, Luxembourg's national CERT — security advisories
  aggregated from six upstream feeds.
- [CISA's Known Exploited Vulnerabilities catalog](https://www.cisa.gov/known-exploited-vulnerabilities-catalog)
  (US public domain) — CVEs with confirmed in-the-wild exploitation, each with a listing date,
  a federal remediation deadline, and a ransomware flag.

I picked security data because 2026 made "which vulnerabilities are actually being exploited?"
a question people started asking in earnest. Three disclosures:

- **June 30, 2026** — Anthropic's Fable 5 [spent eighteen days under US export controls](https://www.anthropic.com/news/redeploying-fable-5) after Amazon researchers found a jailbreak that had the model enumerating software vulnerabilities and, in one case, writing code to exploit one.

- **July 21, 2026** — OpenAI disclosed that GPT-5.6 Sol and an unreleased sibling [escaped a sandboxed evaluation](https://openai.com/index/hugging-face-model-evaluation-security-incident/) through a zero-day vulnerability in package-registry infrastructure, and reached Hugging Face's production systems to retrieve a benchmark answer key.

- **July 30, 2026** — [A misconfiguration with an evaluation partner](https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals) of Anthropic left the machines in a cyber evaluation with live internet access, and three Claude models went on to breach real production infrastructure at three organizations, one of them publishing a booby-trapped Python package to PyPI, which is one of the six feeds in the table below.

If you don't work in security, the next few paragraphs are worth it, so you can get familiar with the terminology in the data. If you are familiar, feel free to skip ahead.

**CVE** — *Common Vulnerabilities and Exposures*, the standard identifier for a specific flaw.
Somebody finds a flaw, reports it, and the CVE Program issues an ID like `CVE-2024-3400`; from
then on the whole industry refers to that flaw by that string. The middle segment is the year
the ID was assigned. It is, in effect, a primary key that competitors, governments and open
source projects all agreed to use — which is rarer and more useful than it sounds.

**Advisories** — a vendor's own writeup of a flaw. When a vulnerability affects their users,
GitHub, Red Hat, Cisco, the Python packaging ecosystem and CISA's industrial-controls team
each publish their own writeup, under their own identifier scheme, several of them describing
the same underlying CVE from different angles:

| feed | what it is | example ID | records (rounded) |
|---|---|---|---|
| `cvelistv5` | the CVE Program's own records | `CVE-1999-0002` | 321,000 |
| `github` | GitHub Security Advisories | `GHSA-2222-76gx-28mm` | 320,000 |
| `csaf_redhat` | Red Hat errata | `RHBA-2004:164` | 24,000 |
| `pysec` | Python packaging advisories | `PYSEC-2006-1` | 6,500 |
| `csaf_cisa` | CISA industrial control systems | `ICSA-10-147-01` | 5,400 |
| `csaf_cisco` | Cisco security advisories | `cisco-20180718-nexus-9000-dos` | 3,500 |

**CPE** — *Common Platform Enumeration*, the standardized naming system for affected software
and versions. A CVE points at software through a structured string:

```
cpe:2.3:a:google:chrome:134.0.6998.88:*:*:*:*:*:*:*
```

After `cpe:2.3` the fields are *part* (`a` = application), *vendor* (`google`), *product*
(`chrome`) and *version* (`134.0.6998.88`). Here's why it gets messy: **a CPE identifies one
release**, so a flaw present in Chrome 134.0.6998.80 through .89 isn't one row, it's ten. A
vulnerability affecting many releases produces hundreds. That's why `affected` holds around 2.5
million rows for about 255,000 advisories — roughly ten per CVE on average, and several
thousand for the worst offender — and why counting rows there counts releases, not
vulnerabilities.

**CVSS** — *Common Vulnerability Scoring System*, which compresses a flaw's severity into a
number from 0.0 to 10.0, with 9.0 and above conventionally called Critical. Except CVSS isn't
one thing: versions 2.0, 3.0, 3.1 and 4.0 are all in active circulation, designed at different
times to capture different notions of severity, and **they often disagree about the same flaw**.
Coverage is also partial: across the full set of CVE records, v3.1 reaches about 60%,
v2.0 about half, v3.0 under a fifth and v4.0 under a tenth.

**KEV** — *Known Exploited Vulnerabilities*, CISA's catalog of CVEs with reliable evidence of
exploitation in the wild. The distinction that matters: CVSS estimates how bad a flaw *could*
be; KEV records that someone is *actually* using it. It holds under two thousand entries against a
catalog of more than 300,000 CVEs, and it's a curated list of what CISA confirmed and chose to
publish — a floor on exploitation, not
a measurement of it.

To summarize all these terms in one sentence: one CVE, described by several advisories, 
affecting many product versions, scored under several severity scales (that disagree), 
and sometimes listed as exploited. 

## Setting up Publisher with our Data

First, let's clone the repo, install the skills and download the data:

```bash
git clone https://github.com/credibledata/cve-malloy-demo
cd cve-malloy-demo
npm install          # the Malloy agent skills -> .claude/skills/
python3 prep.py      # ~290MB download, once; writes ~37MB of Parquet
```

The skills teach the agent to use the tools well — the modeling workflow, the analysis checks,
the data-app build recipe. `npm install` brings them in from
[`@malloy-publisher/skills`](https://www.npmjs.com/package/@malloy-publisher/skills), and they
land as plain markdown in `.claude/skills/`, so read them and change them to fit your business.

`prep.py` deliberately keeps all six feeds and every product row. What to exclude is a
modeling decision, as we see later.

### Start Publisher and connect the tools via MCP

The tools an agent uses to build and use a model are Publisher's tools, so Publisher comes up before
there's anything to serve.

```bash
npx @malloy-publisher/server --init --watch-env security --port 4000 --host 127.0.0.1
```

Keep `--host 127.0.0.1`. Publisher has no authentication of its own, so without it the server binds to every interface and anyone on your network can query the data.

Then start Claude Code in the same folder. The repo ships an `.mcp.json`, so the `malloy_*`
tools are there from the first prompt. Start the server first — a session reads that file only
at startup.

Now with the set-up behind us, we're ready to get started with the fun part.

## Stage 1 — Build the model for the CVE data

*Skills:* [`malloy-modeling`](https://github.com/malloydata/publisher/tree/main/skills/malloy-modeling) orchestrating [`malloy-discover`](https://github.com/malloydata/publisher/tree/main/skills/malloy-discover), [`malloy-scope`](https://github.com/malloydata/publisher/tree/main/skills/malloy-scope), [`malloy-define`](https://github.com/malloydata/publisher/tree/main/skills/malloy-define), [`malloy-model`](https://github.com/malloydata/publisher/tree/main/skills/malloy-model).

We start by creating `cve/cve.malloy` — the Malloy model for the data
(a final version — for reference — lives in `reference/`, which the repo blocks the agent from reading).

Start Claude Code and say:

> model my data

The [modeling post](/blog/posts/how-an-agent-builds-a-semantic-model)
walks through the eight phases that you will see happen here too. It may take a while,
and it likely stops and asks you things.

The discovery phase is worth watching on this data, because it turns up something that's not obvious: 
`advisory_id` looks like a key — it is unique within `cvelistv5`, `github`, `csaf_redhat` and `pysec`. 
It is not unique in the table as a whole: `csaf_cisa` repeats 1,706 ids and `csaf_cisco` repeats 926. 
Join on it as a key across all six feeds and you quietly fan out. 
The agent proves cardinality per feed rather than trusting the column name, and that finding leads directly
to two decisions it can't make for you:

1. **Which feed counts as a vulnerability?** We chose *cvelistv5* only. It's the authoritative CVE
record; the other five re-describe the same flaws under their own IDs and can't be reliably
deduplicated. Counting all six roughly doubles the Critical total. We keep those feeds reachable
through a separate source, but exclude them from the governed vulnerability count.
2. **Which CVSS scale wins when a CVE carries more than one?** We chose the newest available of
v4.0, v3.1, and v3.0, and exclude v2.0 entirely. v2.0 uses a different scale with no Critical
band, so mapping a v2.0 score into one would invent a rating the scale never assigned. 
Records with only v2.0, or no score at all, go into `Unrated`, which is not a severity rating.

Those are the key places where the data can expose the ambiguity, but human judgment still has to
define the intent and the meaning.

The workflow writes those decisions into the model next to the definitions they affect. 
In this model, for example, the CVSS precedence and severity definition live on the base advisories source, 
while the definition of a vulnerability is enforced structurally by the vulnerabilities source:

```malloy
source: vulnerabilities is advisories extend {
  where: advisory_source = 'cvelistv5'

  join_one: kev on advisory_id = kev.cve_id
  join_many: affected on advisory_id = affected.advisory_id
  ...
}
```

The result is one file, `cve/cve.malloy`. It starts with three base sources — advisories, kev, and affected — that describe each table on its own. 
On top of those it defines two analysis sources at different grains: vulnerabilities, one row per CVE, and affected_products, 
one row per vulnerability/product-version pair. The vulnerabilities source carries the cvelistv5 scope filter 
and joins in exploitation and affected-product data; the other five advisory feeds remain available through 
the underlying advisories source for coverage analysis.

## Stage 2 — Lock it

*Skills:* [`malloy-model`](https://github.com/malloydata/publisher/tree/main/skills/malloy-model) (the curate phase), plus the `compile` and `reloadPackage` tools.

This is something the earlier posts didn't cover, and it's where a *documented* model and a
*governed* one behave differently.

The `#(doc)` strings make the governed definition easy for an agent to *find* — that's what
`getContext` retrieval is for. But for an LLM-powered AI agent, easy to find is not the same as hard to avoid.
Nothing so far stops a query from reading `cvss_v3_1` directly, and `cvss_v3_1 >= 9.0` is a
perfectly reasonable thing for an agent to write (which of course results in a different answer).

So now — let's give Claude Code one more instruction:

> Now make those decisions binding in the model. Prevent queries and source extensions from
> directly using the raw per-version CVSS score fields, while keeping the governed
> `cvss_score` available. Keep useful alternative severity policies available only as clearly
> named and documented measures. Compile and reload the model, then verify that querying a raw
> CVSS column fails while the governed `critical_count` still works.

Malloy does this with **access modifiers** — three keywords controlling what the outside world
may reference. If you learn one piece of Malloy syntax from this post, learn these:

| keyword | what it means |
|---|---|
| `public:` | anything querying the source can use the field. The default. |
| `internal:` | a query can't name it, but a source that extends this one can still use it in its own definitions. Good for raw columns you've replaced with a cleaner dimension. |
| `private:` | a query can't name it, and neither can an extending source. This is the one that makes a definition binding.   |

They go inside an `include { ... }` block.

One exception matters, because this model depends on it: the extend block immediately following an include can still see the fields that include made private — later extensions cannot. That's what lets advisories define its alternative scoring measures over the raw columns while nothing downstream can reach them. 

## Stage 3 — Ask It

*Skills:* [`malloy-analysis`](https://github.com/malloydata/publisher/tree/main/skills/malloy-analysis), [`malloy-analysis-pitfalls`](https://github.com/malloydata/publisher/tree/main/skills/malloy-analysis-pitfalls), [`malloy-phrase-detection`](https://github.com/malloydata/publisher/tree/main/skills/malloy-phrase-detection), [`malloy-queries`](https://github.com/malloydata/publisher/tree/main/skills/malloy-queries).

Now put the [analysis skills](/blog/posts/how-an-agent-turns-a-question-into-a-trustworthy-answer)
to work on a question with real traps in it:

> Which vendors have the most critical vulnerabilities?

The result is as follows:

| # | Vendor | Critical CVEs | Products | Known exploited |
|---|---|---|---|---|
| 1 | microsoft | 853 | 777 | 368 |
| 2 | debian | 842 | 103 | 91 |
| 3 | apple | 725 | 160 | 123 |
| 4 | google | 694 | 185 | 96 |
| 5 | adobe | 576 | 172 | 63 |
| 6 | redhat | 519 | 618 | 83 |
| 7 | qualcomm | 461 | 3,162 | 8 |
| 8 | tenda | 420 | 182 | 2 |
| 9 | fedoraproject | 392 | 17 | 79 |
| 10 | oracle | 383 | 995 | 69 |

Scope is unchanged: distinct CVEs (not CPE rows), Critical = CVSS v3.0/v3.1/v4.0 ≥ 9.0 with v2.0 excluded, covering 21,994 of 30,419 Critical CVEs — the other 8,425 name no product, so no vendor ranking can see them. The three count columns are separate populations, not nested ones: only Critical CVEs is severity-filtered, Products counts every product the vendor has any CVE against, and Known exploited counts confirmed exploitation at any severity — so none of these columns is a subset of another. 

The answer that comes back wears its assumptions: governed severity policy, cvelistv5 scope, unnormalised vendor tokens, 
KEV as a floor rather than a measurement. 

The query is right there, so disagreeing costs nothing: no ticket, no thread, just read the view and argue with the definition.

## Stage 4 — Ship it

*Skills:* [`malloy-html-data-apps`](https://github.com/malloydata/publisher/tree/main/skills/malloy-html-data-apps), [`malloy-html-data-app-runtime`](https://github.com/malloydata/publisher/tree/main/skills/malloy-html-data-app-runtime).

The verified answer shouldn't evaporate when the chat ends. As we've seen in
[dashboards aren't dead](/blog/posts/dashboards-arent-dead), a
Malloy package becomes a data web-app by having a `public/` directory. 

We've built such an example app in the `reference/` folder — let's give it a try:

```bash
cp -R reference/public cve/
```

Then open
[http://localhost:4000/environments/security/packages/cve/index.html](http://localhost:4000/environments/security/packages/cve/index.html).

We see seven tiles: headline posture, severity mix, CVEs by year, critical by vendor, exploited by
vendor, what the severity policy covers, and a triage queue ordered by CISA's remediation
deadline.

The top of the CVE posture app — five of its seven tiles. Every number comes from a named view in `cve.malloy`, printed under the tile that used it.

Look at what each tile prints under itself: the exact Malloy that produced it — `run:
vulnerabilities -> kpis`, `run: vulnerabilities -> severity_mix`. A reader who doesn't trust a
number stops filing a ticket and starts reading the query. 

The headline row is the model's `kpis` view unchanged: about 321,000 CVEs in scope, some
30,000 Critical, fewer than 1,500 known exploited, around 500 both, about 300
ransomware-linked. The page does no arithmetic of its
own — it prints what the model returns, so the dashboard and the model cannot drift apart.
That is the property that makes it safe to hand to someone who can't tell a correct dashboard
from a broken one.

What makes this safe to hand to someone is the rule the app is built on, stated at the top of
`tiles.js` — the tile catalog, split out from `app.js` here because there are seven of them:

> No thresholds, no severity logic, no arithmetic on counts. "Critical" is defined in
> `cve.malloy` and nowhere else. What lives here is presentation: labels, column order, and
> the caveats the model documents.

Every tile is a constant string naming a view that exists in the model:

```js
{
  id: "triage_queue",
  title: "Triage queue",
  query: "run: vulnerabilities -> triage_queue",
  ...
}
```

Which means the app *cannot* disagree with the model, and it cannot be built before the model
exists. Stage 2 is what makes that guarantee real: a page that could reach `cvss_v3_1` could
quietly ship a different definition of Critical to whoever opens it. It can't, so it doesn't.

## Try it

An agent read 3.1 million rows of messy public security data and came out with a model I'd defend in a review. It asked me the two things the data couldn't settle — which advisory feeds count as vulnerabilities and what "Critical" means — and those answers went into the model once, where every query has to pass through them.

That's what the tools and skills that ship with Malloy Publisher buy you, and all of it is open source.

Now try it on your own data. The recipe is above; the [tutorial post](/blog/posts/query-any-data-with-an-agent) has the one-command path that scaffolds a package, the server config, and the agent wiring around a CSV, Parquet, JSON, or Excel file you already have. This is where Malloy Publisher gets especially interesting: governed analytics no longer has to be limited to the people who know the BI tool or understand the warehouse. More people — and more importantly, AI agents — can ask questions while working from the same trusted definitions.

- **Reproduce this post**: [credibledata/cve-malloy-demo](https://github.com/credibledata/cve-malloy-demo) — the data prep, the finished model in `reference/`, and the app
- **Read the skills your agent ran**: [the `skills/` directory in malloydata/publisher](https://github.com/malloydata/publisher/tree/main/skills), or install them with [`@malloy-publisher/skills`](https://www.npmjs.com/package/@malloy-publisher/skills)
- **Share what you built**: [Malloy community Slack](https://malloydata.github.io/slack), and [star Publisher](https://github.com/malloydata/publisher) if it earned it

When you're ready to bring the same model into production, [Credible](https://credibledata.com) adds the operational pieces that are awkward to run yourself: per-user access control, OAuth-protected agent endpoints, managed materialization, and immutable versioning. [Contact us](https://credibledata.com/get-access) and we'll help you get your first governed model into production.