A governed dataset end to end: Claude Code and Malloy on real CVE data

TL;DR — Our recent series of blog posts explains how to use open-source Malloy with AI agents. We've been building the pieces one at a time: what are the tools and skills, how to build the Malloy model, getting a trustworthy answer, and building a data app. In this post we use them all step-by-step on one real non-trivial dataset — public security advisories.

Introduction

In our recent series of blog posts, we explained how to use Malloy Publisher with AI agents. First we saw what skills and tools are available; then we explained how to use an agent to build a semantic model, the mechanics of how an agent turns a question into a trustworthy answer, and how Publisher data apps democratize building dashboards.

This post runs all four stages against a complex (but important) dataset: security vulnerabilities. This data is ambiguous in ways that change the answer, which makes it a good demonstration of what a governed semantic model buys you. We've put all relevant code in one repo, so you can follow along and reproduce the results.

Claude Code, Codex (and others) are in an arms race for the best agentic harness, and the features land weekly. The agents bundled into data warehouse products aren't in that race and likely won't be able to keep pace. More of the work already happens inside these general-purpose harnesses — writing code, reviewing it, shipping it — and analytics is moving in with it. That's what makes governed data a requirement rather than a nicety: the harness has no idea what your data means, and it will answer anyway. Open-source Malloy Publisher supplies that meaning over MCP, via files that you own and control:

• The Malloy model — version controlled, reviewable, easy to share and evolve.
• The method — the agentic skills in plain markdown, that you can understand and rewrite if needed to fit your needs.

New to Malloy Publisher? Start here or here to learn the foundations, then come back.

The Data

For this blog post, we use two public datasets:

• CIRCL/vulnerability-scores (Hugging Face, CC-BY-4.0), from CIRCL, Luxembourg's national CERT — security advisories aggregated from six upstream feeds.
• CISA's Known Exploited Vulnerabilities catalog (US public domain) — CVEs with confirmed in-the-wild exploitation, each with a listing date, a federal remediation deadline, and a ransomware flag.

I picked security data because 2026 made "which vulnerabilities are actually being exploited?" a question people started asking in earnest. Three disclosures:

• June 30, 2026 — Anthropic's Fable 5 spent eighteen days under US export controls after Amazon researchers found a jailbreak that had the model enumerating software vulnerabilities and, in one case, writing code to exploit one.

• July 21, 2026 — OpenAI disclosed that GPT-5.6 Sol and an unreleased sibling escaped a sandboxed evaluation through a zero-day vulnerability in package-registry infrastructure, and reached Hugging Face's production systems to retrieve a benchmark answer key.

• July 30, 2026 — A misconfiguration with an evaluation partner of Anthropic left the machines in a cyber evaluation with live internet access, and three Claude models went on to breach real production infrastructure at three organizations, one of them publishing a booby-trapped Python package to PyPI, which is one of the six feeds in the table below.

If you don't work in security, the next few paragraphs are worth it, so you can get familiar with the terminology in the data. If you are familiar, feel free to skip ahead.

CVE — Common Vulnerabilities and Exposures, the standard identifier for a specific flaw. Somebody finds a flaw, reports it, and the CVE Program issues an ID like CVE-2024-3400; from then on the whole industry refers to that flaw by that string. The middle segment is the year the ID was assigned. It is, in effect, a primary key that competitors, governments and open source projects all agreed to use — which is rarer and more useful than it sounds.

Advisories — a vendor's own writeup of a flaw. When a vulnerability affects their users, GitHub, Red Hat, Cisco, the Python packaging ecosystem and CISA's industrial-controls team each publish their own writeup, under their own identifier scheme, several of them describing the same underlying CVE from different angles:

CPE — Common Platform Enumeration, the standardized naming system for affected software and versions. A CVE points at software through a structured string:

cpe:2.3:a:google:chrome:134.0.6998.88:*:*:*:*:*:*:*

After cpe:2.3 the fields are part (a = application), vendor (google), product (chrome) and version (134.0.6998.88). Here's why it gets messy: a CPE identifies one release, so a flaw present in Chrome 134.0.6998.80 through .89 isn't one row, it's ten. A vulnerability affecting many releases produces hundreds. That's why affected holds around 2.5 million rows for about 255,000 advisories — roughly ten per CVE on average, and several thousand for the worst offender — and why counting rows there counts releases, not vulnerabilities.

CVSS — Common Vulnerability Scoring System, which compresses a flaw's severity into a number from 0.0 to 10.0, with 9.0 and above conventionally called Critical. Except CVSS isn't one thing: versions 2.0, 3.0, 3.1 and 4.0 are all in active circulation, designed at different times to capture different notions of severity, and they often disagree about the same flaw. Coverage is also partial: across the full set of CVE records, v3.1 reaches about 60%, v2.0 about half, v3.0 under a fifth and v4.0 under a tenth.

KEV — Known Exploited Vulnerabilities, CISA's catalog of CVEs with reliable evidence of exploitation in the wild. The distinction that matters: CVSS estimates how bad a flaw could be; KEV records that someone is actually using it. It holds under two thousand entries against a catalog of more than 300,000 CVEs, and it's a curated list of what CISA confirmed and chose to publish — a floor on exploitation, not a measurement of it.

To summarize all these terms in one sentence: one CVE, described by several advisories, affecting many product versions, scored under several severity scales (that disagree), and sometimes listed as exploited.

Setting up Publisher with our Data

First, let's clone the repo, install the skills and download the data:

git clone https://github.com/credibledata/cve-malloy-demo
cd cve-malloy-demo
npm install          # the Malloy agent skills -> .claude/skills/
python3 prep.py      # ~290MB download, once; writes ~37MB of Parquet

The skills teach the agent to use the tools well — the modeling workflow, the analysis checks, the data-app build recipe. npm install brings them in from @malloy-publisher/skills, and they land as plain markdown in .claude/skills/, so read them and change them to fit your business.

prep.py deliberately keeps all six feeds and every product row. What to exclude is a modeling decision, as we see later.

Start Publisher and connect the tools via MCP

The tools an agent uses to build and use a model are Publisher's tools, so Publisher comes up before there's anything to serve.

npx @malloy-publisher/server --init --watch-env security --port 4000 --host 127.0.0.1

Keep --host 127.0.0.1. Publisher has no authentication of its own, so without it the server binds to every interface and anyone on your network can query the data.

Then start Claude Code in the same folder. The repo ships an .mcp.json, so the malloy_* tools are there from the first prompt. Start the server first — a session reads that file only at startup.

Now with the set-up behind us, we're ready to get started with the fun part.

Stage 1 — Build the model for the CVE data

Skills: malloy-modeling orchestrating malloy-discover, malloy-scope, malloy-define, malloy-model.

We start by creating cve/cve.malloy — the Malloy model for the data (a final version — for reference — lives in reference/, which the repo blocks the agent from reading).

Start Claude Code and say:

model my data

The modeling post walks through the eight phases that you will see happen here too. It may take a while, and it likely stops and asks you things.

The discovery phase is worth watching on this data, because it turns up something that's not obvious: advisory_id looks like a key — it is unique within cvelistv5, github, csaf_redhat and pysec. It is not unique in the table as a whole: csaf_cisa repeats 1,706 ids and csaf_cisco repeats 926. Join on it as a key across all six feeds and you quietly fan out. The agent proves cardinality per feed rather than trusting the column name, and that finding leads directly to two decisions it can't make for you:

1. Which feed counts as a vulnerability? We chose cvelistv5 only. It's the authoritative CVE record; the other five re-describe the same flaws under their own IDs and can't be reliably deduplicated. Counting all six roughly doubles the Critical total. We keep those feeds reachable through a separate source, but exclude them from the governed vulnerability count.
2. Which CVSS scale wins when a CVE carries more than one? We chose the newest available of v4.0, v3.1, and v3.0, and exclude v2.0 entirely. v2.0 uses a different scale with no Critical band, so mapping a v2.0 score into one would invent a rating the scale never assigned. Records with only v2.0, or no score at all, go into Unrated, which is not a severity rating.

Those are the key places where the data can expose the ambiguity, but human judgment still has to define the intent and the meaning.

The workflow writes those decisions into the model next to the definitions they affect. In this model, for example, the CVSS precedence and severity definition live on the base advisories source, while the definition of a vulnerability is enforced structurally by the vulnerabilities source:

source: vulnerabilities is advisories extend {
  where: advisory_source = 'cvelistv5'

  join_one: kev on advisory_id = kev.cve_id
  join_many: affected on advisory_id = affected.advisory_id
  ...
}

The result is one file, cve/cve.malloy. It starts with three base sources — advisories, kev, and affected — that describe each table on its own. On top of those it defines two analysis sources at different grains: vulnerabilities, one row per CVE, and affected_products, one row per vulnerability/product-version pair. The vulnerabilities source carries the cvelistv5 scope filter and joins in exploitation and affected-product data; the other five advisory feeds remain available through the underlying advisories source for coverage analysis.

Stage 2 — Lock it

Skills: malloy-model (the curate phase), plus the compile and reloadPackage tools.

This is something the earlier posts didn't cover, and it's where a documented model and a governed one behave differently.

The #(doc) strings make the governed definition easy for an agent to find — that's what getContext retrieval is for. But for an LLM-powered AI agent, easy to find is not the same as hard to avoid. Nothing so far stops a query from reading cvss_v3_1 directly, and cvss_v3_1 >= 9.0 is a perfectly reasonable thing for an agent to write (which of course results in a different answer).

So now — let's give Claude Code one more instruction:

Now make those decisions binding in the model. Prevent queries and source extensions from directly using the raw per-version CVSS score fields, while keeping the governed cvss_score available. Keep useful alternative severity policies available only as clearly named and documented measures. Compile and reload the model, then verify that querying a raw CVSS column fails while the governed critical_count still works.

Malloy does this with access modifiers — three keywords controlling what the outside world may reference. If you learn one piece of Malloy syntax from this post, learn these:

They go inside an include { ... } block. They're still experimental, so the file turns them on at the top with ##! experimental.access_modifiers.

One exception matters, because this model depends on it: the extend block immediately following an include can still see the fields that include made private — later extensions cannot. That's what lets advisories define its alternative scoring measures over the raw columns while nothing downstream can reach them.

Stage 3 — Ask It

Skills: malloy-analysis, malloy-analysis-pitfalls, malloy-phrase-detection, malloy-queries.

Now put the analysis skills to work on a question with real traps in it:

Which vendors have the most critical vulnerabilities?

The result is as follows:

Scope is unchanged: distinct CVEs (not CPE rows), Critical = CVSS v3.0/v3.1/v4.0 ≥ 9.0 with v2.0 excluded, covering 21,994 of 30,419 Critical CVEs — the other 8,425 name no product, so no vendor ranking can see them. The three count columns are separate populations, not nested ones: only Critical CVEs is severity-filtered, Products counts every product the vendor has any CVE against, and Known exploited counts confirmed exploitation at any severity — so none of these columns is a subset of another.

The answer that comes back wears its assumptions: governed severity policy, cvelistv5 scope, unnormalised vendor tokens, KEV as a floor rather than a measurement.

The query is right there, so disagreeing costs nothing: no ticket, no thread, just read the view and argue with the definition.

Stage 4 — Ship it

Skills: malloy-html-data-apps, malloy-html-data-app-runtime.

The verified answer shouldn't evaporate when the chat ends. As we've seen in dashboards aren't dead, a Malloy package becomes a data web-app by having a public/ directory.

We've built such an example app in the reference/ folder — let's give it a try:

cp -R reference/public cve/

Then open http://localhost:4000/environments/security/packages/cve/index.html.

We see seven tiles: headline posture, severity mix, CVEs by year, critical by vendor, exploited by vendor, what the severity policy covers, and a triage queue ordered by CISA's remediation deadline.

Look at what each tile prints under itself: the exact Malloy that produced it — run: vulnerabilities -> kpis, run: vulnerabilities -> severity_mix. A reader who doesn't trust a number stops filing a ticket and starts reading the query.

The headline row is the model's kpis view unchanged: about 321,000 CVEs in scope, some 30,000 Critical, fewer than 1,500 known exploited, around 500 both, about 300 ransomware-linked. The page does no arithmetic of its own — it prints what the model returns, so the dashboard and the model cannot drift apart. That is the property that makes it safe to hand to someone who can't tell a correct dashboard from a broken one.

What makes this safe to hand to someone is the rule the app is built on, stated at the top of tiles.js — the tile catalog, split out from app.js here because there are seven of them:

No thresholds, no severity logic, no arithmetic on counts. "Critical" is defined in cve.malloy and nowhere else. What lives here is presentation: labels, column order, and the caveats the model documents.

Every tile is a constant string naming a view that exists in the model:

{
  id: "triage_queue",
  title: "Triage queue",
  query: "run: vulnerabilities -> triage_queue",
  ...
}

Which means the app cannot disagree with the model, and it cannot be built before the model exists. Stage 2 is what makes that guarantee real: a page that could reach cvss_v3_1 could quietly ship a different definition of Critical to whoever opens it. It can't, so it doesn't.

Try it

An agent read 3.1 million rows of messy public security data and came out with a model I'd defend in a review. It asked me the two things the data couldn't settle — which advisory feeds count as vulnerabilities and what "Critical" means — and those answers went into the model once, where every query has to pass through them.

That's what the tools and skills that ship with Malloy Publisher buy you, and all of it is open source.

Now try it on your own data. The recipe is above; the tutorial post has the one-command path that scaffolds a package, the server config, and the agent wiring around a CSV, Parquet, JSON, or Excel file you already have. This is where Malloy Publisher gets especially interesting: governed analytics no longer has to be limited to the people who know the BI tool or understand the warehouse. More people — and more importantly, AI agents — can ask questions while working from the same trusted definitions.

• Reproduce this post: credibledata/cve-malloy-demo — the data prep, the finished model in reference/, and the app
• Read the skills your agent ran: the skills/ directory in malloydata/publisher, or install them with @malloy-publisher/skills
• Share what you built: Malloy community Slack, and star Publisher if it earned it

When you're ready to bring the same model into production, Credible adds the operational pieces that are awkward to run yourself: per-user access control, OAuth-protected agent endpoints, managed materialization, and immutable versioning. Contact us and we'll help you get your first governed model into production.