Migrate from Databricks Metric Views

What to export from Databricks, how Unity Catalog metric view sources, joins, dimensions, measures and filters map to Malloy, and how to rebuild them with the modeling agent, for data engineers who maintain metric views in Databricks

Unity Catalog metric views move into Credible through the modeling agent. The agent reads the exported YAML as reference material and drafts the Malloy model from it, checking its proposals against your data. Expect to review the translations more closely than for Looker or Power BI, which have dedicated skills.

What to Export

  • Metric view definitions: the YAML of each metric view. If you don't keep it in source control, get it from the workspace with DESCRIBE TABLE EXTENDED <catalog>.<schema>.<view> AS JSON.
  • Governance you rely on: a list of the Unity Catalog grants, row filters and column masks on the underlying tables. These are not part of the YAML.
  • Usage: the dashboards and Genie spaces that query the metric views, to set the scope.

How Databricks Concepts Map to Malloy

DatabricksMalloyNotes
source (a table)source: over connection.table('catalog.schema.table')
source (a SQL query)A query-based source, or a table built upstream
Top-level filterwhere: on the source
joinsjoin_one:Set the join condition from on or using
dimensions (name, expr)dimension:
DATE_TRUNC('MONTH', d)d.month
measures (name, expr)measure:
SUM(CASE WHEN ... THEN x END)sum(x) { where: condition }Filtered aggregate
COUNT(DISTINCT x)count(x)count(x) is distinct in Malloy
MEASURE(m) in a queryA plain reference to the measure
comment#(doc) tag
Format metadataRender tags such as # currency, # percent
Grants, row filters, column masks#(authorize), #(access_filter), private:Not in the YAML; re-create by hand

Steps

  1. Set up a local modeling directory with cred init (Cred CLI). Save the YAML in a subfolder such as databricks/ and add it to .credignore.
  2. Ask the agent to build the model, naming the folder and the domain: "Build a model of our orders domain using the metric views in databricks/ as prior art."
  3. Review the proposed sources, joins and fields.
  4. Compare results with SELECT <dimension>, MEASURE(<measure>) FROM <view> GROUP BY ALL in Databricks against the same query on the Malloy model, as described in Prove Parity.
  5. Publish.

What Needs Manual Review

  • Spec version. The metric view YAML specification is versioned and still changing. Check the version in each file and confirm newer features (such as window measures) translate as intended.
  • Measures that are not simply additive. Ratios and distinct counts must be re-aggregated from their parts, not summed. Check each at more than one grain.
  • Governance. Grants, row filters and column masks stay in Unity Catalog. Re-create the ones the model needs with access rules.
  • Dashboards and Genie spaces. Not converted.

Next Steps

On this page