Databricks
Connect a Databricks SQL warehouse to Credible
To connect Databricks to Credible you need a SQL warehouse and credentials for either a personal access token (PAT) or an OAuth machine-to-machine (M2M) service principal.
Prerequisites
- A Databricks workspace with Unity Catalog enabled
- Permission to create or use a SQL warehouse
- Permission to create a personal access token, or workspace admin access to create an OAuth service principal
Step 1: Find or Create a SQL Warehouse
Credible queries Databricks through a SQL warehouse.
- In the Databricks workspace, go to SQL Warehouses in the left sidebar
- Use an existing warehouse or click Create SQL warehouse
- Give it a name (e.g.,
credible-warehouse) and pick a size. ASmallServerless warehouse is a good starting point - Click Create
Once the warehouse exists, open it and note the following from the Connection details tab:
- Server hostname: looks like
dbc-xxxxxxxx-xxxx.cloud.databricks.com. This is the Host field in Credible. - HTTP Path: looks like
/sql/1.0/warehouses/<warehouse-id>. This is the HTTP Path field in Credible.
Step 2: Create Credentials
Choose one of the following authentication methods.
Option A: Personal Access Token (PAT)
Best for getting started or for connections owned by an individual user.
- In Databricks, click your user avatar (top right) > Settings
- Go to Developer > Access tokens > Manage
- Click Generate new token
- Add a comment (e.g.,
credible) and choose a lifetime - Click Generate and copy the token. You cannot view it again
Treat the PAT like a password. Anyone with the token can act as you in Databricks.
Option B: OAuth M2M Service Principal
Recommended for production. The connection is owned by a service principal rather than a person, so it survives user offboarding.
- In Databricks, go to Settings > Identity and access > Service principals
- Click Add service principal, name it (e.g.,
credible-sp), and create it - Open the service principal and go to the Secrets tab
- Click Generate secret and copy both the Client ID and Client Secret. The secret is shown only once
- Grant the service principal access to your SQL warehouse and Unity Catalog data:
- In SQL Warehouses, open the warehouse, click Permissions, and grant the service principal Can use
- In Unity Catalog, grant the service principal
USE CATALOG,USE SCHEMA, andSELECTon the catalogs/schemas/tables you want Credible to read
Step 3: Grant Data Access
Whichever identity you chose (your user for a PAT, or the service principal for OAuth), it needs Unity Catalog privileges on the data you plan to model. At a minimum:
USE CATALOGon the target catalogUSE SCHEMAon each schemaSELECTon each table or view
You can grant these in the Databricks Catalog Explorer or via SQL:
GRANT USE CATALOG ON CATALOG main TO `credible-sp`;
GRANT USE SCHEMA ON SCHEMA main.sales TO `credible-sp`;
GRANT SELECT ON SCHEMA main.sales TO `credible-sp`;Step 4: Connect in Credible
- Go to
your-org.app.credibledata.com - Navigate to your environment
- Click Add Connection
- Select Databricks
- Fill in:
- Host: the workspace hostname from Step 1
- HTTP Path: the warehouse HTTP path from Step 1
- Default Catalog: the Unity Catalog to use by default (e.g.,
main) - Default Schema: optional, the default schema within the catalog (e.g.,
default) - Access Token: paste the PAT from Step 2A, or
- OAuth Client ID + OAuth Client Secret: paste the credentials from Step 2B
- Click Test Connection to verify connectivity
- Click Connect
Provide either a personal access token or OAuth client credentials, not both.
From the CLI
To add the connection with cred add connection, put an entry like this in the JSON file:
[
{
"name": "warehouse",
"type": "databricks",
"databricksConnection": {
"host": "dbc-xxxxxxxx-xxxx.cloud.databricks.com",
"path": "/sql/1.0/warehouses/<warehouse-id>",
"defaultCatalog": "main",
"defaultSchema": "default",
"token": "..."
}
}
]For an OAuth service principal, use oauthClientId and oauthClientSecret instead of token. host, path and defaultCatalog are required.
Troubleshooting
| Problem | Cause | Fix |
|---|---|---|
Invalid access token / 401 Unauthorized | The personal access token has expired or was revoked | Generate a new one in Databricks user settings |
PERMISSION_DENIED on a table | The user or service principal is missing USE CATALOG, USE SCHEMA or SELECT | Grant the missing privilege in Unity Catalog |
| The connection times out | A Serverless warehouse may be starting, or a classic warehouse is stopped | Open the warehouse in Databricks and check that it starts |
Catalog not found | Default Catalog doesn't match a catalog the identity has USE CATALOG on | Correct the catalog name, or grant USE CATALOG |