DuckLake
Connect a DuckLake lakehouse (a PostgreSQL catalog plus S3 or GCS storage) to Credible
DuckLake is a lakehouse format that keeps table metadata in a SQL catalog database and the data itself as files in object storage. To connect a DuckLake you already run, give Credible both: the catalog for metadata, and the storage for data.
You do not need DuckLake to persist tables: the engine keeps its own. Connect DuckLake when you want to read a lake you already have.
Prerequisites
- A catalog database holding the DuckLake metadata. PostgreSQL is supported
- An object storage location for the data: Amazon S3 or Google Cloud Storage (GCS)
- Credentials that can read both
Credible attaches a DuckLake connection read-only, so read access is enough.
Step 1: Catalog (PostgreSQL)
Have the catalog database's details ready: Host, Port (default 5432), Database Name, Username and Password, or a single connection string (postgresql://username:password@hostname:port/database).
The catalog form shows the same fields as a PostgreSQL connection, but Database TLS and Statement timeout are not applied to a DuckLake catalog.
Step 2: Storage (S3 or GCS)
Every DuckLake connection needs a Bucket URL: the data path the lake was created with, such as s3://my-bucket/path or gs://my-bucket/path.
For Amazon S3:
| Field | Notes |
|---|---|
| Access Key ID and Secret Access Key | Required. Read and list access on the bucket and prefix |
| Region | Optional, default us-east-1 |
| Custom Endpoint | Optional, for an S3-compatible store such as MinIO. Host and port, no https:// |
| Session Token | Optional, for temporary STS credentials |
For Google Cloud Storage, enter an HMAC Access Key ID and Secret Access Key for a service account with Storage Object Viewer on the bucket. Create one under Cloud Storage > Settings > Interoperability in the Google Cloud Console.
Step 3: Connect in Credible
- In the connection form, choose DuckLake
- Under Catalog, choose PostgreSQL and enter the details from Step 1
- Under Storage, choose S3 or GCS, enter the Bucket URL, then the credentials from Step 2
- Click Test Connection, which checks that both the catalog and the storage are reachable
- Click Connect
From the CLI
To add the connection with cred add connection, put an entry like this in the JSON file:
[
{
"name": "lake",
"type": "ducklake",
"ducklakeConnection": {
"catalog": {
"postgresConnection": {
"host": "catalog.example.com",
"port": 5432,
"databaseName": "ducklake",
"userName": "credible_readonly",
"password": "..."
}
},
"storage": {
"bucketUrl": "s3://my-bucket/path",
"s3Connection": {
"accessKeyId": "...",
"secretAccessKey": "...",
"region": "us-east-1"
}
}
}
}
]For GCS storage, use gcsConnection with keyId and secret in place of s3Connection.
Troubleshooting
| Problem | Cause | Fix |
|---|---|---|
| The catalog connection fails | Wrong PostgreSQL host, port or credentials, or the catalog database isn't reachable from Credible | Check the catalog details. See Network Access |
| Storage access is denied | The bucket URL is wrong, or the key can't list and read objects under that prefix | Check the bucket URL and the key's access. For an S3-compatible store, set Custom Endpoint |
| Temporary credentials expired | STS credentials from a Session Token are short-lived | Generate new ones, or use a long-lived key |