Embedded Data
Ship CSV and Parquet files inside your packages
You can include CSV or Parquet files directly in your packages. When you publish your package, these data files are published along with your models and become queryable via DuckDB.
Embedding data files in your packages is valuable when you need to:
- Package sample data — Example datasets for testing or demos
- Build standalone models — Models that don't require database connections
- Version control data — Keep data synchronized with model changes in your package
Currently, embedded data files work best for standalone models. Support for querying embedded data alongside database connections (e.g., joining embedded lookup tables with warehouse data) is coming soon.
File Structure
Create a data/ folder in your package directory and add your files — CSV (.csv, with a header row) or Parquet (.parquet, more efficient for larger datasets):
my-package/
├── publisher.json
├── ecommerce.malloy
└── data/
├── country_codes.csv
├── product_categories.parquet
└── exchange_rates.csvReferencing Embedded Data in Models
Use duckdb.table() to reference embedded files in your Malloy models — the same syntax works for CSV and Parquet:
source: country_codes is duckdb.table('data/country_codes.csv') extend {
dimension:
country_code is code
country_name is name
region is geographic_region
}