ETL Engine¶
A high-performance ETL toolkit for transforming arbitrary health-data sources into the OMOP Common Data Model.
The ETL Engine consumes the output of existing OHDSI tooling — Rabbit-in-a-Hat and Usagi — and turns table-to-table mappings into a fully orchestrated transformation, so data owners can focus on the data rather than the plumbing.
Get started Download Mapping language reference
What it does¶
- Reads from CSV, MariaDB / MySQL, PostgreSQL (native or ODBC), or SQL Server sources and writes to any of the same as a CDM target — every driver works on both sides and combinations can be freely mixed.
- Runs your mappings straight from the Rabbit-in-a-Hat scan-report file: you write the transformation logic as annotations in the report itself — trivial where the source data is clean, as involved as it needs to be where it isn't — instead of building and maintaining a separate ETL application.
- Resolves vocabulary mappings via Usagi files and the OMOP standard vocabulary out of the box.
- Adds an expressive mapping language for the cases that go beyond simple field-to-field copies — joins, conditional chains, vocabulary lookups, multi-sized results, virtual fields.
- Ships as a Docker / Podman image for reproducible deployment.
Where to go next¶
| If you are… | Start here |
|---|---|
| Installing the engine for the first time | Getting Started → Installation |
| Writing your first mapping | DSL Guide → Mapping Language Syntax |
| Learning by example | Examples — Synthmas walkthrough + pattern recipes |
| Looking up a specific rule | Rule Reference |
| Debugging a failed run | Troubleshooting |
Configuring etl.conf |
Configuration Reference |
Licensing & support¶
The Free tier translates a Synthmas-sized dataset end-to-end — including table rules, joins, normalisation, the lot — on a single worker thread and a fixed record budget, with no licence file. RDBMS drivers, automation, multi-thread scaling, and unbounded record counts come with the Paid tier, which ships with a support package.
About¶
The ETL Engine is built and maintained by Rook IT ApS, specialising in secure data transformation and high-performance computing for sensitive health data.