Steps & RiaHs

A step is one unit of mapping work inside a pipeline. The actual mapping rules live in a RiaH — a gzipped JSON file produced by Rabbit-in-a-Hat (the OHDSI tool that lets you author CDM mappings visually).

Step kinds

Kind	Owner	Editable
etl_map	You	Yes — create, rename, reorder, upload RiaH versions.
pre_automation	Orchestrator	Toggles only (pipeline-level vocab + create-CDM are fixed on; logging knobs etc. are env-level).
post_automation	Orchestrator	Toggle observation_period + individual eras + preceding-visit-IDs at the pipeline level.

Pre- and post-automation are created automatically when you make a pipeline; you can't delete or reorder them.

Configuring an etl_map step

Pipelines → \<your pipeline> → \<step> → Configure:

• RiaH — the mapping file. Upload a .json.gz from Rabbit-in-a-Hat (or RiaH-style .json.gz produced by any compatible tool). On upload the orchestrator parses it and surfaces:
  • CDM version — must match the pipeline's omop_version, otherwise upload is rejected.
  • Expected tables — every CDM table the RiaH writes (derived from tableToTableMaps).
  • Produces — named StoreSequence + StoreIndexMap artifacts.
  • Consumes — cross-step GetCDMTableId(@<table>) lookups resolved against sibling steps' produces.
• Usagi files — CSV lookup tables in OHDSI Usagi format (sourceCode, sourceName, sourceFrequency, targetConceptId, mappingStatus, equivalence, statusSetBy, statusSetOn, …). Referenced by UsagiMap rules inside the RiaH.
• CSVMap files — plain CSV lookups. Referenced by CSVMap rules.

Every file is versioned — upload a new one and the prior version moves to "archived" status. Each version has Restore + Delete buttons, plus a download button so you can pull the file back out of the orchestrator to edit locally.

What "expected tables" buys you

The expected-tables list is the authoritative answer to "did this step load everything it was supposed to" — the run-mode modal shows per- table outcomes (loaded / error / missing) against it, and the orchestrator downgrades the step verdict from success to partial if any expected table came back with zero rows.

If the auto-derived list is wrong (e.g. you have a RiaH that uses InsertRecord to seed a constant-row table without tableToTableMaps), edit the list manually on the step configure page.

Multi-RiaH wiring

Most pipelines need more than one RiaH because a single RiaH can't express everything (multi-source fan-out into the same CDM table being the canonical example). The orchestrator auto-wires steps when:

• Step A has a StoreIndexMap named X and step B has a GetCDMTableId(@X, …) — that's a "consumes" edge from A to B.
• Steps A and B both write into CDM table T with named StoreSequence(T_id_seq) — they share IDs, and the dispatcher ensures A runs before B (or vice versa, whichever is consistent with other constraints).

See DAG wiring for the full mechanics.