Reading the run-mode UI¶
Three surfaces tell you what's going on during a run.
The DAG¶
The pipeline view in run mode renders every step as a node, colored by state. During an in-flight run:
| Color | Meaning |
|---|---|
| Green | Step completed successfully in this run, OR has valid prior-run data that's not part of this re-run's scope. |
| Blue | Step currently executing. |
| White | Step is queued/stale: the dispatcher hasn't reached it yet (full re-run), OR the cascade includes it but it hasn't started, OR its prior data is potentially stale because something it depends on (FK / shared writer / artifact wiring) is being re-run. |
| Grey dotted | Step skipped — either because it's not in the cascade scope, or it's intentionally inert (e.g. a step with no expected tables). |
Hover any node to see its expected tables. Right-click for a context menu:
- etl_map steps → ↻ Rerun (cascade) with a preview showing which other steps will also re-run (shared-table writers, transitively).
- post-automation → ↻ Re-run this step (single-step pipeline_run).
- pre-automation has no right-click rerun — equivalent to ▶ Run pipeline.
The ribbon bar¶
The big bar next to the timer shows overall progress as a percentage of
"the work this run intends to do" — i.e. everything in will_run_set.
For a full re-run that's every step; for a cascade re-run that's just
the cascade members; for a single-step re-run that's just one step.
The number is honest in the sense that pre-auto and post-auto have a
synthetic rows-equivalent budget (so the bar moves during their silent
vocab-load / index-restore / era phases); they contribute their share
of the percentage but the strip cards beneath show just (82%) rather
than a misleading "822k / 1M rows".
The per-step modal¶
Click any step node to open its modal. Three branches by status:
Running¶
- For etl_map steps: a per-expected-table progress list with rows loaded vs scan-based / calibrated estimate, plus a "current table" pointer showing what the engine is mapping right now. Click a table row to open its engine log (live-tailing while the step runs — auto- refreshes every 2 s, preserves your scroll position so you can read older lines mid-stream).
- For pre-automation with vocab loading: "Loaded records" table fills
in as each vocab table completes (
concept_ancestor: 38,832,744 etc.), then an "Indexes: N of 17 added" progress bar for the index-restore phase. - For post-automation:
observation_period(one row per person — ~1,162 on the synthea-1k set) on both CSV and RDBMS, plus, on an RDBMS target only, era counts as each task completes (condition_era: 46,187 records,drug_era: 9,500,dose_era: 26,661) and preceding-visit-IDs counts.
Finished (success / partial / failed / skipped)¶
- Per-table outcomes table: rows_loaded, rejected (with a sample message hinting at why), per-table error kind.
- A "↻ Rerun cascade" button on etl_map steps that re-opens the preview + confirm flow.
- A download link for the per-step engine log (gzipped, named after the step + run id).
Queued / not in scope¶
- Static configuration (current RiaH filename, expected tables) and a message explaining why the run hasn't touched this step.
Run history¶
The "Recent runs" table beneath the DAG auto-updates as the in-flight run progresses — no page reload needed. Each row links to a per-run detail page with the full step-by-step outcome history and downloadable engine logs.
That's the whole tour. From here, look at the Concepts section if you want to understand WHY the cascade walks shared-table writers transitively, or Reference for the bundle formats and HTTP endpoints.