Synthmas — values and dates¶

The observation/measurement/exposure tables are where the engine's value-handling and date-arithmetic patterns get a workout. Three techniques recur:

Mode-dispatch with Selector — picking which source field is meaningful for this CDM field, based on a type or kind column.
Date arithmetic for end-dates — when the source has a start date and a duration but no explicit end date.
Pipeline regex parsing — extracting structured sub-fields out of a single source string (UDI codes).

`observation` — mode dispatch on `:type`¶

The OMOP observation table has separate fields for numeric and string values: value_as_number and value_as_string. The Synthmas source has one value column plus a type column that says which kind of value this row carries.

value_as_number:
  logic: Selector(:type, "=", "numeric", :value)

value_as_string:
  logic: Selector(:type, "=", "text", :value)

Selector here is a gate: "if :type equals 'numeric', return :value; otherwise return nothing (the field stays unmapped for this row)". Both fields look at the same source value, but only one is populated per row.

One source CSV, two CDM tables: how `observations.csv` splits¶

observations.csv is the source for both observation and measurement. The RiaH wires up two table-to-CDM arrows:

observations.csv ──▶ observation
observations.csv ──▶ measurement

Each arrow runs as its own mapping pipeline. Both pipelines read every row of observations.csv; what differs is which rows each one keeps. The observation pipeline uses the Selector-based gates shown above — text-typed rows land, numeric rows leave the value-fields empty. The measurement pipeline takes the opposite slice, but expresses it differently: a single rule on the chain that rejects non-numeric rows outright:

# On the observations.csv → measurement.value_as_number arrow:
logic: Assert("numeric")

Assert on a chain that's evaluated against the row's :type field throws when the source value isn't "numeric". A thrown rule drops the whole row from the pipeline — the engine won't write a measurement record for it. The net effect: observation gets the text rows, measurement gets the numeric rows, both from the same input file.

The same shape — two arrows, two pipelines, one filter on each — shows up any time a source table needs to fan out by a category column. The thing to remember is that both pipelines read every source row independently; the engine doesn't optimise out the "already-routed" rows. That's why measurement's Records: counter in run.log climbs noticeably higher than its final Inserted line — the Assert-rejected rows still get read and counted (see the FAQ).

`measurement.value_as_concept_id` — pick between two source fields¶

A different Selector pattern shows up on measurement: pick between two source value fields based on a description column.

value_as_concept_id:
  logic: Selector(:description, "=", "QOLS", :qols_value, :qaly_daly_value)

If the row's description is "QOLS", use the :qols_value field; otherwise use :qaly_daly_value. This is a clean way to handle source schemas where the meaningful column depends on which kind of record this is.

Filtering before vocabulary lookup¶

The same measurement field also wants the standard concept id mapped from a LOINC code in the source — but the source includes a few rows with synthetic codes (QOLS, QALY, DALY) that are not real LOINC. Filter them out before VocabMap:

measurement_concept_id:
  logic: AssertNot("QOLS")
  logic: AssertNot("QALY")
  logic: AssertNot("DALY")
  logic: VocabMap("LOINC")

The chained AssertNots reject the synthetic codes per row (logging them but not failing the whole table), so only real LOINC values reach VocabMap.

`drug_exposure_end_date` — pipeline arithmetic with `Selector` fallback¶

The drug-exposure source has a start date and a stop date, but stop is sometimes null. When it's null, derive the end as start + interval, where interval is set elsewhere in the chain.

drug_exposure_end_date:
  logic: Add(:start, :interval) => :end_date
  logic: Selector(:stop, "<>", NULL, :stop, :end_date)

Read top-down:

Add(:start, :interval) => :end_date — Add sums a date and an integer-seconds interval into a new date. Because the chain uses the pipeline form => :end_date, the result lands in the named binding rather than the chain's primary output. See DSL Guide → Pipelines.
Selector(:stop, "<>", NULL, :stop, :end_date) — if :stop is non-null, use it; otherwise use the computed :end_date.

This is the "computed default" pattern: prefer the source value when it's there, fall back to a derivation when it isn't.

`device_exposure_end_date` — fixed interval¶

Devices in Synthmas don't have a stop date at all, so the end is just start + a fixed 30 days. The new-mode parser supports duration literals (30d, 2h, 1m, …), which collapse the whole thing into one line:

device_exposure_end_date:
  logic: Add(:start, 30d)  // 30-day default exposure window

30d resolves to "30 days expressed as integer seconds" at parse time — no intermediate binding, no separate Days() call. The older shape

logic: Days(30) => :interval
logic: Add(:start, :interval)

still works in old mode. Use whichever reads more clearly to you; the duration-literal form is the easy readability win.

Device UDI parsing — pipeline regex extraction¶

Devices also have a UDI (Unique Device Identifier) column — something like (01)0345678123(10)BATCH42(11)241015(17)270101(21)SN9876. The CDM stores the UDI verbatim in device_source_value, but the engine can also pick out the named subfields with a series of Regex extractions:

device_source_value:
  logic: :udi => Regex(/\(10\)[0-9]+/) => :lot
  logic: :udi => Regex(/\(11\)[0-9]+/) => :date
  logic: :udi => Regex(/\(17\)[0-9]+/) => :exp
  logic: :udi => Regex(/\(21\)[0-9]+/) => :serial
  logic: Concat(:lot, :date, :exp, :serial)

Each Regex extracts one parenthesised UDI segment into a named binding (:lot, :date, :exp, :serial). Concat then joins the four into one string for the source-value field.

The standard concept id uses a different pattern — one regex, followed by SubString:

device_concept_id:
  logic: Regex(/\(01\)[0-9]+/)
  logic: SubString(4)
  logic: AsString()
  logic: VocabMap("SNOMED")

Regex returns the matched substring; SubString(4) drops the leading (01) prefix; VocabMap looks up the resulting bare GTIN as a SNOMED code.

`AsFloat` with a default¶

Numeric source fields that may be empty get an explicit default at the type cast:

value_as_number:
  logic: AsFloat(0.0)

Without the default, an empty string would fail the cast and skip the row. With AsFloat(0.0), an empty string becomes 0.0. See AsFloat.

Recap¶

The patterns introduced here:

Selector(:flag, "=", value, :a, :b) — pick between fields based on a flag column.
Chained AssertNot to filter values before a lookup.
Add(:date, :interval) => :end_date — pipeline arithmetic, result into a named binding.
Days(N) => :interval — duration literals stored in a binding.
Regex(/.../) => :name — extract subfields with regex; Concat to recombine.
Type casts with defaults (AsFloat(0.0), etc.).

Next: Derived tables — death and observation_period, populated via LeftJoin and cross-table lookups rather than direct source-table mappings.

Synthmas — values and dates¶

observation — mode dispatch on :type¶

One source CSV, two CDM tables: how observations.csv splits¶

measurement.value_as_concept_id — pick between two source fields¶