Mapping Language Syntax¶

This page documents the formal grammar of the mapping language and the semantics of each production.

Grammar¶

Each row below names a leaf in the syntax tree (left) and its expansion (right), written as a mix of leaf names and regular expressions. The most general production — RuleChain — is at the top.

Leaf	Expansion
`RuleChain`	`Rule_Definition+`
`Rule_Definition`	`[Input_Field_Filter =>]? Rule [\\|\\| Conditional_Rule]* [=> Output_Fields_List]?`
`Input_Field_Filter`	`Field_List`
`Conditional_Rule`	`Rule [\[ [Integer]? \]]?`
`Rule`	`Rule_Name\(Rule_Parameters\)`
`Output_Fields_List`	`Field_List`
`Field_list`	`Field [, Field_list]*`
`Rule_Name`	`[A-Z][A-Za-z]+`
`Rule_Parameters`	`Parameter [, Rule_Parameters]*`
`Parameter`	`Keyword_Parameter \\| Table_Field \\| Table \\| Field \\| String \\| Date \\| Integer \\| Float \\| Regex \\| NULL`
`Keyword_Parameter`	`[a-z][a-z0-9._]= ( Table_Field \\| Table \\| Field \\| String \\| Date \\| Integer \\| Float \\| Regex \\| NULL )`
`Table_Field`	`@[A-Za-z][A-Za-z0-9._]+ :[A-Za-z][A-Za-z0-9._]+`
`Table`	`@[A-Za-z][A-Za-z0-9._]+`
`Field`	`:[A-Za-z][A-Za-z0-9._]+`
`String`	`".*"`
`Date`	`'.*'`
`Integer`	`[0-9]+`
`Float`	`[0-9]+\.[0-9]+`
`Regex`	`/.*/`

Semantics¶

`RuleChain`¶

At the start of execution the environment is set up:

In a field-to-field map the environment contains exactly one input field (the source field on the arrow's tail) and one output field (the destination field on the arrow's head). The output field is unnamed and doesn't need a specific name. By the end of execution exactly one field must be present — multiple values cause an error and the table is not mapped.
In a field rule (rules on a CDM field directly) the environment contains every field that maps into that CDM field — possibly zero. By the end of execution exactly one field must remain.

Rules in a RuleChain execute top-down: changes to a field or to the environment affect every subsequent rule.

`Rule_Definition`¶

A rule definition is a set of inputs (explicit or implicit), a rule (or conditional rule chain) to execute, and a list of outputs (explicit or implicit).

`Input_Field_Filter`¶

A filter placed in front of a rule restricts the fields passed to it. Every field referenced in the filter must already exist in the environment — either as a source field or via an Output_Fields_List from a preceding rule.

When an input filter is present but no Output_Fields_List is, behaviour depends on the rule type:

Mapping rules update the fields named in the filter.
Aggregation rules delete those fields and create a new output field.

`Conditional_Rule`¶

Rules can be joined into a conditional chain with ||: if the first rule fails to produce a mapping, the second is tried, and so on.

Mapping rules can only be conditionally chained with other mapping rules.
Aggregating rules can only be conditionally chained with other aggregating rules.
Not all rules support being conditionally chained at all — see the individual rule pages.

`Rule`¶

Rules behave differently depending on family:

Mapping rules execute one-to-one over input fields. When applied to a field rule (not a field-to-field map), the rule runs on every input field unless an input filter is present. If an output list is given, it must have exactly the same length as the inputs.
Aggregating rules always have a single output, but the number of inputs is unbounded. An input filter works exactly as for mapping rules, but the output list may name only a single field.

Rules can also produce multi-sized results — see Multi-Sized Results.

`Output_Fields_List`¶

Behaviour depends on rule family:

Mapping rules write outputs to fields in the order of the inputs. New fields can be declared in the output list — those become virtual fields.
Aggregating rules delete every input field except those mentioned in the output list. If the output list names a field that already exists in the input, that field is preserved with the new value.

Worked examples¶

Static value mapping¶

Map([
  ["M", 8507],
  ["F", 8532]
])

A mapping rule on a field-to-field map. The input is expected to be "M" or "F"; the rule replaces the source value with the matching code.

Conditional chaining for the same effect¶

Map([["M", 8507]]) || Map([["F", 8532]])

If the first Map does not match, the second is tried. The end result is identical to the single map above.

Explicit input/output filters¶

:gender => Map([
  ["M", 8507],
  ["F", 8532]
]) => :result

:result => AssertNot(8532)
Use(:gender)

The first chain stores the map result into a new virtual field :result. AssertNot then runs against :result only — :gender is not asserted on. Finally Use(:gender) declares :gender as the output field for the chain and discards :result. The same effect could be achieved by simply writing AssertNot("F").

Use filters sparingly

Input filters and output lists are expensive at runtime. Reach for them only when there is no simpler way to express the chain.

Aggregation on a CDM field¶

logic: Selector(:int1, "=", :int2, "Equal", "Unequal")

Two source fields (int1, int2) are mapped into the CDM field. If they are equal, the string "Equal" is stored in a virtual field called Selector; otherwise "Unequal". All input fields, used or not, are deleted by the aggregation.

Combining several aggregations¶

logic: :int1, :int4 => Min(:int1, :int4) => :min
logic: :int2, :int3 => Max(:int2, :int3) => :max
logic: Selector(:min, "<", :max, :min, :max)

The first two lines find :min and :max, leaving the chain with five fields (:int1 … :int4, :min, :max). The final Selector chooses the lower of the two and stores it in a virtual field Selector — the only remaining field.

Multi-sized results¶

UsagiMap("usagi.csv")[]

The [] suffix turns the rule into a multi-sized rule: each input record yields one output record per match. A bounded form [3] requires exactly three matches and skips the row otherwise.