Synthmas demo bundle for the ETL Engine
========================================

This tarball is an overlay on top of an ETL Engine install. Together
they give a complete Free-tier proof-of-concept: synthetic patient
data, a working Rabbit-in-a-Hat mapping, the Usagi files it needs,
and a preconfigured etl.conf — running on the engine's built-in
free tier with no licence file required.


How to use
----------

  1. Unpack the engine tarball into a working directory:

        tar xzf etl-engine-X.Y.Z.tar.gz
        cd etl-engine-X.Y.Z

  2. Overlay this demo tarball on top of it. The --strip-components=1
     flag drops the synthmas-single-RiaH top-level directory, so the files
     land where the engine expects them:

        tar xzf ../synthmas-single-RiaH-X.Y.Z.tar.gz --strip-components=1

     (You can also unpack normally and merge by hand, but the
     overlay is the intended flow.)

  3. Install the engine image:

        ./bin/install-etl-engine.sh

  4. Pull the Synthmas source CSVs from MITRE:

        ./bin/setup-synthmas.sh

  5. Find out which OMOP vocabularies the RiaH needs:

        ./bin/check-vocabs.sh

  6. Download those vocabularies from https://athena.ohdsi.org/
     (free Athena account; accept the terms for the constituent
     vocabularies you select), then unpack into the CDM output dir:

        ./bin/setup-vocab.sh ~/Downloads/vocabulary_download_v5_*.zip

  7. Run the engine:

        ./bin/run-etl-engine.sh

  8. Find the populated CDM as CSV files in:

        etc/etl-engine.d/cdm-out/


What this overlay adds
----------------------

  bin/setup-synthmas.sh                 Demo-specific: downloads MITRE's
                                        Synthmas CSVs into db/.

  bin/clean-cdm.sh                      Clears engine-written CDM tables
                                        (lowercase names) in cdm-out/
                                        while preserving Athena vocab
                                        files (uppercase names). Run
                                        before re-running the engine
                                        after any RiaH change.

  etc/etl-engine/synthmas.json.gz       The Rabbit-in-a-Hat file that
                                        maps the Synthmas CSV schema
                                        onto the OMOP CDM.

  etc/etl-engine/etl.conf               Replaces the engine's generic
                                        template with one preconfigured
                                        for the Synthmas demo (CSV→CSV,
                                        Free tier, hat file already
                                        set).

  etc/etl-engine.d/usagi/               Starter Usagi mappings the
                                        Synthmas RiaH references:
                                        units-exported.csv (UCUM
                                        units), specialities.csv
                                        (provider specialties → SNOMED),
                                        and payers.csv (Synthea payers
                                        → SOPT). Extend or replace as
                                        needed for your own concepts.


What this overlay does NOT include
-----------------------------------

  Source CSVs                Pulled by setup-synthmas.sh from MITRE.
  OMOP vocabulary CSVs       Downloaded by you from Athena under your
                             own account.


About Synthmas
--------------

  Synthmas is a synthetic patient dataset generated by Synthea
  (https://synthea.mitre.org/), MITRE Corporation, Apache 2.0. The
  dataset MITRE publishes is for unrestricted research and educational
  use. No real-patient data is involved.

  See docs/site/reference/licenses/ in the engine install for the
  full attribution and licence details.


Compatibility
-------------

  This demo bundle was built against ETL Engine X.Y.Z and is
  expected to work with any engine release in the same minor series.
  If you mix this demo overlay onto a much newer engine, run

      ./bin/check-vocabs.sh

  to verify the RiaH still parses cleanly against the engine version
  you've installed.
