CDISC external dictionaries
===========================

This directory is a READ-ONLY FALLBACK. It is not where the application writes.

Where the store lives
---------------------
The dictionary store (MedDRA, WHODrug, SNOMED CT, LOINC, UNII, MED-RT, Neoplasm)
lives in your own user directory:

    ~/.cumbaDataBrowser/dictionaries          (Linux / macOS)
    %USERPROFILE%\.cumbaDataBrowser\dictionaries   (Windows)

It is there rather than beside the application because the application WRITES to
it, and an install directory is not reliably writable: Program Files needs
elevation, a signed macOS .app bundle would have its code signature broken, and a
shared install serves several users from one directory.

The application points the CDISC CORE engine at that store when it starts,
creating the directory first. It does NOT do so if you have already said where the
store is; anything set from outside wins, in this order:

    COREJ_DICTIONARIES_DIR environment variable
    -Dcorej.dictionariesDir=<dir> on the command line
    a corej.dictionariesDir entry in databrowser.conf (in the installation directory),
      written as  corej.dictionariesDir = ${sys:corej.dictionariesDir:-<your dir>}
      so that a -D on the command line still wins

This directory is still read as a fallback, but only if it already contains an
installed store AND your user directory does not. Nothing is ever written here.

Installing dictionaries
-----------------------
Use Tools > Manage CDISC Dictionaries in the application. Three of the seven can
be downloaded with no account (MED-RT, UNII and the SEND CT neoplasm subset); the
other four you supply yourself from a folder:

    LOINC            needs a free loinc.org account
    MedDRA           your own licensed distribution
    WHODrug          your own licensed distribution
    SNOMED CT        your own licensed distribution

Each supplier's own licence or terms-of-use notice is shown before anything is
acquired and must be acknowledged. The notice is then written beside the installed
data, in <store>/<type>/<version>/LICENSES, and where the terminology came from is
recorded in <store>/SOURCES.md.

What the folder must contain
----------------------------
When you install a dictionary from a folder, the folder must hold the supplier's
distribution in the layout described below. This is the input the installer
expects; anything else is refused or, where noted, misread.

Rules for every dictionary:

  - Choose the folder that holds the files themselves. Unpack a zip first: the
    installer reads folders, not archives. (SNOMED CT is searched in subfolders,
    LOINC also accepts a LoincTable/ subfolder; every other dictionary needs the
    files directly in the chosen folder.)
  - One release per folder. Do not unpack two releases into the same folder.
  - Text files are read as UTF-8; a leading byte-order mark is fine. A byte that
    is not valid UTF-8 is replaced and reported as a warning after the install:
    the term that contained it will not match your data.
  - Terms are installed exactly as the supplier spells them; case is kept.
  - The release version is read from the files (see each entry). A folder whose
    version cannot be read is not installed.
  - Files not listed below are ignored.
  - Column and field numbers below count from 1.

MedDRA (the MedAscii distribution)
  Required files: llt.asc, pt.asc, hlt.asc, hlgt.asc, soc.asc, mdhier.asc,
  meddra_release.asc.
  Format: one record per line, fields separated by "$". A leading and a
  trailing "$" are allowed.
    llt.asc, pt.asc, hlt.asc, hlgt.asc, soc.asc
        field 1 = code, field 2 = name. Lines with fewer than two fields or an
        empty code or name are skipped.
    mdhier.asc
        field 1 = PT code, field 2 = HLT code, field 3 = HLGT code,
        field 4 = SOC code, field 12 = primary SOC flag. Only rows flagged "Y"
        (the primary path) are read.
    meddra_release.asc
        field 1 of the first non-empty line = the version, e.g. 27.0.
  Two spellings of one term that differ only in case within one file abort the
  install.

WHODrug (B3 format only)
  Only the WHODrug B3 format is supported. The C3 format is NOT supported yet:
  do not install a C3 distribution. Its columns sit at different positions and
  would be read at the wrong places.
  Required files: DD.txt, INA.txt, version.txt. (DDA.txt and the other B3
  files are not read.)
  Format: fixed-width text, one record per line.
    DD.txt (drug names)
        columns 1-6   drug record number
        columns 7-8   sequence number 1
        columns 9-11  sequence number 2
        column 31-    drug name, to the end of the line
        Lines of 30 characters or fewer are skipped. The preferred name of a
        drug record is its row with sequence 1 = 01 and sequence 2 = 001.
    INA.txt (the full ATC index)
        columns 1-7   ATC code
        column 8      ATC level
        column 9-     ATC text, to the end of the line
    version.txt
        The first non-empty line; its last five characters name the release
        as a three-letter month and a two-digit year, so a line ending in
        "Sep20" installs as version SEP_2020.

SNOMED CT (an RF2 release: International Edition, a national edition, or
the Global Patient Set)
  Required file: one sct2_Description*.txt file, found anywhere below the
  chosen folder. An RF2 release has Snapshot/, Full/ and Delta/ views: when a
  Snapshot file is present only the Snapshot is read, otherwise Full. If more
  than one description file remains (for example a language extension, or two
  editions unpacked together) the install is refused.
  Format: tab-separated, with a header row. The columns are found by their
  header names: active, conceptId, typeId, term. Only rows with active = 1 are
  read.
    Concept codes: every active concept that has a description.
    Terms: per concept, its first active synonym (typeId 900000000000013009).
    The Fully Specified Name, e.g. "Headache (finding)", is never used as a
    term.
  Version: the last 8-digit date in the description file's name, e.g.
  sct2_Description_Snapshot-en_INT_20240901.txt installs as 20240901.

LOINC
  Required file: Loinc.csv, in the folder itself or in LoincTable/.
  Format: a standard CSV file with a header row. Quoted fields may contain
  commas, doubled quotes and line breaks. The columns are found by their
  header names: LOINC_NUM, LONG_COMMON_NAME, VersionLastChanged.
  Version: the highest VersionLastChanged value in the file, e.g. 2.77.

UNII (downloaded automatically; folder install optional)
  Required file: UNII_Records_<date>.txt. If several files match, the first
  in name order is read, so keep one per folder.
  Format: tab-separated, with a header row. The columns UNII and DISPLAY_NAME
  are found by header name (else columns 1 and 2 are used).
  Version: the <date> part of the file name, e.g. UNII_Records_4Aug2026.txt
  installs as 4Aug2026.

MED-RT (downloaded automatically; folder install optional)
  Required file: MEDRT.txt.
  Format: tab-separated, no header: column 1 = concept name, column 2 = NUI.
  Version: the version element of Core_MEDRT_*_DTS.xml in the same folder;
  without it, the version named in MEDRT_Release_Notes*.txt.

Neoplasm (downloaded automatically; folder install optional)
  Required files: SEND Terminology.txt (or SEND_Terminology.txt) and
  SEND Publication Date Stamp.txt, exactly one of each.
  Format: the CDISC SEND controlled terminology, tab-separated with a header
  row: column 1 = concept code, column 2 = codelist code, column 5 =
  submission value. Only rows of the Neoplasm Type codelist (C88025) whose
  value ends in ", BENIGN" or ", MALIGNANT" are read.
  Version: the date in SEND Publication Date Stamp.txt, e.g. 2026-03-27.

Why nothing ships here
----------------------
Every one of those dictionaries is licensed content, per organisation. They cannot
be redistributed inside an application, and four of them cannot be fetched on your
behalf at all. Obtaining those under your own organisation's licence is a step only
you can take.

What happens with an empty store
--------------------------------
A dictionary is loaded only when something names its version -- the store's
selected-versions.json, a per-run version chosen in the CORE check dialog, or an
ExternalCodeList/@Version in your define.xml. With nothing installed, nothing names
a version, so the dictionary is not loaded and the rules that depend on it report
SKIPPED rather than failing. Validation of everything else is unaffected.

So an empty store is a working configuration, not a broken one. You will simply see
dictionary-dependent rules skipped in the report.

Removing a dictionary
---------------------
Removal is a manual filesystem step: delete <store>/<type>/<version> and, if it
names that version, its entry in <store>/selected-versions.json.

The licence texts that apply to each dictionary are distributed by the respective
organisations, not by us.
