02 — System Diagram

🔍 Open the Interactive Diagram Viewer — pan, mouse-wheel zoom, fit-to-screen, full-screen, per-area tabs, the full ERD, and a focus mode that draws one model and its neighbours at a chosen depth. DataAgreement, DataRequest, DatasetDefinition and SubmissionBatch are hub tables, so their links can be hidden to declutter the overview.

Regenerate the viewer after any DSL, menu_config.yaml or diagram_config.yaml change: python scripts/build_diagram_viewer.py adidam.

The intake spine

flowchart TD
  Entity[Entity]
  Eng[AuditEngagement]
  DA[DataAgreement]
  Scope[AgreementScope]
  Req[DataRequest]
  DS[DatasetDefinition]
  Fld[DatasetField]
  Spec[ExtractSpecification]
  VRule[ValidationRule]
  Sub[SubmissionBatch]
  VRun[ValidationRun]
  VRes[ValidationResult]
  CT[ControlTotalDefinition]
  Rec[Reconciliation]
  Exc[IntakeException]
  Dec[SubmissionDecision]
  AG[AccessGrant]
  Ret[RetentionDisposition]
  SH[StatusHistory]

  Entity --> DA
  Eng --> DA
  DA --> Scope
  DA --> Req
  Entity --> Req
  Req --> DS
  Scope --> DS
  DS --> Fld
  DS --> Spec
  DS --> VRule
  Req --> Sub
  DS --> Sub
  Sub --> VRun
  VRun --> VRes
  VRule --> VRes
  DS --> CT
  Sub --> Rec
  CT --> Rec
  Req --> Exc
  Sub --> Exc
  Sub --> Dec
  DS --> AG
  Sub --> Ret
  Req --> SH

How to read it

  • Four hubs. An agreement authorises requests; a request defines a dataset; a dataset receives submissions. Most detail hangs off those four — the viewer lets you hide their edges to see local structure.
  • Authorisation first. A DataRequest must sit under a current DataAgreement and AgreementScope — the scope says which fields and periods are permitted.
  • Quality gates acceptance. A SubmissionBatch is checked by a ValidationRun (per-rule ValidationResults) and reconciled against ControlTotalDefinitions; a blocking failure raises an IntakeException, and only a clean run leads to an accept SubmissionDecision.
  • Stewardship after. An accepted submission is governed by AccessGrants and a RetentionDisposition through to secure disposal.
  • StatusHistory is the audit trail of the request's state transitions.

Backing data (denormalised for display)

Each foreign key carries a denormalised <fk>_name display column so lists read in plain language (e.g. a SubmissionBatch shows its request; a Reconciliation shows its control total) without a join at read time — the house style for these apps.