Implementation Plan: Compiler Chain (Unified Delivery Pipeline)

Implement the seven-stage compiler-style delivery pipeline defined in ADR-012:

1
Specs → Plan → SEA™ DSL → AST → IR → Manifest → Generated Code → Drift Check

This unified plan consolidates four previously separate plans (P001, P003, P013, P018) that all target the same pipeline stages. The consolidation eliminates overlap (e.g., both P003 and P018 touched ir_to_manifest.py and manifest.schema.json) and establishes a single source of truth for the compiler chain implementation.

Provenance & Traceability

Architectural Decisions (ADRs)

Product Requirements (PRDs)

Software Design Specifications (SDS)

Provenance Chain

graph TD
  ADR12[ADR-012: Delivery Pipeline] --> PRD4[PRD-004: Delivery Pipeline]
  ADR3[ADR-003: Isomorphism] --> PRD4
  ADR4[ADR-004: Semantic Core] --> PRD4
  PRD4 --> SDS21[SDS-021: Delivery Pipeline Service]

Consolidation Rationale

Architecture and Design

Design Principles Applied

• Isomorphism: Each compilation stage is a deterministic projection of the prior stage; the chain is reconstructible.
• Invariants: Flows declare CQRS kind + required metadata; generated code is output-only; drift is a CI failure.
• Idempotency: Re-running the full pipeline from identical specs yields byte-identical artifacts.

Pipeline Stages (ADR-012 §Pipeline Stages)

Dependency Justification

• No new dependencies for plan completion—uses existing jsonschema, schemas, and generator tooling.

Expected Filetree

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
/
├── docs/specs/**                              # Source-of-truth specs
├── tools/
│   ├── schemas/
│   │   ├── ast-v3.schema.json                 # AST contract
│   │   ├── sea_ir.schema.json                 # IR contract
│   │   └── manifest.schema.json               # Manifest contract
│   ├── flow_lint.py                           # AST annotation lint
│   ├── ast_to_ir.py                           # Stage 4: AST → IR
│   ├── ir_to_manifest.py                      # Stage 5: IR → Manifest
│   ├── codegen/
│   │   └── gen.py                             # Stage 6: Manifest → code
│   └── regen_check.py                         # Stage 7: Drift detection
├── generators/**                              # Nx generators (templates + schemas)
└── **/src/gen/**                              # Generated output only

Proposed Cycles

Task Breakdown

Wave 1: Semantic Foundation (Parallel)

Goal: Lock the semantic contract from SEA™ DSL through AST to IR.

• C1A: Flow Annotation Contract
  • Implements: Runtime semantics contract for Flow nodes (CQRS kind, @tx, @outbox, @idempotency, @read_model)
  • Satisfies: SDS-021 §3.2 (Semantic Compiler validation)
  • Files: tools/flow_lint.py, tools/schemas/ast-v3.schema.json
  • Acceptance:
    • flow_lint.py --strict fails on missing @cqrs annotation
    • Nested JSON annotation format enforced (no dotted keys)
    • GitHub Actions output format (--gha)
• C1B: AST → IR Compilation
  • Implements: Deterministic AST-to-IR transformation with symbol ID generation, span mapping, CQRS/runtime tag extraction
  • Satisfies: SDS-021 §3.2–3.3 (Semantic → Manifest flow)
  • Files: tools/ast_to_ir.py, tools/schemas/sea_ir.schema.json
  • Acceptance:
    • ast_to_ir.py <ast.json> <ir.json> produces schema-valid IR
    • CQRS classification (command, query, event) extracted and tagged
    • Stable output on repeated runs

Wave 2: Structural Projection (Parallel, depends on Wave 1)

Goal: Project IR into manifests with canonical schema.

• C2A: IR → Manifest Projection
  • Implements: Complete IR-to-Manifest mapping for commands/queries/events, policies, and output roots
  • Satisfies: SDS-021 §3.3 (Manifest Compiler)
  • Files: tools/ir_to_manifest.py, tools/schemas/manifest.schema.json
  • Acceptance:
    • ir_to_manifest.py <ir.json> <manifest.json> produces schema-valid manifest
    • All outputRoots restricted to **/src/gen/**
    • Stable diffs on re-projection
• C2B: Manifest Spec Alignment
  • Implements: Synchronize REF-011 documentation with actual manifest.schema.json
  • Satisfies: Documentation accuracy
  • Files: docs/specs/shared/reference/011-manifest-schema.md
  • Acceptance:
    • Examples in REF-011 validate against schema
    • Schema changes reflected in spec

Wave 3: Generation & Guardrails (Parallel, depends on Wave 2)

Goal: Deterministic code generation with strict output containment.

• C3A: Codegen Guardrails
  • Implements: Enforce generator output paths to declared outputRoots (only **/src/gen/**)
  • Satisfies: SDS-021 §3.5 (Generator Runner) + “no handwritten runtime code” invariant
  • Files: tools/codegen/gen.py
  • Acceptance:
    • Generator refuses to write outside src/gen/ paths
    • Clear error on path violation
• C3B: Determinism Enforcement
  • Implements: Regeneration check that compares output to committed state
  • Satisfies: SDS-021 §3.6 (Drift Detector)
  • Files: tools/regen_check.py
  • Acceptance:
    • regen_check.py returns exit 0 when regen == committed
    • Non-zero exit and clear diff output on mismatch

Wave 4: CI Integration & Contracts (Depends on Wave 3)

Goal: Full pipeline enforcement in CI; generated contracts and tests.

• C4A: CI Gates
  • Implements: Mandatory pipeline stages in CI/CD (semantic validate → manifest validate → generate → test → drift check)
  • Satisfies: SDS-021 §7 (CI/CD Integration)
  • Files: .github/workflows/**, tools/nx/workspace/project.json
  • Acceptance:
    • CI fails on any stage failure
    • Drift detection runs post-generation
• C4B: Contract & Test Scaffolding
  • Implements: Generate schema-driven contracts (events, DTOs) and baseline test stubs
  • Satisfies: Spec-to-code “proof” via contract validation
  • Files: schemas/contracts/**, generated **/src/gen/**
  • Acceptance:
    • Contracts generated from manifest
    • Test stubs compilable and runnable

Validation & Verification

Spec Validation

• Confirm PRD-004 → SDS-021 mapping in docs/specs/shared/reference/009-traceability-matrix.md
• Verify SDS-021 sections §3.1–3.6 are fully addressed by cycles

Implementation Validation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# Wave 1: Semantic foundation
python tools/flow_lint.py <ast.json> --strict
python tools/ast_to_ir.py <ast.json> /tmp/ir.json

# Wave 2: Structural projection
python tools/ir_to_manifest.py /tmp/ir.json /tmp/manifest.json

# Wave 3: Code generation
python tools/codegen/gen.py /tmp/manifest.json

# Wave 3: Drift check
python tools/regen_check.py

# Full pipeline (via just/Nx)
just pipeline <ctx>
nx run workspace:check

CI Integration Validation

• nx run workspace:check includes all gates (lint, compile, gen, test, drift)
• GitHub Actions workflow mirrors local pipeline
• Drift failure blocks merge

Open Questions

Risks & Mitigation

Rollback Strategy

1. Lint too strict? Temporarily run --warn-only while updating specs; re-enable strict mode.
2. Projection too strict? Allow missing optional fields while keeping outputRoots + CQRS required.
3. CI gate failures? Temporary override record (per ADR-012 §Special-Case Manual Code) with expiry and backfill ref.

Deprecated Plans

The following plans are superseded by this consolidated plan and should be marked status: Superseded:

Related Documents

Reference Documents

• ADR-012: Delivery Pipeline — Authoritative decision
• SDS-021: Delivery Pipeline Service — Target spec
• REF-011: Manifest Schema
• SEA™ IR Specification
• Traceability Matrix

Linked Specifications