SEA-Forge Capability Analysis

Date: 2026-01-29
Version Analyzed: dev (project_state.md reports v0.6.1 on 2026-01-22)
Analysis Scope: Evidence-based capability summary from codebase

1. Core Capabilities (Implemented / Present in Repo)

Semantic Modeling / Specifications

Evidence: Schemas in tools/schemas/, SDS validator at tools/validate_sds.py, compiler recipes in just/62-compiler.just.

Compilation / Transformation Pipeline

Evidence: Compiler recipes in just/62-compiler.just, tools in tools/.

Validation / Governance / Invariant Checking

Evidence: tools in tools/, CI recipes in just/50-ci.just, drift commands in just/60-spec.just, Semantic Kernel plugin in libs/sea/adapters/semantic-kernel/.

Code / Artifact Generation

Templates: Jinja2 templates under tools/codegen/templates/.

Supported Adapter Patterns (per tools/codegen/gen_adapter_template.py):

• Repository
• Outbox Publisher
• Inbox Consumer
• OPA Client
• PgVector Store
• SPARQL Client
• LLM Adapter
• Flipt Client

CLI Tooling

Evidence: apps/sea-forge-cli/src/commands/.

Applications (Repo)

Services (Repo)

AI/Agent Integration Hooks

Semantic Kernel Functions (from governance_check.py):

• check_policy()
• validate_authority()
• check_invariants()
• detect_policy_drift()
• suggest_policy_update()

2. Supported Workflows

Workflow 1: Spec → Compile → Generate

1
just pipeline <context>

Steps: SDS validate → SEA validate → SEA parse → AST → IR → Manifest
Evidence: just/62-compiler.just.

Workflow 2: Last-Mile Adapter Generation

1
just last-mile <context>

Phases: Gap Analysis → Generate Adapters + Tests → Verify Gaps → Env Setup → Tests → Semantic Check
Evidence: just/64-lastmile.just.

Workflow 3: Drift Detection & Remediation (Spec/Code)

1
2
3
4
just drift-scan
just drift-check <context>
just drift-heal
just drift-report

Evidence: just/60-spec.just, CLI at tools/drift_heal.py, drift API routes in services/workbench-bff/src/api/drift_routes.py.

Workflow 4: Behavior Drift Gate (CI)

1
just ci-drift

Runs scripts/ci/behavior_drift_gate.sh (warn mode locally), with a Python variant available. If the behavior API is unavailable, the gate loads fixtures (scripts/tests/fixtures/behavior_drift_fixtures.json).

Workflow 5: Determinism Check

1
just ci-determinism

Evidence: just/50-ci.just + tools/run_codegen_affected.sh (if present).

Workflow 6: Performance Test Suite

1
just test-performance

Runs pytest tests/performance -v with baseline-based thresholds.
Evidence: just/40-test.just, tests/performance/. Latest Run: 2026-01-29 — just test-performance (2 passed, 1 skipped).

Workflow 7: Chaos Testing Suite

1
2
3
just chaos-up
just run-chaos <scenario>
just chaos-down

Evidence: just/80-devservices.just, tests/chaos/run_chaos.py, infra/docker/docker-compose.chaos.yml. Note: Smoke tests present in tests/chaos/test_runner.py. Latest Run: 2026-01-29 — pytest tests/chaos -v (2 passed).

3. Guardrails, Constraints, and Guarantees

Compile-Time Enforcement

Validation-Time Enforcement

Runtime Enforcement (Plugin)

4. Drift / Misalignment Handling

What Exists

Notes:

• Behavior drift gate uses API data if available, otherwise CI fixtures.
• Remediation engine operates via RemediationEngine and supports PR creation (see services/workbench-bff/src/adapters/remediation_engine.py).

5. What Is Missing or Unclear (Based on Evidence)

Likely Missing / Deferred

Observed Uncertainties

• The repo references domainforge-lsp as an external repo (see just/20-setup.just, just/30-cli.just). It is not present in this repo.
• VS Code extension source exists under extensions/sea-dsl, but publish status is not verifiable from code alone.

6. One-Paragraph “What This System Really Is”

SEA-Forge is a spec-first compiler and generator stack for SEA DSL and SDS that produces manifests and code from schemas, with deterministic regeneration checks and a drift detection/remediation layer. The repo contains Rust CLI tooling, Python compilation/validation utilities, Nx generators, multiple services (policy gateway, knowledge graph, ingest, workbench BFF), and UI applications. Governance enforcement is wired via Semantic Kernel plugins and OPA integrations, while drift detection includes spec/code scanning, remediation PRs, and a CI behavior drift gate (strict mode supported). The evidence suggests a mature compilation pipeline and strong guardrails, with some deployment and publishing items still in progress.

7. Evidence-Backed Capability List

SEA-Forge currently supports:

• ✅ SEA DSL parsing and validation (apps/sea-forge-cli/, just/62-compiler.just)
• ✅ Multi-stage pipeline: SEA → AST → IR → Manifest (tools/*.py, just pipeline)
• ✅ SDS YAML validation (tools/validate_sds.py + tools/schemas/)
• ✅ Flow annotation linting (tools/flow_lint.py)
• ✅ Jinja2-based code generation (tools/codegen/)
• ✅ Adapter scaffolding + tests (gen_adapter_template.py, gen_test_template.py)
• ✅ Gap analysis and last-mile workflows (gap_report.py, just last-mile)
• ✅ Determinism check (just ci-determinism)
• ✅ Drift detection & remediation CLI (tools/drift_heal.py) and API (services/workbench-bff/)
• ✅ Behavior drift gate in CI with strict mode (scripts/ci/behavior_drift_gate.*)
• ✅ SHACL/KG validation tooling (tools/kg_validate.py, tools/shape_lint.py, just/63-kg.just)
• ✅ Semantic Kernel governance plugin (libs/sea/adapters/semantic-kernel/)
• ✅ Service stack present for policy, KG, A2A, embedding, LLM routing, ingest, workbench BFF
• ✅ Nx generators for bounded-context, adapter, and API surface scaffolding (generators/)
• ✅ Production auth hardening for Workbench UI + BFF (OIDC + JWKS) (apps/workbench/src/lib/auth-factory.ts, services/workbench-bff/src/api/auth.py)
• ✅ Performance test suite with baselines (tests/performance/, just test-performance)
• ✅ Chaos testing suite with smoke tests (tests/chaos/, tests/chaos/test_runner.py, infra/docker/docker-compose.chaos.yml, just chaos-up)
• ✅ Incident runbooks (docs/runbooks/incidents/)

8. Notes for Documentation Accuracy

project_state.md vs. Repo Evidence

docs/workdocs/project_state.md (2026-01-22) claims ~90% completion and automatic drift remediation. The repo now contains a drift CLI + API routes and CI gate to support those claims. Where the report mentions “published” artifacts (WASM, extensions), those remain unverified in the codebase.

README vs. Reality

• Drift detection includes spec/code scanning, remediation PR creation, and a strict-mode behavior gate.
• Workbench UI auth includes OIDC + JWKS verification and production-mode enforcement.
• Some claims (marketplace publication, external LSP) depend on external repos or release channels not present in this repo.

9. Skeptical Senior Engineer’s Assessment

✅ Would Trust Today

⚠️ Would Be Cautious About

🚫 Would NOT Rely On Yet

Bottom Line

For production: SEA-Forge is a solid spec-first compiler + generator system with determinism checks and a real drift detection/remediation surface.
Still verify: Publishing flows and external LSP wiring before hard commitments.