Plan: SHACL Runtime Validation Enforcement

Spec-first artifacts

• ADR-040: docs/specs/semantic-core/semantic-core.adr.md
• PRD-008: docs/specs/semantic-core/prd/005-shacl-runtime-enforcement.md
• SDS updates: docs/specs/semantic-core/sds/003-knowledge-graph-service.md
• Identity specs: docs/specs/shared/sds/050-semantic-identity-provenance.md, docs/specs/cognitive-extension/reference/002-identity-token-specification.md
• Tooling: tools/ir_to_kgs.py, tools/kg_validate.py, tools/shape_impact_analysis.py, tools/shape_lint.py, tools/validate_local_triples.py
• Handbooks: docs/handbooks/shacl-enforcement-migration.md, docs/handbooks/shacl-shape-versioning.md, docs/handbooks/shape-governance.md

Steps

1. Define config inputs (single source of truth)
   • Create a config module in the knowledge-graph service (or equivalent infra layer) that reads:
     • SHACL_ENFORCEMENT_ENABLED (default: true)
     • SHACL_ENFORCEMENT_CONTEXTS (CSV, empty = all)
     • SHACL_MAINTENANCE_MODE (default: false)
     • SHACL_SHAPES_VERSION (default: v1)
   • Pass config into OxigraphAdapter via constructor; avoid global env reads in runtime code paths.
2. Normalize validation API contract
   • Ensure validate_with_shacl(...) returns a consistent type at all call sites (e.g., a ValidationResult dataclass/tuple), and only raises ShaclValidationError when enforcement is enabled + context is enabled + not in maintenance mode.
   • Update every caller accordingly (audit required to prevent mixed return/raise behavior).
3. Fail-closed policy (context-aware)
   • Replace any fail-open return True with explicit, structured error handling:
     • If maintenance_mode: log at error level, return non-conforming result (no raise).
     • If enforcement_enabled and context is active: raise ShaclValidationError with full report.
     • Otherwise: warn and return non-conforming result.
4. Shapes versioning without symlink dependency
   • Rename current shapes file to sea-shapes.v1.shacl.
   • Update loader to resolve sea-shapes.{version}.shacl first, then fall back to sea-shapes.shacl if present.
   • Avoid symlink requirements to preserve Windows/CI portability.
5. Cache shapes parsing safely
   • Make _load_shapes_graph(version: str) a pure function (module-level or @staticmethod).
   • Apply @lru_cache(maxsize=3) on that pure function so cache keys are just the version.
6. Fix validation caching key correctness
   • Replace hashlib.sha256(sorted(graph.serialize())) with deterministic hashing:
     • Serialize to N-Triples, split into lines, sort lines, join with \n, hash the resulting bytes.
     • This avoids per-character sorting and serialization-order nondeterminism.
7. Delta validation with full-graph escape hatch
   • Default to validating only the delta graph for performance but introduce a shape annotation (e.g., sea:validationScope "full") that forces full-graph validation for constraints requiring global context (uniqueness, closed shapes, cross-entity cardinality).
   • If any active shape requires full, merge delta into the current graph for validation.
8. Cross-context reference checks with safe caching
   • Cache existence checks per-session with TTL, but invalidate or bypass cache on write paths that may delete referenced entities.
   • Use SPARQL ASK or EXISTS with batched values for efficiency.
9. Shared shapes layout + governance
   • Create ontologies/shared/ and move universal constraints to ontologies/shared/core.shacl.
   • Use owl:imports and ensure the loader resolves local file URIs correctly.
   • Add governance doc in docs/handbooks/shape-governance.md.
10. Add cross-context dependency annotations
    • In shapes, annotate external references with sea:requiresContext.
    • Parse these annotations in validation to produce explicit missing-context errors.
11. TypeScript validated adapter (handwritten)
    • Implement a ValidatedTripleStoreAdapter only if it is strictly an integration adapter.
    • Confirm module boundaries (do not place in a generated or domain-logic module).
    • Ensure it implements the port interface from SDS and includes tests in adjacent tests/.
12. Improve SHACL errors (actionable diagnostics)
    • Add get_fix_suggestions() to build human-readable fixes from violations.
    • Include violated shape URI, focus node, property path, expected values, and any sea:requiresContext failures.
13. Enable validation for update events
    • Remove any validate = False bypasses for update events.
    • Ensure update pipelines build a temp graph of the updated entity state before validation.
14. IR-to-KGS batch validation
    • Add _validate_triples_batch(...) that validates a single merged graph to avoid per-triple overhead.
    • Ensure this is in an allowed tooling/infra location, not in generated runtime zones.
15. Add diagnostic commands
    • just kg-validate: validates the persisted graph, emits JSON report, non-zero exit on violations.
    • just kg-validate-local: validates changed .sea files via IR pipeline before commit.
16. Shape impact analysis + CI gate
    • Add just kg-impact-analysis to validate entire graph against a candidate shapes version.
    • Add CI workflow for PRs modifying ontologies/** to run impact analysis.
17. Shape lint tool
    • Add just shape-lint to detect cross-context conflicts and import graph issues.
    • Treat errors as merge blockers; warnings as advisory.
18. Wire validated adapter
    • Update DI configuration to wrap the triple store with validation adapter using config flags from Step 1.
19. OpenTelemetry telemetry
    • Add spans and metrics around validation with attributes for: conforms, violation_count, duration_ms, cache_hit, context, version, maintenance_mode.
20. Prometheus alerting
    • Add alerts for validation failures, latency, cache hit rate, maintenance mode, and version changes.
21. Legacy data exemption with expiration
    • Implement sea:legacyData + sea:migrationDeadline checks with a post-deadline grace period.
    • Publish overdue events from a daily job and report progress in Workbench.
22. Shape change notification
    • On startup, compare deployed shapes version with prior metadata and emit change events.
    • Use rdflib.compare.graph_diff() to classify breaking vs non-breaking changes.
23. Migration runbook
    • Add docs/handbooks/shacl-enforcement-migration.md with phased rollout and rollback steps.
24. Shape versioning guide
    • Add docs/handbooks/shacl-shape-versioning.md with compatibility and rollout policy.
25. Shape governance
    • Add docs/handbooks/shape-governance.md with ownership and approval flow.
26. Performance tests (non-blocking suite)
    • Add a separate perf test suite (not default CI gating) for validation latency/throughput.
    • Keep strict thresholds out of shared CI runners to avoid flakiness.
27. Expanded functional tests
    • Update existing SHACL validation tests to cover enforcement flags, context filtering, maintenance mode, caching correctness, cross-context validation, and legacy exemptions.

Notes

• This plan corrects the unsafe delta-only validation, cache key errors, and mixed return/raise behavior.
• It avoids symlink dependence for shapes on Windows/CI.
• It explicitly scopes handwritten code and tests under --hand-write-ok requirements.