Ref-015: Backward Chain Approaches for Spec Drift Detection

Document Type

Reference / Technical Patterns

Purpose

Defines three architectural approaches for implementing Backward Chains (Code → Spec validation) to detect semantic drift between implementation and specifications.

1. The Drift Detection Problem

In forward chaining, specs drive code generation (Spec → Code). In backward chaining, we diagnose whether code still aligns with specs (Code → Spec).

Backward chaining is computationally expensive but critical for:

2. The Three Approaches

APPROACH 1: Annotation Graph (Deterministic)

Mechanism: Hard dependency links via code annotations.

Implementation

1
2
3
4
5
6
7

/**
 * @implements PRD-05
 * @complies ADR-09
 */
export function login(credentials: Credentials): Promise<AuthResult> {
  // ...
}

Backward Chain Trigger

Advantages

Disadvantages

APPROACH 2: Semantic Watchdog (Probabilistic)

Mechanism: Vector embeddings create soft links between code and specs.

Implementation

  1. Embed all ADRs/SDSs/PRDs into Vector Database
  2. On PR, embed the code diff
  3. k-NN search: “Which spec paragraphs match this code?”
  4. Alert if similarity score drops below threshold
1
2
3
4
5
6
7
8
9
10

def semantic_watchdog(diff: CodeDiff, spec_db: VectorDB) -> DriftReport:
    diff_embedding = embed(diff.content)
    matches = spec_db.search(diff_embedding, k=3)
    
    if matches[0].score > 0.85:
        return DriftReport(status="ALIGNED", spec=matches[0].id)
    elif matches[0].score < 0.50:
        return DriftReport(status="DRIFT_ALERT", message="No matching spec")
    else:
        return DriftReport(status="REVIEW_NEEDED", candidates=matches)

Advantages

Disadvantages

APPROACH 3: Fitness Function Oracle (Structural)

Mechanism: ADRs translated to executable architecture tests.

Implementation

ADR Text: “Business Logic must not depend on UI Layer”

Fitness Function (using TsArch):

1
2
3
4
5
6
7
8
9
10
11

describe('Architecture Fitness', () => {
  it('domain should not depend on ui', () => {
    const result = files()
      .inFolder('domain')
      .shouldNot()
      .dependOn()
      .folder('ui');
    
    expect(result).toPass();
  });
});

Advantages

Disadvantages

APPROACH 4: Blame Ledger (Process-Based)

Mechanism: Git commit messages as the chain of custody.

Implementation

Commit Schema:

1

feat(auth): implement login [Ref: PRD-05, SDS-02]

Backward Chain:

1

git blame src/auth/login.ts | grep -o 'PRD-[0-9]*' | sort -u

CI Validation:

1
2
3
4
5
6
7
8

def validate_pr(commits: List[Commit], spec_db: SpecDB) -> ValidationResult:
    for commit in commits:
        spec_ids = extract_spec_refs(commit.message)
        for spec_id in spec_ids:
            spec = spec_db.get(spec_id)
            if spec.status in ["DRAFT", "DEPRECATED"]:
                return ValidationResult.BLOCKED(f"{spec_id} is {spec.status}")
    return ValidationResult.PASSED()

Advantages

Disadvantages

3. Strategic Recommendation

Hybrid Approach

Spec Type Approach Rationale
ADRs Fitness Functions (Approach 3) Architecture must be deterministic
PRDs Semantic Watchdog (Approach 2) Features are fluid, intent matters
Audit Trail Blame Ledger (Approach 4) Fallback for traceability

Decision Matrix

Requirement Annotation Watchdog Fitness Blame
No manual tagging ❌ ✅ ⚠️ ❌
Detects intent drift ❌ ✅ ❌ ❌
Deterministic ✅ ❌ ✅ ✅
Low latency ✅ ❌ ✅ ✅
Handles structure ✅ ⚠️ ✅ ❌
Handles semantics ❌ ✅ ❌ ❌

4. Implementation Examples

Combined CI Workflow

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

name: Spec Compliance Check

on: pull_request

jobs:
  # Approach 3: Architecture Fitness
  architecture:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: pnpm test:arch

  # Approach 2: Semantic Watchdog
  semantic:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Extract Diff
        run: git diff origin/main...HEAD > diff.txt
      - name: Run Watchdog
        uses: sea-forge/semantic-watchdog@v1
        with:
          diff_file: diff.txt
          threshold: 0.5

  # Approach 4: Blame Ledger
  traceability:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Validate Commit Refs
        run: |
          for sha in $(git log origin/main..HEAD --format=%H); do
            refs=$(git log -1 $sha --format=%B | grep -oP '(PRD|ADR|SDS)-\d+')
            for ref in $refs; do
              status=$(sea-forge spec-status $ref)
              if [[ "$status" != "APPROVED" ]]; then
                echo "::error::$ref is $status"
                exit 1
              fi
            done
          done