Drift Detection Guide

Understanding and acting on semantic drift in SEA-Forge.

What is Drift?

Semantic drift occurs when observed runtime behavior diverges from declared specifications. This can indicate:

• Bypass: Required steps being skipped
• Shadow IT: Unauthorized processes emerging
• Process Evolution: Natural changes over time
• Compliance Issues: Regulatory violations

Detection Methods

1. Activity-Based Drift

Compare declared activities vs observed activities:

1
just observe-diff my-context

Missing Activity: A declared activity never appears in logs Extra Activity: An undeclared activity appears in logs

2. Flow-Based Drift

Analyze sequence deviations:

• Bypass: Expected sequence skipped
• Reordering: Activities out of order
• Loop: Unexpected cycles

3. Role-Based Drift

Actor participation analysis:

• Role Mismatch: Wrong actor performing activity
• Handoff Issues: Unexpected work transfers

Interpreting Drift Reports

Severity Levels

• Critical: Immediate action required (e.g., compliance violations)
• High: Important deviations (e.g., bypassed controls)
• Medium: Notable differences (e.g., extra steps)
• Low: Minor variations (e.g., infrequent loops)

Confidence Scores

Drift reports include confidence scores (0-1):

• > 0.8: Strong evidence, highly reliable
• 0.5-0.8: Moderate evidence, worth investigating
• < 0.5: Weak evidence, may be noise

Responding to Drift

1. Investigate

Use the evidence pointers to locate source:

1
just observe-summary my-context

Review raw logs in the UI or exported artifacts.

2. Decide

Choose an action:

• Update Spec: If drift is legitimate evolution
• Fix Process: If drift indicates problem
• Add Exception: If drift is edge case

3. Act

Promote observed to declared with signed commit:

1
2
3
# In the UI, click "Promote to Declared"
# Or use CLI:
just observe-promote my-context --sign

Best Practices

1. Baseline First: Establish drift baseline before changes
2. Monitor Regularly: Set up automated drift scanning
3. Context Matters: Consider business context when interpreting
4. Gradual Adoption: Start with high-value processes

Example Workflow

1
2
3
4
5
6
7
8
9
10
11
12
13
# 1. Establish baseline
just observe-all payment-process logs/payment-baseline.csv
just observe-summary payment-process

# 2. Monitor after deployment
just observe-all payment-process logs/payment-after.csv
just observe-diff payment-process

# 3. Investigate new drifts
cat artifacts/observe/payment-process/drift_diffs.json | jq .

# 4. Act on findings
# (Use UI or promote observed changes)