The Observed Semantics module extracts process models from event logs and compares them against declared SEA specifications to detect semantic drift.

Features

Architecture

1
2
3
4
5
6
7
8
9
10
Event Log -> Ingest -> Canonical Events -> Mine -> Observed Graph
                                                |
                                                v
                                        Declared Spec
                                                |
                                                v
                                            Diff -> Drift Report
                                                |
                                                v
                                            Project -> SEA IR

CLI Usage

Full Pipeline

1
2
# Run everything: ingest, mine, diff, project
just observe-all request-process logs/requests.csv

Step-by-Step

1
2
3
4
5
6
7
8
9
10
11
# 1. Ingest event log
just observe-import request-process logs/requests.csv

# 2. Build process graph
just observe-mine request-process

# 3. Compute drift
just observe-diff request-process

# 4. Project to IR
just observe-project request-process

Extract Rules from Text

1
2
# Extract from policy document
just observe-extract-context request-process docs/policy.txt

API Usage

Ingest Event Log

1
2
3
4
curl -X POST http://localhost:8010/api/observe/ingest \
  -F "file=@events.csv" \
  -F "context_id=my-process" \
  -F "format=csv"

Get Observed Graph

1
curl http://localhost:8010/api/observe/contexts/my-process/graph

Compute Drift

1
2
3
curl -X POST http://localhost:8010/api/observe/diff \
  -H "Content-Type: application/json" \
  -d '{"context_id": "my-process"}'

Drift Types

Artifacts

Directory Structure

Type	Description	Severity
`missing_activity`	Declared activity not observed	High
`extra_activity`	Observed activity not declared	Medium
`bypass`	Declared step being skipped	High
`reordering`	Sequence deviation	Medium
`loop_emergence`	Unexpected cycle	Low
`role_mismatch`	Actor performing unexpected activity	Medium

1
2
3
4
5
6
7
8
9
artifacts/observe/<context>/
├── canonical_events.json      # Normalized events
├── ingest_manifest.json       # Ingestion metadata
├── observed_graph.json        # Process graph
├── graph_summary.json         # Graph statistics
├── candidate_rules.json       # Extracted rules
├── drift_diffs.json           # Drift differences
├── drift_summary.json         # Drift statistics
└── observed_semantics.ir.json # SEA IR bundle

IR Format

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
{
  "irVersion": "1.0.0",
  "meta": {
    "generatedAt": "2024-01-15T10:00:00Z",
    "generator": "semantic-miner",
    "gitSha": "abc123"
  },
  "observedSemantics": [
    {
      "kind": "ObservedProcess",
      "nodeMetadata": {
        "activities": ["submit", "review", "approve"],
        "statistics": {...}
      }
    }
  ]
}

Testing

1
2
3
4
5
6
7
8
# Run all semantic miner tests
pytest tests/semantic_miner/

# Run specific test file
pytest tests/semantic_miner/test_graph_builder.py -v

# Run with coverage
pytest tests/semantic_miner/ --cov=libs/semantic_miner

Observed Semantics

Overview