🎯 Softmax Router Pattern

Transformer-inspired multi-agent orchestration.

Overview

The Softmax Router pattern treats attention mechanism mathematics as a coordination protocol for multi-agent workflows. Agents contribute bounded outputs that are weighted and aggregated like attention heads in a transformer.

Core Invariants

When translating transformer architecture to agent workflows, these invariants must be preserved:

Invariant Transformer Form Agent Workflow Form
Shared State Residual stream ContextSnapshot
Bounded Contribution Bounded vector update Schema + size limits
Normalized Arbitration Softmax ensures bounded influence Router normalizes weights
Identity Stability Token identities stable ConceptId persistence
Admissible Transformations Can’t invent operations Defined tools per agent
Round-Based Iteration Layers = refinement steps Explicit round count

Isomorphic Mapping

Transformer Component Agent Workflow Equivalent
Token stream / residual ContextSnapshot + working memory
Attention head Specialist agent (bounded output)
Q/K/V matrices Query / capability / payload schemas
Softmax routing Router agent (relevance + weights)
Layer One governance-checked round
FFN (MLP) Synthesis agent
LayerNorm Contract validation + normalization
Multi-head Parallel specialist proposals

Router Agent

Responsibility: Determine which specialists are relevant and allocate attention budget.

Output Format

1
2
3
4
5
6
7
8
9

{
  "distribution": [
    { "agentId": "semantic-mapper", "weight": 0.35, "tokenBudget": 500 },
    { "agentId": "rule-analyst", "weight": 0.25, "tokenBudget": 300 },
    { "agentId": "arch-checker", "weight": 0.20, "tokenBudget": 200 },
    { "agentId": "retrieval-agent", "weight": 0.20, "tokenBudget": 200 }
  ],
  "toolPermissions": ["read-kgs", "query-sbvr"]
}

Design Rules

Round Execution Flow

1
2
3
4
5
6
7
8
9
10
11
12

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ Round N                                                      β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚  1. Router receives ContextSnapshot + Query                  β”‚
β”‚  2. Router outputs weight distribution + permissions         β”‚
β”‚  3. Specialists execute in parallel (bounded outputs)        β”‚
β”‚  4. Aggregator combines weighted deltas                      β”‚
β”‚  5. Validator checks invariants                              β”‚
β”‚     - Pass: ContextSnapshot(N+1) committed                   β”‚
β”‚     - Block: Violation event, no commit                      β”‚
β”‚  6. If more rounds needed: goto Round N+1                    β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Configuration

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

router:
  agentId: "router-v1"
  model: "claude-sonnet"
  maxOutputTokens: 200

specialists:
  - agentId: "semantic-mapper"
    skill: "term-to-concept-binding"
    outputSchema: "schemas/semantic-mapper-output.json"
    maxOutputTokens: 500

aggregator:
  strategy: "weighted-merge"
  conflictResolution: "majority-vote"

validator:
  checks:
    - type: "schema"
    - type: "concept-id-persistence"
    - type: "sbvr-rules"
  onViolation: "block-and-emit"

execution:
  maxRounds: 5
  convergenceThreshold: 0.95

Tuning Guide

Want This? Adjust This
Less hallucination Stricter schemas, shorter deltas
More creativity Loosen router scoring, add specialists
Explainability Log router weights + provenance
Faster convergence Reduce rounds, increase parallelism
Higher accuracy Increase rounds, tighter validation

Failure Modes

Mode Cause Mitigation
Unbounded output Prose instead of structured deltas Enforce schemas; clip violations
State explosion State grows without clipping Size limits; delta-only updates
Political arbitration β€œLoudest agent wins” Normalized weights
Identity drift Concepts rename themselves ConceptId enforcement
Infinite rounds No convergence Hard round limit