Route models by agent difficulty

Switch models automatically based on FSM difficulty state — cheaper models on easy steps, more powerful models when the agent is struggling.

ReasonBlocks tracks the agent’s difficulty across every step using a finite state machine. When difficulty stays low for several consecutive steps, the FSM moves to FAST. When it stays high, the FSM moves to SLOW. Model routing maps those states to different model identifiers, so you can run a cheap fast model on easy steps and a more powerful model when the agent is struggling — with no changes to your agent code.

Model routing is disabled by default. It activates only when you pass model_routing to the ReasonBlocks constructor.

How the FSM states affect routing

The FSM has five states: INIT, FAST, NORMAL, SLOW, and SKIP. Two of them affect model routing:

State	When it activates	Effect on routing
FAST	Last 6 difficulty scores all below 0.2	Routes to the FAST model; also skips E-trace retrieval
SLOW	Last 5 difficulty scores all above 0.6	Routes to the SLOW model
NORMAL	Default; between FAST and SLOW thresholds	No routing override — uses the agent’s configured model
SKIP	Last 35 difficulty scores all above 0.85	No routing override — agent is likely stuck

When the FSM is in FAST state, ReasonBlocks skips the E-trace retrieval pipeline (vector search + embeddings) entirely in addition to switching models. This means fast, confident steps cost less in both LLM tokens and API calls to the pattern store.

Configure routing

Pass a model_routing dict to ReasonBlocks mapping FSM state names ("FAST", "SLOW") to model identifiers. The identifiers follow LangChain’s init_chat_model format — "provider:model-name".

from reasonblocks import ReasonBlocks

rb = ReasonBlocks(
    api_key="rb_live_...",
    model_routing={
        "FAST": "anthropic:claude-haiku-4-5-20251001",
        "SLOW": "anthropic:claude-sonnet-4-20250514",
    },
)

You don’t need to specify both states. Omitting a state means that state uses the agent’s default model:

# Only upgrade to Sonnet when struggling — fast steps use the agent's default
rb = ReasonBlocks(
    api_key="rb_live_...",
    model_routing={
        "SLOW": "anthropic:claude-sonnet-4-20250514",
    },
)

Customize FSM thresholds

The default thresholds (fast_threshold=0.2, slow_threshold=0.6, skip_threshold=0.85) work well for most coding agents. You can tune them for your workload via fsm_thresholds:

rb = ReasonBlocks(
    api_key="rb_live_...",
    model_routing={
        "FAST": "anthropic:claude-haiku-4-5-20251001",
        "SLOW": "anthropic:claude-sonnet-4-20250514",
    },
    fsm_thresholds={
        "fast_threshold": 0.2,   # all steps below this for 6 consecutive steps → FAST
        "slow_threshold": 0.6,   # all steps above this for 5 consecutive steps → SLOW
        "skip_threshold": 0.85,  # all steps above this for 35 consecutive steps → SKIP
    },
)

Lower fast_threshold to keep the agent on the cheap model longer. Raise slow_threshold to delay escalation to the powerful model. Both changes reduce cost but may affect quality on genuinely hard problems.

The FSM also has a hysteresis_margin (default 0.1) that prevents thrashing at state boundaries. A state transition requires the difficulty to cross the threshold by at least this margin before switching back.

Full configuration example

This is the complete configuration from the README, combining model routing with all other available options:

from reasonblocks import ReasonBlocks

rb = ReasonBlocks(
    api_key="rb_live_...",
    token_budget=100_000,
    monitor_names=["loop", "confidence", "evidence", "budget", "strategy_exhaustion"],
    fsm_thresholds={
        "fast_threshold": 0.2,
        "slow_threshold": 0.6,
        "skip_threshold": 0.85,
    },
    model_routing={
        "FAST": "anthropic:claude-haiku-4-5-20251001",
        "SLOW": "anthropic:claude-sonnet-4-20250514",
    },
    e_traces_enabled=True,
)

Inspect routing decisions

After a run, mw.step_log shows the resolved model ID for each step. The model_id field is only populated when routing overrode the agent’s default model.

mw = rb.middleware(agent_name="bugfixer")
agent = create_agent(..., middleware=[mw])
result = agent.invoke({"messages": [("user", "Fix the bug.")]})

for entry in mw.step_log:
    routed = f" → {entry.model_id}" if entry.model_id else ""
    print(f"Step {entry.step}: {entry.fsm_state}{routed} (difficulty={entry.difficulty:.3f})")

Example output showing a run that starts in NORMAL, drops to FAST for several steps, then escalates to SLOW:

Step 0: NORMAL (difficulty=0.512)
Step 1: NORMAL (difficulty=0.489)
Step 2: NORMAL (difficulty=0.141)
Step 3: NORMAL (difficulty=0.098)
Step 4: NORMAL (difficulty=0.077)
Step 5: NORMAL (difficulty=0.061)
Step 6: FAST → anthropic:claude-haiku-4-5-20251001 (difficulty=0.055)
Step 7: FAST → anthropic:claude-haiku-4-5-20251001 (difficulty=0.043)
Step 8: NORMAL (difficulty=0.731)
Step 9: SLOW → anthropic:claude-sonnet-4-20250514 (difficulty=0.812)