<p align="center">
  <img src="./assets/harness-router-logo.png" width="500" alt="harness-router">
</p>

<p align="center">
  <strong>Framework-agnostic Jev tool routing for agentic harnesses.</strong>
</p>

<code>harness-router</code> uses [TypeSafeAI Jev](https://www.typesafe.ai/) through the [OpenRouter Decisions API](https://openrouter.ai/) as a fast **System-1 tool-selection layer**.

At decision points where several tools are genuinely plausible, give Jev a compact harness state and a fixed set of candidate tools. Jev chooses the next action; your main planner remains responsible for deep reasoning, free-form argument generation, code generation, obvious linear steps, and ambiguous tasks.

~~~text
User goal + observation
        |
        v
  harness-router
        |
        +---- high confidence ----> selected tool
        |
        +---- low/uncertain ------> planner fallback
~~~

## Why harness-router?

Agentic systems often spend an expensive model call on a decision that is fundamentally discrete:

- read a file or search the repository?
- inspect or mutate?
- run tests or continue editing?
- click, navigate, or extract?
- which MCP or generic tool should run next?

<code>harness-router</code> separates **tool selection** from **reasoning and execution**.

The router is intentionally small:

- Jev-backed discrete routing through OpenRouter
- hybrid Jev + planner fallback
- hierarchical routing for larger tool registries
- optional bounded Monte Carlo Tree Search (MCTS) for multi-step lookahead
- confidence-aware decisions
- MCP and generic tool adapters
- native MCP server over stdio with fast <code>route</code> and bounded <code>route_mcts</code>
- conservative risk metadata
- optional loop detection for long-running harnesses
- no required MCP SDK dependency in the core install
- async Python API
- CLI aliases: <code>harness-router</code> and <code>har</code>

> Jev confidence is never authorization. Keep your normal execution policy, permissions, approval gates, and sandbox boundaries around tool execution.

## Installation

Requires **Python 3.11+**.

~~~bash
uv tool install --force --with 'mcp>=2,<3' 'git+https://github.com/Protocol-Lattice/harness-router.git@main'
~~~

For development:

~~~bash
git clone https://github.com/Protocol-Lattice/harness-router.git
cd harness-router
python -m pip install -e ".[dev]"
~~~

## OpenRouter setup

The default provider uses:

~~~text
model:    typesafe/jev-1.13
endpoint: https://openrouter.ai/api/alpha/decisions
env:      OPENROUTER_API_KEY
~~~

Set your API key:

~~~bash
export OPENROUTER_API_KEY="your-key"
~~~

Do not commit, log, print, echo, or place the key in prompts/routing state. Treat credentials as non-observable runtime inputs. If you only need to check configuration, test whether the environment variable is present without printing its value.

The OpenRouter provider also refuses to send a Jev routing payload if it contains the configured API key value and redacts the key from provider error text.

## CLI

After installation, both commands are available:

~~~bash
harness-router --version
har --version
~~~

Route a tool directly from the terminal:

~~~bash
har route \
  --goal "Fix the failing parser test" \
  --observation "The failing assertion references src/parser.py" \
  --tools-json '[
    {
      "name": "read_file",
      "description": "Read a repository file by path",
      "category": "inspect",
      "risk": "low"
    },
    {
      "name": "search_code",
      "description": "Search source code for a symbol or text",
      "category": "inspect",
      "risk": "low"
    },
    {
      "name": "write_file",
      "description": "Replace a repository file with new content",
      "category": "mutate",
      "risk": "medium"
    }
  ]'
~~~

Example response:

~~~json
{"category":"inspect","confidence":0.93,"fallback":false,"fallback_reason":null,"tool":"read_file"}
~~~

Pass `--verbose` when you need the full probability map for diagnostics.

## Python quick start

~~~python
import asyncio

from harness_router import (
    HarnessState,
    JevToolRouter,
    OpenRouterConfig,
    OpenRouterJevProvider,
    RiskLevel,
    RoutingConfig,
    ToolDescriptor,
)


async def main() -> None:
    provider = OpenRouterJevProvider.from_config(OpenRouterConfig())
    router = JevToolRouter(provider, RoutingConfig())

    tools = [
        ToolDescriptor(
            name="read_file",
            description="Read a repository file by path",
            category="inspect",
            risk=RiskLevel.LOW,
        ),
        ToolDescriptor(
            name="search_code",
            description="Search repository source code",
            category="inspect",
            risk=RiskLevel.LOW,
        ),
        ToolDescriptor(
            name="write_file",
            description="Replace a repository file",
            category="mutate",
            risk=RiskLevel.MEDIUM,
        ),
    ]

    try:
        decision = await router.route(
            HarnessState(
                goal="Fix the failing parser test",
                observation="The failure points to src/parser.py",
            ),
            tools,
        )

        if decision.fallback:
            print("Planner fallback:", decision.fallback_reason)
        else:
            print("Selected:", decision.tool)
            print("Confidence:", decision.confidence)
    finally:
        await provider.aclose()


asyncio.run(main())
~~~

## The routing model

<code>HarnessState</code> keeps the decision context intentionally compact:

~~~python
HarnessState(
    goal="Fix the failing parser test",
    observation="The assertion references src/parser.py",
    last_action="search_code",
    constraints=["Do not modify generated files"],
)
~~~

A candidate action is represented by <code>ToolDescriptor</code>:

~~~python
ToolDescriptor(
    name="read_file",
    description="Read a repository file by path",
    category="inspect",
    risk=RiskLevel.LOW,
)
~~~

The result is a <code>RouteDecision</code> containing:

- selected tool, when one is appropriate
- inferred or explicit category
- confidence
- per-choice probabilities
- fallback flag
- fallback reason

## Routing modes

Three routing modes are available.

### Hybrid

Recommended default.

~~~python
from harness_router import RoutingConfig, RoutingMode

config = RoutingConfig(
    mode=RoutingMode.HYBRID,
    direct_execution_threshold=0.85,
    fallback_threshold=0.60,
)
~~~

Default behavior:

| Confidence | Result |
| --- | --- |
| < 0.60 | planner fallback |
| 0.60 - 0.85 | planner confirmation |
| >= 0.85 | tool can be routed directly, subject to your execution policy |

### Jev only

~~~python
RoutingConfig(mode=RoutingMode.JEV_ONLY)
~~~

Provider/router errors propagate instead of silently falling back to the planner.

### Planner only

~~~python
RoutingConfig(mode=RoutingMode.PLANNER_ONLY)
~~~

The router immediately returns a planner fallback decision. No Jev provider is required.

## Monte Carlo Tree Search

For decisions where the best **first** tool depends on what is likely to happen several
steps later, use `MCTSToolRouter`.

MCTS is intentionally optional and bounded. It requires a side-effect-free
`SearchEnvironment` that predicts:

- which tools would be available in a simulated state
- the next simulated `HarnessState`
- an immediate reward
- a heuristic value for a simulated state

The simulator must not execute real shell commands, writes, browser actions, or other
external side effects. Only the final first action selected by the search should go through
the normal harness executor and approval policy.

Jev can be supplied as a policy prior. By default, MCTS performs at most **one policy-router
evaluation** at the first ambiguous node (normally the root). The remaining simulations
are local, so increasing `simulations` does not automatically multiply policy-router
calls. For registries above `hierarchical_threshold`, that one router evaluation may
internally use category-first Jev routing.

~~~python
from harness_router import (
    HarnessState,
    MCTSConfig,
    MCTSToolRouter,
    SimulatedStep,
)


class Simulator:
    async def tools(self, state):
        return available_tools_for(state)

    async def transition(self, state, tool):
        # Predict only; do not execute the real tool here.
        next_state, reward, terminal = predict(state, tool)
        return SimulatedStep(next_state, reward=reward, terminal=terminal)

    async def evaluate(self, state):
        return heuristic_value(state)


mcts = MCTSToolRouter(
    Simulator(),
    policy_router=router,  # optional Jev prior
    config=MCTSConfig(
        simulations=64,
        max_depth=4,
        max_policy_evaluations=1,
    ),
)

result = await mcts.search(state, tools)

print(result.decision.tool)
print(result.principal_variation)
print(result.root_visits)
~~~

`route(...)` is also available when you only want the resulting `RouteDecision`.

Search uses a PUCT-style selection score. `MCTSResult` additionally exposes the principal
variation, root visit counts, root action values, simulation count, and the number of Jev
policy evaluations used. For an MCTS-produced `RouteDecision`, `confidence` is the
selected root action's **visit share** and `probabilities` are normalized root visit
counts; they are not calibrated Jev confidence values. If the policy router falls back,
MCTS uses a neutral prior rather than overriding that fallback with its probability map.


## Hierarchical routing

Large flat tool lists are harder to route efficiently, but category-first routing costs an
extra network round trip and repeats the state.

When the number of available tools exceeds <code>hierarchical_threshold</code> (default:
<code>24</code>), <code>JevToolRouter</code> now estimates the request size of flat routing
versus category-first routing. It only pays for the second Jev request when the hierarchical
shape is estimated to save at least <code>hierarchical_min_savings_ratio</code> (default:
<code>15%</code>) of the routing input. Otherwise it stays flat and completes in one request:

~~~text
                 +--> inspect --> read_file / search_code / list_files
Harness state -->+--> mutate  --> write_file / patch_file
                 +--> verify  --> run_tests / lint
                 +--> git     --> diff / commit
~~~

First Jev selects a category, then it selects a tool inside that category.

Set <code>adaptive_hierarchy=False</code> to force the previous threshold-only behavior.

Repeated calls with the exact same compact state and tool registry are served from a bounded
in-process LRU cache (default: <code>128</code> routes). Set <code>route_cache_size=0</code>
to disable it.

The OpenRouter provider also sends router-generated JSON state as a native JSON object rather
than a JSON-escaped string. OpenRouter's Decisions API supports structured state directly,
which avoids unnecessary wire bytes while preserving the provider-agnostic string protocol
used by custom providers.

You can provide categories explicitly or let the built-in adapter infer common categories such as:

- <code>inspect</code>
- <code>mutate</code>
- <code>execute</code>
- <code>verify</code>
- <code>git</code>
- <code>browser</code>
- <code>memory</code>
- <code>network</code>
- <code>finish</code>
- <code>general</code>

## Tool adapters

You do not need to convert every tool registry manually.

### Generic tools

~~~python
from harness_router import normalize_tools

tools = normalize_tools([
    {
        "name": "read_file",
        "description": "Read a file",
        "input_schema": {
            "type": "object",
            "properties": {
                "path": {"type": "string"}
            }
        }
    }
])
~~~

### MCP

~~~python
from harness_router import MCPToolAdapter, normalize_tools

tools = normalize_tools(mcp_tools, adapter=MCPToolAdapter())
~~~

The adapters normalize tool names, descriptions, schemas, categories, and risk metadata. The MCP adapter does not require an MCP SDK dependency.


## Safety and execution policy

Routing and execution are deliberately separate.

A high-confidence Jev decision means:

> "This is probably the best candidate tool."

It does **not** mean:

> "This action is authorized."

The package includes a conservative <code>DefaultExecutionPolicy</code>:

~~~python
from harness_router import DefaultExecutionPolicy

policy = DefaultExecutionPolicy()
allowed = await policy.allow(decision, tool)
~~~

By default:

- low-risk tools may be allowed
- medium-risk tools require an explicit configured confidence threshold
- high- and critical-risk tools are denied

For production harnesses, implement your own <code>ExecutionPolicy</code> around the permissions and approval model of your application.

## Keep argument generation in the planner

Jev should choose among known alternatives.

Good routing questions:

- read vs search
- inspect vs mutate
- which browser action to take
- which MCP or generic tool to call
- run tests vs inspect another file
- which small fixed action advances the current state

Keep these tasks in the main reasoning model:

- generating source code
- writing patches
- constructing non-trivial shell commands
- generating long free-form arguments
- open-ended planning
- interpreting ambiguous intent
- deciding user authorization

A cost-aware agent loop looks like this:

~~~text
1. Main planner creates/updates the goal
2. Harness builds compact state
3. If tool choice is genuinely ambiguous, harness-router chooses the next tool
4. If multi-step consequences matter and a safe simulator exists, optionally run MCTS
5. Otherwise, use the planner's obvious next tool directly
6. Main planner generates required arguments
7. Execution policy checks permission
8. Harness executes the tool
9. Result becomes the next observation
~~~

## Stateful routing and loop detection

The core router is stateless.

For long-running agent loops, <code>RoutingSession</code> adds:

- route-step counting
- configurable maximum route steps
- repeated-fallback circuit breaking
- repeated-action detection
- A/B loop detection

~~~python
from harness_router import RoutingSession

session = RoutingSession(router)

decision = await session.route(state, tools)

loop_detected = session.record_execution(
    "read_file",
    arguments={"path": "src/parser.py"},
    result_class="success",
)

if loop_detected:
    # Escalate to the planner, change strategy, or stop.
    ...
~~~

The session reports loops and opens the fallback circuit after repeated planner fallbacks. Call `reset_fallbacks()` after the planner materially changes the routing state.

## Custom provider

<code>JevToolRouter</code> depends on the small <code>DecisionProvider</code> protocol rather than directly on OpenRouter.

A custom provider only needs to implement:

~~~python
async def choose(
    *,
    state: str,
    instructions: str,
    criteria: Mapping[str, str],
) -> ChoiceDecision:
    ...
~~~

This keeps the routing layer provider-agnostic while <code>OpenRouterJevProvider</code> provides the default Jev integration.

## Native MCP server

For Codex and other MCP hosts, prefer the native MCP server over the routing-helper skill.
It exposes two tools: fast `route` for ordinary ambiguity and bounded `route_mcts` for multi-step lookahead over a caller-supplied side-effect-free state graph. The Jev provider stays alive for the process lifetime so HTTP connections and the router cache are reused.

Install the MCP server:

~~~bash
uv tool install --force --with 'mcp>=2,<3' 'git+https://github.com/Protocol-Lattice/harness-router.git@main'
export OPENROUTER_API_KEY="your-key"
~~~

Start the stdio server:

~~~bash
harness-router-mcp
~~~

The MCP tool accepts a compact payload:

~~~json
{
  "goal": "Fix the failing parser test",
  "observation": "Failure points to src/parser.py",
  "tools": [
    {"name": "read_file", "description": "Read source", "category": "inspect", "risk": "low"},
    {"name": "search_code", "description": "Search repo", "category": "inspect", "risk": "low"},
    {"name": "run_tests", "description": "Run tests", "category": "verify", "risk": "low"},
    {"name": "write_file", "description": "Write source", "category": "mutate", "risk": "medium"}
  ]
}
~~~

The fast `route` response intentionally omits the full probability map:

~~~json
{"tool":"read_file","confidence":0.93,"fallback":false,"reason":null}
~~~

MCTS is available through `route_mcts`. It does **not** execute real tools during search. The caller supplies predicted states, available tools, transitions, rewards, and heuristic state values. By default use about 32 simulations and depth 3; Jev may be used once as a root policy prior, while the remaining simulations stay local.

For Codex, add the stdio server to `~/.codex/config.toml`:

~~~toml
[mcp_servers.harness-router]
command = "harness-router-mcp"
~~~

Then keep the instruction small: use `route` only at genuine ambiguity points; skip it for obvious linear steps. Use `route_mcts` only when downstream consequences matter and the host can provide a side-effect-free simulated graph. Never use real writes, shell commands, browser mutations, or network mutations as MCTS transitions. The fast route uses equal `0.72` direct/fallback thresholds, a 2 second provider timeout, compact state fields, and flat routing through 48 candidates.

## Codex skill

This repository includes a ready-to-use skill at:

~~~text
skills/harness-router/SKILL.md
~~~

For Codex, copy the <code>skills/harness-router</code> directory into a location Codex scans for skills, or expose that directory through your existing Codex skill configuration.

The skill tells Codex to use Jev selectively for **ambiguous tool selection**, while retaining Codex for reasoning and free-form argument generation.

For unmodified Codex, prefer the native MCP server above. If you use the helper skill instead, it is limited to at most one routing-helper call per task, skips obvious linear tool steps, and stops routing after the first fallback or planner override. The helper emits compact JSON by default; use `--verbose` only for diagnostics.

It also contains a direct routing helper:

~~~bash
python skills/harness-router/scripts/route.py \
  --goal "Fix the failing parser test" \
  --observation "The failing assertion points to src/parser.py" \
  --tools-json '[
    {"name":"read_file","description":"Read a source file","category":"inspect","risk":"low"},
    {"name":"search_code","description":"Search repository text","category":"inspect","risk":"low"},
    {"name":"write_file","description":"Write a source file","category":"mutate","risk":"medium"}
  ]'
~~~

## Configuration

### RoutingConfig

~~~python
RoutingConfig(
    mode=RoutingMode.HYBRID,
    direct_execution_threshold=0.85,
    fallback_threshold=0.60,
    hierarchical_threshold=24,
    max_same_action_repeats=2,
    max_route_steps=50,
    max_consecutive_fallbacks=2,
    description_limit=160,
    history_limit=3,
    state_field_limit=800,
    constraint_limit=4,
)
~~~

### MCTSConfig

~~~python
MCTSConfig(
    simulations=64,
    max_depth=4,
    exploration_constant=1.5,
    discount=0.95,
    max_policy_evaluations=1,
    min_prior=1e-6,
)
~~~

### OpenRouterConfig

~~~python
OpenRouterConfig(
    model="typesafe/jev-1.13",
    url="https://openrouter.ai/api/alpha/decisions",
    api_key_env="OPENROUTER_API_KEY",
    timeout_seconds=5.0,
)
~~~

## Error handling

In hybrid mode, provider failures are converted into planner fallback decisions.

Common fallback reasons include:

- <code>planner_only</code>
- <code>no_tools</code>
- <code>no_matching_tool</code>
- <code>no_matching_category</code>
- <code>low_confidence</code>
- <code>low_category_confidence</code>
- <code>planner_confirmation</code>
- <code>provider_error</code>
- <code>router_error</code>
- <code>max_route_steps</code>
- <code>fallback_circuit_open</code>

In <code>jev_only</code> mode, provider/router failures propagate so the harness can handle them explicitly.

## Architecture

~~~mermaid
flowchart LR
    A[Harness state] --> B[JevToolRouter]
    T[Tool registry] --> D[Adapters]
    D --> B

    B -->|small registry| J[Jev decision]
    B -->|large registry| C[Category decision]
    C --> J
    B -->|optional lookahead| M[MCTS + simulator]
    J -->|policy prior| M

    J --> P{Confidence policy}
    M --> X
    P -->|high| X[Selected tool]
    P -->|uncertain| F[Planner fallback]

    X --> E[Execution policy]
    E -->|allowed| R[Harness executor]
    E -->|denied| F
~~~

## Development

Install development dependencies:

~~~bash
python -m pip install -e ".[dev]"
~~~

Run tests:

~~~bash
pytest
~~~

Lint:

~~~bash
ruff check .
~~~

Type-check:

~~~bash
mypy
~~~

## Project status

<code>harness-router</code> is currently **alpha** software. APIs may still evolve as integrations with real coding, browser, computer-use, generic-tool, and MCP harnesses are exercised.

If you build an integration, benchmark the complete harness loop rather than assuming routing automatically improves latency or cost.

## License

MIT.