synapse-router
Routes agent steps to Google Gemini endpoints (gemini-2.5-flash-lite and gemini-2.5-flash) across the FAST_LOCAL and BALANCED tiers, including HTTP 429 fast-trip circuit breaking and failover to peer providers.
Routes agent steps to locally hosted Ollama models (e.g., ollama/qwen2.5-coder-7b) as the cheapest FAST_LOCAL tier, with health tracking and latency EMA used for failover and Pareto utility scoring.
Routes complex reasoning steps to OpenAI frontier reasoning endpoints (openai/o3-reasoning) as the final cascade tier when lower-cost models fail schema/confidence validation.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@synapse-router route this agent step to the cheapest healthy model and show the receipt"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
synapse-router
Cost-Aware Multi-Model AI Agent Router, Confidence-Gated Cascade Escalation, 3-State Circuit Breaker & 64-Bit SimHash Semantic Cache Gateway.
🚀 Live Interactive Developer Portal & Playground: Visit
https://synapse-router.vercel.appin your web browser to test routing live, inspect schemas, and copy 1-click configurations for Claude Code, Cursor, Windsurf, and Claude Desktop.
Why synapse-router? (The 2026 Multi-Agent Infrastructure Bottleneck)
Production multi-agent systems (LangGraph, CrewAI, Pydantic AI, and MCP tool loops) execute 10–50 LLM calls per user workflow. Sending every trivial intent classification, entity extraction, JSON formatting, or tool-routing step to a frontier reasoning model (o3-reasoning, claude-3-7-sonnet) wastes 75–90% of token budgets and violates sub-second latency SLAs—while relying on a single model provider causes total workflow failure during HTTP 429 rate-limit spikes or regional 5xx outages.
synapse-router is a zero-external-API-key, deterministic Multi-Model Agent Routing, Confidence Cascade, Circuit-Breaking & Semantic Caching Engine with a built-in JSON-RPC 2.0 Model Context Protocol (MCP) Server and reproducible CLI benchmark suite.
Related MCP server: llm-router-mcp
System Architecture
flowchart TD
Req["AgentStepRequest<br/>(prompt, tools_schema, require_json_schema, latency_sla_ms, max_cost_usd)"] --> Budget["TokenBudgetGovernor<br/>Hard Cap (100%) & Soft-Cap Tier Downgrade (80%)"]
Budget -->|Allowed| Cache["SemanticPromptCache<br/>64-Bit Char/Word Shingle SimHash (Hamming <= 3)<br/>+ Tool/Schema Context Fingerprint Isolation"]
Cache -->|Cache Hit (0.0 USD)| Receipt["Signed RoutingDecisionReceipt<br/>(HMAC-SHA256 Audit Trail)"]
Cache -->|Cache Miss| Classifier["TaskComplexityClassifier<br/>Reasoning Markers + Code/AST Density + Tool Depth + Token Volume"]
Classifier --> TierSelect["Pareto Utility Endpoint Selector<br/>U(m) = Q_prior - w_c * Cost_norm - w_l * Latency_EMA"]
Health["EndpointHealthRegistry<br/>3-State Circuit Breaker (CLOSED / OPEN / HALF_OPEN)<br/>HTTP 429 Fast-Trip & EMA Latency Tracker"] <--> TierSelect
TierSelect --> Fast["Tier 1: FAST_LOCAL<br/>(ollama/qwen2.5-coder-7b, gemini-2.5-flash-lite)"]
Fast --> Validator["ResponseValidator<br/>JSON Schema + Non-Empty Tool Args + Refusal Check + Confidence >= min_confidence"]
Validator -->|Valid & Confident| CacheStore["Update SimHash Cache & Budget Ledger"]
Validator -->|Low Confidence / Invalid JSON / HTTP 429| Balanced["Tier 2: BALANCED<br/>(gemini-2.5-flash, claude-3-5-haiku)"]
Balanced --> Validator2["ResponseValidator"]
Validator2 -->|Valid & Confident| CacheStore
Validator2 -->|Escalated| Frontier["Tier 3: FRONTIER_REASONING<br/>(claude-3-7-sonnet, openai/o3-reasoning)"]
Frontier --> CacheStore
CacheStore --> ReceiptKey Capabilities
Sub-Millisecond Prompt & Agent Step Complexity Analyzer (
classifier.py):Deterministic feature extraction across formal proof/architecture markers, code/AST syntax density, recursive JSON/tool schema depth, and log-scaled token volume.
Maps steps into
Tier.FAST_LOCAL,Tier.BALANCED, orTier.FRONTIER_REASONINGwith automatic tight-SLA adjustments (latency_sla_ms < 280ms).
Pareto Utility Router & Confidence-Gated Cascade Engine (
router.py):Evaluates registered
ModelEndpointprofiles against hard capability constraints (supports_tools,supports_json_schema,context_window,latency_sla_ms,max_cost_usd) and ranks healthy candidates by Pareto utility: $$\text{Utility}(m) = Q_{\text{prior}}(m) - w_c \cdot \widehat{C}(m) - w_\ell \cdot \widehat{L}_{\text{EMA}}(m)$$Executes on the selected lower-cost tier first, validates output with
ResponseValidator(strict JSON schema key/type validation, non-empty tool arguments, refusal/truncation detection, andconfidence >= min_confidence), and escalates (FAST_LOCAL -> BALANCED -> FRONTIER_REASONING) only when validation or confidence fails.Emits an HMAC-SHA256 signed
RoutingDecisionReceiptrecording every hop, latency, and exact USD saved vs. frontier-only routing.
Rolling 3-State Provider Circuit Breaker (
health.py):Tracks per-endpoint rolling error rates, HTTP 429 rate-limit counters, 5xx errors, timeouts, and exponential moving average (EMA) latency.
Implements
CLOSED -> OPEN -> HALF_OPENstate transitions with< 0.05 msfailover to healthy peer endpoints.
64-Bit Locality-Sensitive SimHash Semantic Cache (
cache.py):Computes 64-bit SimHash fingerprints over weighted word unigrams, word bigrams, and 4-char shingles using
blake2b.Matches near-identical agent sub-prompts within a configurable bitwise Hamming distance (
<= 3bits) while isolating cache entries by a canonical SHA-256context_fingerprintofsystem_prompt,tools_schema, andrequire_json_schema.
Tenant & Workflow Token/USD Budget Governor (
budget.py):Enforces per-workflow and per-tenant USD and token caps, automatically downgrading model tiers when utilization reaches
80%(soft_cap_downgrade) and blocking execution at100%.
Built-In JSON-RPC 2.0 MCP Server (
server.py):Exposes
route_agent_step,classify_task_complexity,inspect_endpoint_health, andcheck_budget_statusover standard MCPtools/listandtools/call.
Reproducible Benchmark Results (500 Multi-Agent Requests across 6 Endpoints)
Run the benchmark suite directly via CLI:
python -m synapse_router.cli benchmark --requests 500Metric | Measured Value | Notes |
Workload Size |
| Multi-step agent workflow across |
Routing Engine P50 Latency |
| End-to-end classification + SimHash lookup + Pareto scoring + receipt signing |
Routing Engine P95 Latency |
| Includes multi-hop cascade escalation & failover paths |
Frontier-Only Baseline Cost |
| Baseline cost if all 500 steps routed to frontier reasoning tier |
Synapse-Router Actual Cost |
| Actual spend across tiered routing, cascades, and cache hits |
Total USD Cost Reduction |
| 10.6x reduction in multi-agent token spend |
Cascade Quality Retention |
| All 500 completed steps met |
Provider Failover Recovery Rate |
| Zero dropped workflow steps during injected HTTP 429 rate-limit spikes |
Semantic Cache Hit Rate |
| 64-bit SimHash near-duplicate detection ( |
Final Tier Distribution |
|
|
20-Step Multi-Agent Workflow Simulation (demo)
Run the 20-step workflow simulation (featuring injected HTTP 429 rate-limit spikes on google/gemini-2.5-flash, malformed JSON cascade escalation, low-confidence cascade escalation, and SimHash cache hits):
python -m synapse_router.cli demo==================================================================================================
SYNAPSE-ROUTER: 20-STEP MULTI-AGENT WORKFLOW DEMO (CASCADE + CIRCUIT BREAKER + SIMHASH CACHE)
==================================================================================================
STEP ID | SCORE | INIT TIER | FINAL ENDPOINT | HOPS | COST ($)
--------------------------------------------------------------------------------------------------
step-01-intent | 0.04 | FAST_LOCAL | google/gemini-2.5-flash-lite | 1 | 0.000022
step-02-intent-near-dup | 0.04 | FAST_LOCAL | google/gemini-2.5-flash-lite [CACHE] | 0 | 0.000000
step-03-extract-entities | 0.03 | FAST_LOCAL | google/gemini-2.5-flash-lite | 1 | 0.000030
step-04-tool-lookup | 0.45 | BALANCED | google/gemini-2.5-flash | 1 | 0.000232
step-05-cascade-json | 0.05 | FAST_LOCAL | google/gemini-2.5-flash | 2 | 0.000121
step-06-rate-limit-429 | 0.43 | BALANCED | anthropic/claude-3-5-haiku | 2 | 0.001001
step-07-rate-limit-failover | 0.45 | BALANCED | anthropic/claude-3-5-haiku | 2 | 0.001001
step-08-frontier-arch | 0.91 | FRONTIER_REASONING | anthropic/claude-3-7-sonnet | 1 | 0.009240
step-09-tag-severity | 0.02 | FAST_LOCAL | google/gemini-2.5-flash-lite | 1 | 0.000011
step-10-tag-severity-cache | 0.02 | FAST_LOCAL | google/gemini-2.5-flash-lite [CACHE] | 0 | 0.000000
step-11-code-audit | 0.80 | FRONTIER_REASONING | anthropic/claude-3-7-sonnet | 1 | 0.005367
step-12-summarize-slack | 0.03 | FAST_LOCAL | google/gemini-2.5-flash-lite | 1 | 0.000034
step-13-summarize-slack-dup | 0.03 | FAST_LOCAL | google/gemini-2.5-flash-lite [CACHE] | 0 | 0.000000
step-14-sql-query-gen | 0.49 | BALANCED | anthropic/claude-3-5-haiku | 1 | 0.000844
step-15-low-conf-cascade | 0.03 | FAST_LOCAL | anthropic/claude-3-5-haiku | 2 | 0.000510
step-16-frontier-proof | 0.68 | FRONTIER_REASONING | anthropic/claude-3-7-sonnet | 1 | 0.008355
step-17-translate-note | 0.03 | FAST_LOCAL | google/gemini-2.5-flash-lite | 1 | 0.000028
step-18-extract-owners | 0.04 | FAST_LOCAL | google/gemini-2.5-flash-lite | 1 | 0.000020
step-19-postmortem-draft | 0.70 | FRONTIER_REASONING | anthropic/claude-3-7-sonnet | 1 | 0.004356
step-20-final-check | 0.02 | FAST_LOCAL | google/gemini-2.5-flash-lite | 1 | 0.000015
--------------------------------------------------------------------------------------------------
Frontier-Only Baseline Cost : $0.148660
Synapse-Router Actual Cost : $0.031187
Total USD Cost Saved : $0.117473 (79.02% reduction)
Semantic Cache Hits : 3/20
Cascade / Failover Recoveries: 4 (100.0% workflow completion)
==================================================================================================Quickstart
Installation
git clone https://github.com/codesphere-u/synapse-router.git
cd synapse-router
pip install -e ".[dev]"Python SDK Usage
from synapse_router import AgentStepRequest, SynapseRouter
router = SynapseRouter()
# 1. Route a structured JSON extraction step (automatically selects Tier.FAST_LOCAL)
request = AgentStepRequest(
prompt="Classify customer ticket intent into billing, outage, or security_audit.",
require_json_schema={
"type": "object",
"required": ["intent", "priority"],
"properties": {
"intent": {"type": "string"},
"priority": {"type": "string"},
},
},
latency_sla_ms=300.0,
min_confidence=0.80,
tenant_id="acme-corp",
workflow_id="ticket-triage-42",
step_id="step-1",
)
receipt = router.execute_cascade(request)
print(f"Selected Endpoint : {receipt.selected_endpoint_id} ({receipt.final_tier})")
print(f"Actual Cost (USD) : ${receipt.total_cost_usd:.6f} (Saved ${receipt.cost_saved_usd:.6f})")
print(f"HMAC Receipt Sig : {receipt.hmac_sha256_signature}")MCP Server Integration (Claude Desktop / Cursor / LangGraph MCP)
Start the stdio JSON-RPC 2.0 MCP server:
python -m synapse_router.cli mcpOr configure in mcp.json:
{
"mcpServers": {
"synapse-router": {
"command": "python",
"args": ["-m", "synapse_router.cli", "mcp"]
}
}
}Exposed MCP Tools (tools/list) & Detailed Usage Guide
Tool Name | What is its use? (Purpose) | Key Arguments | Typical Output |
| Route an AI agent workflow step across |
|
|
| Compute deterministic task complexity score ( |
|
|
| Inspect 3-state circuit breaker status ( |
|
|
| Query tenant and workflow token/USD budget utilization, remaining allowance, and 80% soft-cap tier downgrade state. |
|
|
| Query sliding-window RPM and TPM consumption, ceiling limits, and saturation status for a model endpoint. |
|
|
| Export canonical RFC 6962 G2_BUDGET_ROUTING attestation leaf for a routing receipt or simulation parameters. |
|
|
Tool Usage Examples
1. route_agent_step — Intelligent Multi-Model Routing with Cost SLA
Route an agent's sub-task with strict latency and cost bounds:
{
"jsonrpc": "2.0",
"id": 1,
"method": "tools/call",
"params": {
"name": "route_agent_step",
"arguments": {
"prompt": "Extract JSON fields from invoice: invoice_id, amount, due_date.",
"latency_sla_ms": 250.0,
"max_cost_usd": 0.002,
"min_confidence": 0.85
}
}
}2. classify_task_complexity — Pre-Execution Task Triage
Determine whether a task needs an expensive frontier model (Claude 3.5 Sonnet / o1) or can be safely delegated to an ultra-fast local model (Llama-3.1-8B):
{
"jsonrpc": "2.0",
"id": 2,
"method": "tools/call",
"params": {
"name": "classify_task_complexity",
"arguments": {
"prompt": "Design a formal TLA+ distributed consensus spec with proof of non-blocking atomicity.",
"tools_schema": []
}
}
}Exposed MCP Resources (resources/list, resources/read)
Resource URI | MIME Type | Purpose / Content Description |
|
| Live catalog of configured model endpoints across tiers ( |
|
| Real-time circuit breaker states, sliding-window rate limit counters, and budget spend utilization across all endpoints. |
Exposed MCP Prompts (prompts/list, prompts/get)
Prompt Name | Purpose / Use Case | Required Arguments | Workflow Output |
| Guides an LLM agent to analyze task complexity, token requirements, and recommend the optimal model tier and fallback cascade. |
| Pre-formatted advisory prompt instructing the model to estimate complexity, recommend tiers, and outline latency SLAs. |
| Guides an LLM agent to design budget limits with 80% soft-cap degradation rules for multi-tenant swarms. |
| Pre-formatted allocation prompt breaking down spend tiers, cache targets, and rate limiting thresholds. |
Architecture Decision Records (ADRs)
ADR-001: Pareto Utility Model Selection & Confidence-Gated Cascade Escalation
ADR-002: 64-Bit Locality-Sensitive SimHash Cache & 3-State Circuit Breaker
Testing
Run the full 93-test suite:
pytest -vAll 93 tests validate Pareto utility routing, SimHash semantic caching, confidence cascade escalation, 3-state circuit breakers, token/USD budget enforcement, rate limiters, RFC 6962 G2 attestation export, and Vercel serverless gateways.
Hosted endpoint (Vercel)
A free, public, stateless deployment of Synapse Router runs over MCP Streamable HTTP:
MCP endpoint |
|
Health probe |
|
curl -s -X POST https://synapse-router.vercel.app/mcp -H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'Client Configuration
1. Claude Code CLI (Recommended)
# Add to current project:
claude mcp add --transport http synapse-router https://synapse-router.vercel.app/mcp
# Or add globally across all projects:
claude mcp add --scope user --transport http synapse-router https://synapse-router.vercel.app/mcp2. Claude Desktop / Cursor / Windsurf (mcp.json)
{
"mcpServers": {
"synapse-router": {
"type": "http",
"url": "https://synapse-router.vercel.app/mcp"
}
}
}3. Smithery Registry (1-Click Install)
npx -y @smithery/cli install @codesphere-u/synapse-router --client claude4. Local Stdio via uvx
{
"mcpServers": {
"synapse-router": {
"command": "uvx",
"args": [
"--from",
"git+https://github.com/codesphere-u/synapse-router.git",
"synapse-router",
"mcp"
]
}
}
}Hosted limitations: All 6 tools are available. Router/cache/budget state is in-memory and ephemeral per serverless instance.
The hosted gateway is api/index.py; it enforces a 1 MB body limit, strict JSON-RPC envelopes and the tool restrictions above. Deploy your own copy with npx vercel deploy --prod (entrypoint is configured in pyproject.toml under [tool.vercel]).
License
Released under the MIT License. Copyright (c) 2026 Akandwanaho Ivan Alvin (@codesphere-u).
Available Tools
6 toolscheck_budget_statusB
Query tenant and workflow token/USD budget utilization, remaining allowance, and 80% soft-cap tier downgrade state.
Inputs: tenant_id, workflow_id.
classify_task_complexityC
Compute deterministic complexity score [0.0, 1.0] and recommended model Tier (FAST_LOCAL, BALANCED, FRONTIER_REASONING) for an agent step.
Inputs: prompt (required), tools_schema, system_prompt, latency_sla_ms, +2 more.
inspect_endpoint_healthB
Inspect 3-state circuit breaker status (CLOSED, OPEN, HALF_OPEN), rolling error rates, HTTP 429 rate-limit counters, and EMA latency for registered model endpoints.
Inputs: endpoint_id.
route_agent_stepC
Route an AI agent workflow step across FAST_LOCAL, BALANCED, and FRONTIER_REASONING models using Pareto cost/latency optimization, SimHash semantic caching, circuit-breaker failover, and confidence-gated cascade escalation.
Inputs: prompt (required), step_id, tenant_id, workflow_id, +8 more.
synapse_export_aegis_g2C
Export canonical RFC 6962 G2_BUDGET_ROUTING attestation leaf for a routing receipt or simulation parameters.
Inputs: task_id, cost_usd, model_id, task_prompt, +2 more.
synapse_rate_limit_statusA
Query sliding-window RPM and TPM consumption, ceiling limits, and saturation status for a model endpoint.
Inputs: endpoint_id (required).
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 6 tools
Each tool has a distinct object: step routing, complexity scoring, rate-limit status, budget status, endpoint health, and attestation export. The three status/query tools (rate_limit, budget, endpoint_health) sit close together and could be briefly confused, but their descriptions clearly separate RPM/TPM, token/USD budget, and breaker/error metrics.
Most tools follow a verb_noun pattern (route_agent_step, classify_task_complexity, inspect_endpoint_health, check_budget_status), but two carry an unrelated 'synapse_' prefix (synapse_export_aegis_g2, synapse_rate_limit_status), and one name uses a cryptic domain acronym (AEGIS G2). The convention is readable but not uniform across the set.
Six tools is well-scoped for a model-routing service: one router, one classifier, three observability queries, and one attestation export. Each tool maps to a distinct operational need with no filler.
The routing/observability loop is covered, but there are no tools to register or list model endpoints, reset an OPEN circuit breaker, or configure budgets/limits — only to read them. These gaps would force an agent to fall back to external configuration channels.
Maintenance
Related MCP Connectors
Hosted MCP server for LLM cost estimation, model comparison, and budget-aware routing.
AI routing, memory, guardrails, and governance. Routes across Claude, GPT, Gemini.
Enterprise AI Control Plane: governance, guardrails, spend tracking, compliance & smart routing.
Discover, compare, route, and execute machine-accessible capabilities for AI agents.
Related MCP Servers
- FlicenseNot gradedqualityDmaintenanceEnables efficient tool routing and context budgeting for AI agents handling hundreds of MCP tools, with circuit breaker, RBAC, and federation for reliable serving.-
- AlicenseNot gradedqualityDmaintenanceMCP server that routes AI chat requests to the best available provider (Anthropic, OpenAI, Groq, etc.) with automatic fallback, budget caps, and model-tier routing.MIT
- FlicenseNot gradedqualityBmaintenanceEnables AI agents and MCP clients to dispatch queries to the most suitable model among 70+ LLMs by scoring intent, budget, cost, and latency, returning structured routing decisions with execution telemetry. It runs as a zero-dependency Python MCP server that plugs into Claude Desktop, Cursor, and other MCP-compatible clients.8-
- AlicenseNot gradedqualityBmaintenanceEnables selecting the best AI model and reasoning effort for any task by analyzing task requirements and live pricing, balancing intelligence, speed, and cost. Provides model recommendations, comparisons, and routing status through MCP tools.26 npm3MIT