Provides pre_tool_call hooks for schema validation, loop detection, circuit breaker, context budget, and routing awareness in small-model agentic workflows via Hermes plugin.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@Small-Model Harness MCP Classify task: extract product names from reviews"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Defensive harness for running 1B–12B parameter models in production agentic workflows.
Five-layer architecture: routing → validation → constraint → circuit break → context management. Delivered as a Hermes plugin (deterministic enforcement) + MCP server (analysis tools).
Small models (1B–12B) have dramatically closed the quality gap with frontier models, but the reliability gap persists:
Failure Mode | Impact | Source |
Doom Loops — repetitive death spirals under greedy sampling | Qwen3.5-4B: 22.9% loop rate | Antidoom (FTPO) |
Context Rot — degradation with input length, not position | Effective window = ~1/3 of stated window | Chroma/NVIDIA |
Format Drift — tool call fragility on long chains | 87% accuracy vs GPT-4o's 92% | Qwen3-32B eval |
Related MCP server: RelayPlane
┌─────────────────────────────────────────┐
│ ENTRY POINT / TASK DISPATCH │
└────────────────┬────────────────────────┘
│
┌────────────────▼────────────────────────┐
LAYER 1 │ TASK CLASSIFIER │
ROUTING │ Classify complexity, route to model │
│ T1 (<4B) → T2 (4-8B) → T3 (9-12B) → │
│ T4 (cloud) cascade with confidence │
└────────────────┬────────────────────────┘
│
┌────────────────▼────────────────────────┐
LAYER 2 │ PRE-CALL VALIDATION │
GUARDRAIL │ Schema check, loop detection, budget │
│ Blocks bad calls BEFORE execution │
└────────────────┬────────────────────────┘
│
┌────────────────▼────────────────────────┐
LAYER 3 │ CONSTRAINED DECODING ENGINE │
OUTPUT │ XGrammar token masking (40μs) │
ENFORCEMENT │ Guarantee valid JSON/tool call output │
│ (NOT YET IMPLEMENTED) │
└────────────────┬────────────────────────┘
│
┌────────────────▼────────────────────────┐
LAYER 4 │ CIRCUIT BREAKER │
LOOP │ 3-state: closed → open → half-open │
DETECTION │ Detect loops, break circuits, escalate │
└────────────────┬────────────────────────┘
│
┌────────────────▼────────────────────────┐
LAYER 5 │ CONTEXT BUDGET │
CONTEXT │ Sliding window compaction │
│ 1/3 effective window rule │
└─────────────────────────────────────────┘Component | What It Does | Status |
Hermes Plugin |
| ✅ Phase 1 |
MCP Server | 5 tools: context status, compaction, task classification, routing, reset | ✅ Phase 2+3 |
Task Classifier | Rule-based complexity scoring and model tier assignment | ✅ Phase 3 |
Context Router | Tier cascade (T1→T2→T3→T4) with failure-based escalation | ✅ Phase 3 |
Output Enforcement | Constrained decoding (XGrammar/Outlines) for guaranteed valid output | ⏳ Planned |
Output Verifier | Post-generation validation of tool call correctness | 📋 Planned |
# Copy plugin to Hermes plugins directory
cp -r hermes-plugin ~/.hermes/profiles/dev/plugins/small-model-harness
# Enable it
hermes plugins enable small-model-harnessAdd to ~/.hermes/config.yaml or ~/.hermes/profiles/dev/config.yaml:
mcp_servers:
small-model-harness:
command: python3
args: ["/path/to/mcp-server/server.py"]
enabled: trueTool | Description |
| Query context budget utilization for a session |
| Compact session context (sliding window summarization) |
| Classify task complexity and suggest model tier |
| Route task to model tier with cascade logic |
| Reset all harness state for a session |
Tier | Model Size | Example Models | Use Case |
T1 | <4B | SmolLM3, phi-4-mini | Simple extraction, classification |
T2 | 4-8B | Qwen3-8B, Llama 3.2-8B | Single tool calls, basic routing |
T3 | 9-12B | Ornith-1.0-9B, Qwen3-30B-A3B | Multi-step, reasoning, planning |
T4 | Cloud | DeepSeek V4, GPT-4o | Complex chains, security-critical |
# Setup
uv sync --extra dev
# Run tests
uv run pytest tests/ -v
# Test MCP server
echo '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2026-07-28","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}' | python3 mcp-server/server.pyWorkflow | Status |
CodeQL |
|
MIT
This server cannot be deployed
AI routing, memory, guardrails, and governance. Routes across Claude, GPT, Gemini.
Intelligent context infrastructure for AI teams: knowledge graph, sessions, tasks, documents.
Git-backed platform for skills, tools, and context for AI agents
Customer-context and ops layer for AI agents
