A
licenseNot graded
qualityC
maintenanceExposes a deterministic AI-slop scanner and RAG grounding grader as MCP Tools, Resource, and Prompt, enabling any MCP client to evaluate text quality and context faithfulness.
MIT
mcp-llm-behave
by Swanand33
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@mcp-llm-behave Run a behavior test: prompt='Explain AI', expected='define AI', output='AI is...'"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
mcp-llm-behave
MCP server exposing llm-behave behavioral regression testing as callable tools inside Claude Desktop, Claude Code, and any MCP-compatible client.
Runs offline — no API calls, no external services. Uses sentence-transformers for embedding-based similarity.
Tools
Tool | What it does |
| Assert that a model output matches an expected behavior description |
| Detect semantic drift between a baseline and a new LLM output |
| Browse the built-in behavioral checks shipped with llm-behave |
Related MCP server: mycelium
Quickstart — Claude Desktop
Add to your claude_desktop_config.json (no install needed, uvx handles it):
{
"mcpServers": {
"mcp-llm-behave": {
"command": "uvx",
"args": ["mcp-llm-behave"]
}
}
}Config file location:
macOS:
~/Library/Application Support/Claude/claude_desktop_config.jsonWindows:
%APPDATA%\Claude\claude_desktop_config.json
Restart Claude Desktop after editing. The first run downloads the sentence-transformers model (~80 MB) once and caches it.
Quickstart — Claude Code (CLI)
claude mcp add mcp-llm-behave uvx mcp-llm-behaveInstall via pip / uv
pip install mcp-llm-behave
# or
uv add mcp-llm-behaveRun the server directly:
mcp-llm-behaveTool reference
run_behavior_test
Check whether a model output semantically satisfies an expected behavior.
Arguments
Name | Type | Description |
| str | The original prompt sent to the LLM (used for context/logging) |
| str | Plain-language description of what the output should do |
| str | The actual text returned by the LLM |
Returns
{
"score": 0.82,
"passed": true,
"threshold": 0.45
}compare_outputs
Detect semantic drift between a known-good baseline and a new output. Useful in CI after prompt or model changes.
Arguments
Name | Type | Description |
| str | The reference / previous LLM output |
| str | The new LLM output to compare |
Returns
{
"similarity_score": 0.91,
"drift_detected": false,
"interpretation": "Outputs are nearly identical — no drift."
}list_builtin_behaviors
Returns the catalog of pre-defined behavioral checks available in llm-behave, with method signatures and descriptions.
Returns — list of objects with name, method, and description keys.
Requirements
Python 3.10+
No API keys needed
~80 MB disk for the sentence-transformers model (downloaded once on first run)
Development
git clone https://github.com/Swanand33/mcp_llm_behave
cd mcp-llm-behave
uv sync
uv run pytestLicense
MIT — see LICENSE.
Available Tools
3 toolscompare_outputsA
Compare two LLM outputs for semantic similarity (regression detection).
Useful for catching silent model regressions: run this in CI against a known-good baseline output to detect drift when you change prompts or models.
Inputs: baseline (required), candidate (required). Returns: object
list_builtin_behaviorsA
Return the catalog of built-in behavioral checks available in llm-behave.
Returns: list of dicts, each with 'name', 'method', and 'description' keys.
Inputs: no named parameters. Returns: result
run_behavior_testA
Run a single behavioral assertion using llm-behave embedding similarity.
Checks whether the model output semantically satisfies the expected behavior. Uses sentence-level max similarity so long outputs are handled correctly.
Inputs: prompt (required), expected_behavior (required), model_output (required). Returns: object
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
A4.1/5.0
Scored across 3 tools
Disambiguation5/5
Each tool has a distinct and clearly separated purpose: comparing outputs, listing built-in behaviors, and running a behavior test. No overlap or ambiguity.
Naming Consistency5/5
All tool names follow the same verb_noun pattern in snake_case: compare_outputs, list_builtin_behaviors, run_behavior_test. Perfectly consistent.
Tool Count4/5
Three tools is small but sufficient for the server's focused domain of LLM behavior testing. The set covers the essential operations without being overly minimal.
Completeness4/5
The tools cover listing available behaviors, running a test, and comparing outputs. While additional features like adding custom behaviors could be useful, the core workflow is complete.
Maintenance
ActivityInactive
ResponsivenessNo recent community issues
Related MCP Connectors
MCP-native AI evaluation: rubric audits, eval suites, and proof reports for AI/LLM output.
Find, vet, and run MCP tools through a secure audited gateway with prompt-injection risk scoring
Free MCP tools to validate multi-agent coordination thread turns and retrieve schema/samples.
3rd Generation Testing (3TG) — generate deterministic test suites from Markdown spec tables via MCP.
Related MCP Servers
- AlicenseNot gradedqualityCmaintenanceProvides MCP tools for semantic search over personal knowledge sources using pluggable embeddings and local vector indexing.1MIT
- AlicenseAqualityBmaintenanceMCP server exposing statistical regression testing for LLM agents as a "run" tool: p-value, effect size, and confidence interval on whether agent behavior actually changed.168 PyPIApache 2.0
- FlicenseNot gradedqualityDmaintenanceProvides MCP tools that give LLM agents a full QA engineer workflow: scanning projects, generating deterministic test suites, executing them across browser/API/mobile, diagnosing failures, and proposing fixes that require human approval.-