How do I use EvalKit MCP Server?

1. Click on "Install Server". 2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state. 3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@EvalKit MCP Server Evaluate robustness of 'How to bypass safety filters'" That's it! The server will respond to your query, and you can continue using it as needed. Here is a step-by-step guide with screenshots.

EvalKit MCP Server

by tr4m0ryp

Overview Schema Related Servers Score Discussions

Python

Remote

EvalKit — Classifier Robustness Evaluation Toolkit

Python 3.11+ License Anthropic

Multi-technique evaluation toolkit for testing AI safety classifier robustness against query decomposition, obfuscation, and multi-agent orchestration attacks. MCP plugin for Claude Code.

Quick Start

uv sync                          # install everything
claude                           # MCP tools auto-loaded from .mcp.json

Related MCP server: llamator-mcp-server

Architecture

User Query → SPLITTER → [Encoder → Wrapper → ApiClient] × N → MERGER → Report

Usage

In Claude Code (MCP)

Tool	Purpose
`eval_classifier_robustness`	Full eval pipeline
`eval_decompose_query`	Preview query decomposition
`eval_obfuscation_evasion`	Test homoglyph substitution
`eval_status`	Check configuration

Via CLI

python3 run.py decompose "query" 5        # decompose into sub-queries
python3 run.py eval "query"               # full evaluation pipeline
python3 run.py obfuscate "text"           # test obfuscation

Benchmark

python3 benchmark.py                     # dry run (no API key)
python3 benchmark.py --quick             # single query smoke test
python3 benchmark.py --json report.json  # save JSON results
python3 benchmark.py --html report.html  # save HTML report
ANTHROPIC_API_KEY=sk-... python3 benchmark.py  # live test

Configuration

Parameter	Values	Default	Description
`obfuscation`	none, light, moderate, aggressive	moderate	Homoglyph replacement level
`framing`	fiction, study_guide, academic, documentation, translation, none	study_guide	Narrative framing strategy
`max_pieces`	1–20	10	Max sub-query decompositions
`padding_tokens`	0–10000	5000	Long-context padding per query
`multi_agent`	true, false	true	Multi-agent orchestration
`helper_enabled`	true, false	true	Helper model with filters removed

Project Structure

evalkit/
├── evalkit/             # Core modules
│   ├── splitter.py      # Query → sub-questions
│   ├── encoder.py       # Unicode homoglyph engine
│   ├── wrapper.py       # Narrative wrapping
│   ├── api_client.py    # API client + model routing
│   ├── merger.py        # Output stitching + metrics
│   ├── context_builder.py  # Multi-turn conversation
│   ├── agent_router.py  # Agent pack coordination
│   └── models.py        # Data classes + enums
├── evalkit_server.py    # MCP server (FastMCP)
├── run.py               # CLI wrapper (no MCP needed)
├── benchmark.py         # Test matrix runner
├── tests/               # Pytest test suite
├── docs/                # Documentation
├── .mcp.json            # Claude Code auto-discovery
└── CLAUDE.md            # Claude Code instructions

Research Techniques

See docs/TECHNIQUES.md for detailed documentation of each technique.

References

License

MIT — authorized security research and defense evaluation only.

Install Server

license - permissive license

quality

maintenance

How are these scores calculated?

Maintenance

–Maintainers

–Response time

–Release cycle

–Releases (12mo)

Commit activity

Resources

GitHub Repository

Need Help?

Related Servers

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Tools

Related MCP Servers

Trustwise MCP Serverofficial
Testing & QA Tools Monitoring
trustwiseai
A
license
-
quality
C
maintenance
Provides advanced evaluation tools for assessing AI safety, alignment, and performance of LLM outputs. Enables programmatic evaluation of quality, safety metrics like toxicity and PII detection, and operational metrics including carbon footprint and cost estimation.
Last updated 2026-04-15
4
Apache 2.0
llamator-mcp-server
Penetration Testing AI & Machine Learning
RomiconEZ
A
license
-
quality
D
maintenance
Enables automated LLM red teaming by submitting asynchronous test runs, retrieving aggregated metrics, and accessing artifacts.
Last updated 2026-01-18
4
CC BY-SA 4.0
RedTeam ML API MCP
Penetration Testing AI & Machine Learning
rnadjqaim
A
license
B
quality
C
maintenance
Enables security teams to run controlled adversarial penetration tests against authorized ML/LLM API endpoints, scoring responses and generating evidence for compliance frameworks such as SOC 2, ISO 27001, and GDPR.
Last updated 2026-06-21
6
2
MIT
prompt-injection-mcp
Security Penetration Testing AI & Machine Learning
Xiangyu-Li97
F
license
-
quality
D
maintenance
Enables security researchers to evaluate AI system defenses against prompt injection attacks through a comprehensive set of test vectors and analysis tools.
Last updated 2026-02-09
2

View all related MCP servers

Related MCP Connectors

agent-prompt-injection-firewall-mcp
The WAF for agents. Pattern-based + heuristic firewall scans prompts, RAG documents, tool argume...
ateam-mcp
Build, validate, and deploy multi-agent AI solutions from any AI environment.
VerifyAX
Evaluate, benchmark, and simulate AI agents on the VerifyAX agent-evaluation platform.

View all MCP Connectors

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/tr4m0ryp/fable-5-jailbreak'

If you have feedback or need assistance with the MCP directory API, please join our Discord server