Provides security enforcement and policy control for GitHub API actions, blocking malicious operations such as unauthorized git pushes, secret exfiltration, and path traversal attempts.
AI agents can read your files, push code, query databases, execute shell commands, and call external APIs. They do it autonomously, at machine speed.
Traditional security controls weren't built for that.
Mastyf.ai acts as a perimeter security layer for AI. It intercepts every tool call, evaluates it against your security policies using multi-agent swarm analysis, and blocks malicious or unauthorized actions before they execute.
Every decision is enforced, logged, and auditable.
Related MCP server: Agent Prompt Injection Firewall MCP
Threat | What it looks like |
Prompt injection | Malicious instructions embedded in tool arguments to hijack agent behavior |
Path traversal | Attempts to access |
Secret exfiltration | API keys and tokens leaking through tool arguments |
Shell injection | Reverse shells, |
Data exfiltration | Bulk SQL dumps, |
SSRF | Calls to metadata endpoints, localhost, and private IP ranges |
Encoding evasion | Base64 blobs and Unicode homoglyphs used to bypass pattern detection |
Cost abuse | Runaway agent loops burning through token budgets |
Rug-pull attacks | Tool definitions that silently change mid-session |
Clone the repository and run the setup script.
Requirements:
Git
Linux (the setup script installs Nix automatically if needed)
git clone https://github.com/mastyf-ai/mastyf.ai.git
cd mastyf.ai
chmod +x setup.sh
./setup.shThe setup script automatically:
Installs Nix (if required)
Enables Nix flakes
Creates the development environment
Installs all project dependencies
Rebuilds native packages
Builds the entire project
Adds a convenient mastyf shell alias
Once installation completes, start the proxy and dashboard:
node dist/cli.js startOr simply use the alias after opening a new terminal:
mastyfThe dashboard will be available at:
Dashboard: http://localhost:4000
If the dashboard is running, verify the HTTP bridge:
curl -X POST http://localhost:4000/mcp -H "Content-Type: application/json" -d '{"jsonrpc":"2.0","id":"1","method":"tools/list","params":{}}'Full visibility into every action your AI takes.
Section | What you see |
Protection | Block rate, top triggered rules, live threat feed |
Activity | Every tool call with full arguments, allow or block status, timestamp |
Policy | Live rule editor with hot-reload from YAML |
Threat Lab | AI-suggested attack tests, reviewed and approved before anything applies |
Cost | Token usage and cost estimates broken down per tool call |
Do not expose port 4000 publicly without enabling dashboard auth. The default local dev config has
DASHBOARD_AUTH_DISABLED=true.
Every tool call passes through three layers before it reaches your infrastructure.
Layer 1 - Pattern detection Regex-based scanning for injection, dangerous paths, leaked secrets, shell commands, and encoding tricks. Runs in microseconds with no external dependencies.
Layer 2 - Schema validation Rejects malformed payloads, oversized arguments, and JSON-RPC violations before they reach policy evaluation.
Layer 3 - Semantic review An optional local LLM (Ollama) or cloud model evaluates borderline calls that pass pattern checks. Falls back to heuristics if no model is configured.
Anything that fails is blocked. The tool never runs. Everything is logged.
Your rules live in default-policy.yaml. You own them. mastyf.ai enforces them.
policy:
mode: block
default_action: pass
unicode_strict: true
rules:
- name: block-sensitive-paths
action: block
argPatterns:
- field: path
patterns: ['^/etc/', '/\.ssh', '/\.aws/credentials']
- name: rate-limit-tool-calls
action: block
maxCallsPerMinute: 120
- name: block-shell-injection
action: block
patterns: ['rm\s+-rf', 'curl\s', 'wget\s', '`[^`]+`']Roll out safely with three enforcement modes:
Mode | Behavior | When to use |
| Log everything, block nothing | First week, understand what your AI does |
| Log and flag, still forwards | Tuning phase before enforcement |
| Stops violations before execution | Production |
Pre-built templates for HIPAA, PCI-DSS, GxP, and data residency are in policy-templates/.
mastyf.ai runs two coordinated swarms. The CI Swarm attacks your policy before code ships. The Runtime Swarm enforces and learns from every live tool call in production. Four feedback loops connect them so the system gets harder to bypass over time.
flowchart TB
AI["🤖 AI Clients\nCursor · Claude Desktop · Cline"]
subgraph CI["🔵 CI Swarm (PR + Nightly)"]
direction LR
Scout["🔍 Scout Agent\nSAST, deps, config scan"]
Corpus["📋 Corpus Agent\n228 fixtures eval"]
Evasion["⚡ Evasion Agent\n120+ probes + generate new"]
Parity["🔄 Parity Agent\nNode vs Python"]
ProxyA["🖥️ Proxy Agent\nLive stdio MCP tests"]
Report["📊 Report Agent\nsecurity-swarm/latest.json"]
Scout --> Corpus --> Evasion --> Parity --> ProxyA --> Report
end
subgraph Runtime["🟢 Runtime Swarm (Production Proxy)"]
direction LR
BG["🛡️ BlockGuard\nsync policy"]
IL["📈 InstantLearner\nper-block stats + suggestions"]
SA["🧠 SemanticAuditor\nasync LLM, optional"]
PS["🔗 PatternSynthesizer\nbatch suggestions"]
Cal["⚙️ Calibrator\nlabels + thresholds"]
BG --> IL --> PS --> Cal
BG --> SA --> PS
end
Tools["🗄️ MCP Tools\nfilesystem · GitHub · databases · APIs"]
AI -->|"every tool call"| BG
BG -->|"✅ allowed"| Tools
Report -->|"🔁 Loop A: bypasses to corpus"| Corpus
Cal -->|"🔁 Loop B: blocks to rules"| BG
Cal -->|"🔁 Loop C: labels to LLM"| SA
Report -->|"🔁 Loop D: CI metrics weekly"| Cal
style CI fill:#EFF6FF,stroke:#3B82F6,stroke-width:2px,color:#1E3A5F
style Runtime fill:#F0FDF4,stroke:#22C55E,stroke-width:2px,color:#14532D
style Scout fill:#DBEAFE,stroke:#3B82F6,color:#1E40AF
style Corpus fill:#DBEAFE,stroke:#3B82F6,color:#1E40AF
style Evasion fill:#DBEAFE,stroke:#3B82F6,color:#1E40AF
style Parity fill:#DBEAFE,stroke:#3B82F6,color:#1E40AF
style ProxyA fill:#DBEAFE,stroke:#3B82F6,color:#1E40AF
style Report fill:#DBEAFE,stroke:#3B82F6,color:#1E40AF
style BG fill:#BBF7D0,stroke:#16A34A,color:#14532D
style IL fill:#BBF7D0,stroke:#16A34A,color:#14532D
style SA fill:#BBF7D0,stroke:#16A34A,color:#14532D
style PS fill:#BBF7D0,stroke:#16A34A,color:#14532D
style Cal fill:#BBF7D0,stroke:#16A34A,color:#14532D
style AI fill:#FEF3C7,stroke:#F59E0B,stroke-width:2px,color:#78350F
style Tools fill:#FEF3C7,stroke:#F59E0B,stroke-width:2px,color:#78350FCanonical gates: 228/228 corpus, 0 bypasses, 100% parity
Runs on every PR and nightly. Six agents work in sequence, each one hardening what the previous found.
Agent | What it does |
Scout | SAST scan, dependency audit, config review |
Corpus | Evaluates all 228 attack fixtures against current policy |
Evasion | Runs 120+ bypass probes and generates novel ones using an LLM |
Parity | Verifies Node and Python implementations produce identical decisions |
Proxy | Live stdio MCP session tests against a running proxy instance |
Report | Writes |
Runs inside the production proxy on every tool call.
Component | What it does |
BlockGuard | Enforces the active policy synchronously on every call. Fail-closed. |
InstantLearner | Tracks per-block statistics and surfaces rule suggestions in real time |
SemanticAuditor | Optional async LLM review for calls that clear pattern checks but look suspicious |
PatternSynthesizer | Batches suggestions from InstantLearner and SemanticAuditor into candidate rules |
Calibrator | Labels candidates, tunes thresholds, and promotes approved rules back into BlockGuard |
Loop | Signal | Effect |
A | CI bypass found | Added to corpus, CI now guards against it permanently |
B | Runtime block pattern | Synthesized into a new rule, promoted to BlockGuard |
C | Calibrator label | Used to fine-tune SemanticAuditor thresholds |
D | CI metrics (weekly) | Updates runtime config — keeps CI and production in sync |
The proxy supports five transports: stdio, HTTP, SSE, streamable HTTP, and WebSocket.
For enterprise deployments with Redis, Postgres, and Kubernetes see docs/ENTERPRISE_DEPLOYMENT.md.
Threat Lab watches live traffic and uses a local LLM to propose new attack test cases when it detects suspicious patterns. Nothing is applied automatically. You review and approve every suggestion in the dashboard before it becomes a rule.
Approved discoveries feed back into the CI attack corpus for ongoing regression testing.
ollama serve
ollama pull qwen3:8b
export OLLAMA_BASE_URL=http://127.0.0.1:11434
export MASTYF_AI_LLM_PROVIDER=ollama
export MASTYF_AI_LLM_MODEL=qwen3:8b
pnpm dashboard:proxyBefore installing any MCP server from npm, check its trust score at https://www.mastyf.ai/certified. Scores cover CVE exposure, typo-squat risk, maintainer signals, and known attack patterns. Free, no account required.
Command | What it does |
| Start proxy and dashboard on port 4000 |
| Wrap your MCP config to route through the proxy |
| Health check for DB, policy, and environment |
| Scan MCP configs for CVEs and injection risks |
| Run the full test suite |
| Quick security regression, 5 to 15 minutes |
| Full adversarial analysis |
Problem | Fix |
Dashboard shows no data | Proxy and dashboard must share the same |
| Run |
AI still hitting tools directly | Run |
Ollama warnings at startup | Run |
npm install fails | npm publish is not live yet. Use |
A/B test a proposed policy change against historical attack corpus
| Name | Required | Description | Default |
|---|---|---|---|
| proposedPolicyYaml | Yes | Proposed policy YAML to test |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description must disclose behavioral traits. It only states 'A/B test' without clarifying if the tool is read-only, modifies state, requires authorization, or has side effects. This is insufficient for understanding the tool's impact.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no unnecessary words. It is appropriately sized for a simple tool with one parameter.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the tool's simplicity, the description lacks essential context: it does not explain what 'A/B test' means operationally, what the output looks like, or any return value. Since there is no output schema, the description should provide that information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already describes the parameter 'proposedPolicyYaml' adequately, and the description adds no new semantic detail. With 100% schema coverage, a baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool performs an A/B test on a proposed policy change against historical attack corpus, which distinguishes it from sibling tools like 'tune_policy_rule' or 'suggest_policy_improvements'. However, it could be more explicit about the scope and what 'against historical attack corpus' entails.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as when to prefer it over 'tune_policy_rule' or 'generate_policy_from_observations'. It does not mention prerequisites or limitations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
SARSA — adaptively tune rate limit, latency limit, or confidence threshold via reinforcement learning
| Name | Required | Description | Default |
|---|---|---|---|
| fpRate | No | ||
| blockRate | No | ||
| parameter | Yes | ||
| callVolume | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description mentions 'reinforcement learning' but does not disclose side effects, required data, or state changes. Without annotations, the description carries the full burden, yet it provides minimal behavioral insight.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (one sentence) but fails to convey necessary detail. It is concise but under-specified, so it does not earn high marks for efficiency.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (RL algorithm, four parameters, no output schema), the description is highly incomplete. It lacks parameter explanations, output description, and usage context, making it inadequate for an agent to use effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain any parameters. 'fpRate', 'blockRate', 'callVolume' are not described; only 'parameter' is hinted at via the resource list. This is severely lacking.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool adaptively tunes rate limit, latency limit, or confidence threshold using reinforcement learning. The verb 'tune' and specific resources are mentioned, distinguishing it from sibling tools like 'tune_policy_rule' which may use different methods.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. No context on prerequisites, limitations, or appropriate scenarios (e.g., when RL is needed vs manual tuning).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get overall status of all agentic AI features including metrics, scheduler, and task queue
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description implies a read-only operation but does not disclose potential side effects, authentication needs, or response structure. It adds minimal behavioral context beyond the name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is direct and informative. It front-loads the purpose without unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a no-parameter tool with no output schema, the description adequately states what is returned. However, it could benefit from a note about the format or expected content to fully set expectations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameters, so the description need not add parameter details. Although the baseline is 3, the description effectively covers the tool's behavior with no parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and resource 'overall status of all agentic AI features', with specific items like metrics, scheduler, and task queue. It is distinct from sibling tools that focus on specific aspects.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'check_health' or 'observation_status'. The description does not mention exclusions or use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
View all active trust relationships and session details
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description is the sole source of behavioral information. It only states that the tool 'views' data, implying a read-only operation, but does not disclose any other traits such as authentication requirements, rate limits, or what happens when no trust relationships exist.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no extraneous information. It front-loads the key action and target efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and no output schema, the description is sufficient for a simple view operation. However, it could be more complete by hinting at the structure of the returned data (e.g., list of trust relationship objects).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, and the schema coverage is 100%. The description adds no additional meaning beyond what the schema implies, but since there are no parameters, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses the verb 'View' and resource 'all active trust relationships and session details', making the action and target clear. However, it does not explicitly distinguish this tool from sibling tools like evaluate_agent_trust or negotiate_agent_trust, which have similar trust-related scopes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. There is no mention of prerequisites, typical scenarios, or exclusions, leaving the agent to infer usage solely from the tool name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Audit token usage and estimate costs per MCP server
| Name | Required | Description | Default |
|---|---|---|---|
| serverName | No | Filter to a specific server name. If omitted, audits all. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist; the description only conveys the high-level purpose but does not disclose behavioral traits such as being read-only, potential side effects, authentication needs, or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with one sentence that immediately states the action; no unnecessary words, but could be slightly more informative while remaining concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with one optional parameter and no output schema, the description covers the core purpose, but lacks behavioral or execution details that would make it fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with the parameter 'serverName' described; the description does not add additional meaning beyond the schema, so baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool audits token usage and estimates costs per MCP server, which is a specific and distinct action among sibling tools that focus on security, compliance, and trust.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'full_report' or 'compliance_posture'; no when-not-to-use or contextual hints provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Capture current server state as a known-good behavioral baseline
| Name | Required | Description | Default |
|---|---|---|---|
| serverName | Yes | Server name to capture baseline for |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must carry burden. It indicates a read-like action (capture state) but lacks details on side effects, permissions, or data persistence.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no wasted words. Efficient and to the point.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and minimal description, the tool is incomplete for an AI agent. Lacks explanation of what a baseline is, what output to expect, and how it differs from many sibling tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single parameter, so baseline is 3. Description adds context by explaining the purpose (capturing baseline), which enriches the parameter meaning slightly.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'capture' and resource 'server state as a baseline', which is specific. However, it does not distinguish from siblings like check_health or certify_server that also capture state.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives. Does not mention prerequisites, context, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run MCP server certification (Bronze/Silver/Gold/Platinum)
| Name | Required | Description | Default |
|---|---|---|---|
| cveFree | No | ||
| version | No | ||
| transport | No | ||
| authMethod | No | ||
| serverName | No | ||
| trustScore | No | ||
| packageName | No | ||
| complianceScore | No | ||
| trustedPublisher | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits such as required permissions, whether the operation is read-only or destructive, or what happens to the server. The single sentence gives no behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is minimal (one sentence) but not well-structured; it lacks any breakdown or emphasis on key aspects. It is concise but at the expense of clarity, and does not front-load critical information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 9 parameters with no documentation, no output schema, and no annotations, the description is severely incomplete. An agent cannot understand how to properly invoke the tool or interpret results based solely on this description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain any of the 9 parameters. While parameter names like 'serverName' are partially self-explanatory, others like 'trustScore', 'cveFree', and 'complianceScore' lack context, leaving the agent unable to determine their meaning or usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Run' and resource 'MCP server certification' with levels (Bronze/Silver/Gold/Platinum), clearly indicating the action and scope. However, it does not explicitly differentiate from siblings like 'verify_certification' or 'list_certified_servers', leaving some ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. With many sibling tools related to certification (e.g., verify_certification, list_certified_servers), the description provides no context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Check health, latency, and reliability of MCP servers
| Name | Required | Description | Default |
|---|---|---|---|
| serverName | No | Filter to a specific server name. If omitted, checks all. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are absent, so the description bears full burden. It only mentions checking health/latency/reliability with no disclosure of side effects, authentication needs, or return format. The read-only nature is implied but not stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no wasted words. It is front-loaded with the key action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity (1 optional param, no output schema, no annotations), the description is minimally adequate. However, it lacks details on output format, definition of 'health', and any thresholds, leaving room for ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and already describes the sole parameter (serverName: filter or omit for all). The tool description adds no extra semantic value beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool checks health, latency, and reliability of MCP servers, using a specific verb and resource. It effectively distinguishes from sibling tools like check_sla, which focuses on service level agreements.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., check_sla, audit_costs). The description does not specify prerequisites or conditions for use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Check SLA compliance — p50/p95 latency, error rate, circuit breaker state per tool
| Name | Required | Description | Default |
|---|---|---|---|
| toolName | No | ||
| serverName | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It does not disclose whether the tool is read-only, has side effects, or requires specific permissions. The mention of 'circuit breaker state' hints at potential mutation but is not clarified.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, concise sentence front-loads the purpose and metrics. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Without output schema or annotations, the description lacks detail on return format, filtering, and behavior when parameters are omitted. For a tool with two optional string params, more context is needed for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description must explain parameters. It mentions 'per tool' but does not clarify what toolName and serverName represent, their expected format, or whether they are required. Only implicit connection.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool checks SLA compliance with specific metrics (p50/p95 latency, error rate, circuit breaker state) and scope ('per tool'). This distinguishes it from siblings like check_health or audit_costs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives. Sibling tools include check_health and audit_costs, but the description provides no context for differentiation or conditions for use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Identify missing compliance controls and recommend policies
| Name | Required | Description | Default |
|---|---|---|---|
| framework | Yes | Compliance framework |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It only states the action without indicating whether the tool is read-only, modifies state, requires permissions, or has side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single 7-word sentence that is direct and front-loaded. While concise, it could benefit from slightly more detail without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema is present, yet the description does not describe the return format or structure. With many related sibling tools, the description should provide more context to ensure appropriate selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage with an enum and clear description for 'framework'. The tool description adds no additional semantic value beyond the schema, but the schema itself is adequate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool identifies missing compliance controls and recommends policies, which is a specific verb-resource pair. However, it does not differentiate from similar sibling tools like compliance_posture or run_self_assessment.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., compliance_posture, run_self_assessment). The description provides no context for selection among many siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get current compliance posture score across all frameworks
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description lacks behavioral details such as whether the score is real-time or cached, permissions required, or any side effects. The description does not add context beyond the basic purpose.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that is front-loaded with the key action and resource. It could include slightly more information without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema or annotations, the description is too minimal. It does not explain what the score represents or how to interpret it, which an agent would need for proper use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so the schema coverage is 100%. The description does not add interpretation or context about the output, such as the score range or meaning, which would be helpful.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get') and the resource ('current compliance posture score across all frameworks'), making the tool's purpose explicit and distinguishable from siblings like 'compliance_gap_analysis'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, such as when a specific framework score or gap analysis is needed. No prerequisites or conditions for use are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Compute an A+-F trust score for an MCP server across 8 security dimensions (like SSL Labs for MCP)
| Name | Required | Description | Default |
|---|---|---|---|
| maxCvss | No | Maximum CVSS score | |
| cveCount | No | Number of known CVEs | |
| transport | No | Transport protocol | |
| authMethod | No | Authentication method | |
| serverName | No | Server name to score |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so description must disclose behavior. It mentions 8 security dimensions but does not list them, explain calculation, output format, or prerequisites. The analogy helps but is insufficient for full transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is highly concise and front-loaded with the core purpose. No wasted words; the SSL Labs analogy efficiently conveys meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Lacks output schema or description of return value. With 5 optional parameters and no required fields, the agent lacks guidance on minimal inputs. No mention of error states or dependencies, making it incomplete for a scoring tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all 5 parameters with descriptions (100% coverage). The description adds no parameter-specific detail, so baseline 3 applies. It does not improve understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool computes a trust score (A+ to F) for MCP servers across 8 security dimensions, using the SSL Labs analogy which makes the purpose intuitive. It distinguishes from siblings like evaluate_agent_trust by focusing on server security dimensions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives. Siblings include many trust/reputation tools, but the description does not specify scenarios or exclusion criteria, leaving the agent without clear context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Submit an anonymized threat signature to the cross-deployment intelligence mesh
| Name | Required | Description | Default |
|---|---|---|---|
| pattern | Yes | The threat pattern (will be privacy-hashed before sharing) | |
| category | Yes | Attack category | |
| severity | Yes | Severity level |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the main description only mentions 'anonymized' without details on hashing, effects, permissions, or response. Behavioral context relies on schema description for 'pattern' but is insufficient for a submission tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence that is direct and to the point, with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Without output schema or annotations, the description omits crucial details like return value, side effects, or submission confirmation, leaving the agent underinformed for a non-trivial operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions cover 100% of parameters (pattern, category, severity) with meaningful text, but the main description adds no further context beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the verb 'Submit', the resource 'threat signature', and the destination 'cross-deployment intelligence mesh', making it distinct from sibling tools like 'predict_threats' or 'threat_intel_status'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as 'deploy_honeypot' or 'detect_collusion'. The description does not specify prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Declare session intent and allowed tools for intent-binding enforcement
| Name | Required | Description | Default |
|---|---|---|---|
| intent | Yes | ||
| agentId | No | ||
| sessionId | Yes | ||
| ttlMinutes | No | ||
| allowedTools | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description is the sole source of behavioral info. It fails to disclose side effects, required permissions, idempotency, or whether the tool creates, updates, or reads state, which is critical for a tool named 'declare'.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single 11-word sentence, which is concise but sacrifices necessary detail. It is front-loaded with the core action, but the brevity leaves out essential context, making it underinformative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, no output schema, 5 parameters with no descriptions, the description should provide broader context such as return values, state changes, or typical usage. It does not, leaving significant gaps for a tool that likely modifies session state.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage (no parameter descriptions) and the tool description does not explain any of the 5 parameters, including the required 'sessionId', 'intent', and 'allowedTools'. The agent must guess their meaning from names alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action ('Declare session intent and allowed tools') and mentions the context ('intent-binding enforcement'), effectively distinguishing it from sibling tools that deal with trust or compliance but not intent declaration.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. The description does not specify prerequisites, use cases, or situations where this tool is inappropriate, leaving the agent to infer from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Deploy an ephemeral decoy MCP server to detect adversarial probing
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Honeypot display name | |
| template | Yes | Decoy template | |
| ttlMinutes | No | Auto-destroy after N minutes (default: 30) | |
| alertOnInteraction | No | Alert on every probe (default: true) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description calls it 'ephemeral' but does not detail lifecycle behavior, such as what happens on deployment (e.g., resource creation, IP exposure), how auto-destruction works, or any side effects on the system. An agent would not know what happens beyond deployment.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no fluff. Perfectly concise for conveying the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description is too minimal. It does not explain the deployment output (e.g., how to access the honeypot, status), how to clean up (though destroy_honeypot exists), or how the server interacts with other tools. The tool deploys a resource, but the description lacks operational context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents all 4 parameters adequately. The description adds no extra meaning beyond what the schema provides. For example, 'template' enum values are self-explanatory from the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Deploy', the resource 'an ephemeral decoy MCP server', and the purpose 'to detect adversarial probing'. It distinguishes itself from siblings like destroy_honeypot and list_honeypots.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus other security tools (e.g., scan_security, run_incident_playbook). No mention of prerequisites, such as required permissions or network setup. The description does not indicate when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tear down a specific honeypot and retrieve captured data
| Name | Required | Description | Default |
|---|---|---|---|
| honeypotId | Yes | Honeypot ID to destroy |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses that the tool is destructive and retrieves data, but lacks details on whether the action is reversible, whether data is lost after retrieval, required permissions, or error conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is clear and to the point. Every word contributes meaning without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, and the description only vaguely mentions 'retrieve captured data' without specifying the format or structure of the return value. Missing details on error cases (e.g., honeypot not found) and the overall response make it incomplete for a destructive action.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the schema already describes 'honeypotId' as 'Honeypot ID to destroy'. The description adds no additional meaning, format, or context for the parameter beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('tear down') and resource ('specific honeypot') and includes an additional function ('retrieve captured data'), clearly distinguishing it from sibling tools like 'list_honeypots' or 'deploy_honeypot'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, no when-not-to-use instructions, and no mention of prerequisites or side effects. As a destructive action, it should note that it permanently removes the honeypot.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Detect agent-to-agent collusion patterns (recon-then-exploit, coordinated exfil, token sharing)
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only says 'Detect' but does not disclose whether the tool is read-only, destructive, or any side effects. The return format, operational scope, or resource implications are completely absent. This is minimal disclosure for a tool with no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that efficiently conveys the purpose and examples. It is concise without unnecessary words. Could be slightly more structured (e.g., bullet points) but overall well-sized.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has no input parameters and no output schema. The description only lists collusion patterns but does not describe the output format, how to interpret results, or any constraints. For a tool that likely returns detection results, this is incomplete. With no annotations, more context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters and 100% coverage (no missing descriptions). With no parameters, the description does not need to add parameter semantics. Baseline for 0 params is 4, and the description meets that by not requiring additional param info.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Detect' and the resource 'agent-to-agent collusion patterns', listing examples like 'recon-then-exploit, coordinated exfil, token sharing'. It is specific but does not differentiate from siblings like 'scan_security' or 'run_incident_playbook', which may have overlapping functionality.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. There is no mention of prerequisites, context, or explicit 'when-not-to-use'. The description solely states functionality without usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Compare current MCP server behavior against a known-good baseline to detect anomalies
| Name | Required | Description | Default |
|---|---|---|---|
| baselineId | No | Specific baseline to compare against (uses latest if omitted) | |
| serverName | No | Server name to check for drift |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It indicates a comparison operation to detect anomalies, which is likely non-destructive, but it does not explicitly state whether the tool modifies server state, requires authentication, or has rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence with no superfluous words. It is front-loaded with the key action and purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is adequate for a simple tool with two optional parameters, but it omits information about the output format (e.g., list of anomalies, drift score). Given no output schema, the description could provide more context on return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with both parameters well-described. The description does not add extra information beyond the schema, so it meets the baseline score for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'compare', the resource 'MCP server behavior', and the outcome 'detect anomalies'. It distinguishes itself from siblings like 'capture_baseline' (which creates baselines) and 'drift_history' (which shows historical drifts).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool: when you have a baseline and want to check for drifts. However, it does not explicitly state when not to use it or mention alternatives among the many sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all detected drift events
| Name | Required | Description | Default |
|---|---|---|---|
| serverName | No | Filter to a specific server (optional) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose any behavioral traits such as read-only nature, pagination, ordering, or performance characteristics. The description carries the full burden but only states the basic action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with zero wasted words. It is appropriately front-loaded with the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with one optional filter and no output schema, the description is adequate but could mention what the drift events contain or any default ordering. It misses the opportunity to provide minimal context about the response.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with one parameter ('serverName') already described in the schema. The description adds no additional meaning beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List all detected drift events' with a specific verb (List) and resource (detected drift events). It distinguishes itself from sibling 'detect_drift' which creates drift events.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like 'detect_drift' or other list tools. No context on when filtering is appropriate or when to rely on this over other tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Thompson Sampling — run Bayesian bandit trust sampling for an agent (Beta posterior, exploration/exploitation)
| Name | Required | Description | Default |
|---|---|---|---|
| agentId | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description reveals it updates a posterior distribution but does not disclose whether it mutates state, requires permissions, or what the return value is. Missing critical behavioral details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single concise sentence that front-loads the main concept. However, it could be slightly restructured to include usage scope.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex tool involving Bayesian inference, the description lacks details on output, state changes, and how it differs from sibling tools like agent_trust_status. Incomplete for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter agentId is not mentioned in the description. With 0% schema description coverage, the description adds no meaning about the parameter beyond the name.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it runs Bayesian bandit trust sampling using Thompson Sampling for an agent, mentioning Beta posterior and exploration/exploitation. It distinguishes from static trust scores but could be more explicit about the output.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like compute_trust_score or get_agent_reputation. The description assumes familiarity with Thompson Sampling without setting context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Generate a complete security, cost, and health report for all MCP servers
| Name | Required | Description | Default |
|---|---|---|---|
| format | No | Output format (default: text) | |
| configPath | No | Path to MCP config file (optional) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It implies a read-only report generation (non-destructive), but does not explicitly state safety, performance, or auth requirements. This is adequate but incomplete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that directly states the tool's purpose. Every word contributes value, with no extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with two optional parameters and no output schema, the description is mostly complete. It lacks details on output structure or behavior, but given the low complexity, it is nearly sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, meaning the schema already explains the parameters (format and configPath). The description adds no extra semantic value beyond what the schema provides, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool generates a 'complete security, cost, and health report for all MCP servers', using a specific verb and resource. This differentiates it from sibling tools like check_health or audit_costs, which focus on individual aspects.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus its many siblings (e.g., check_health, audit_costs, scan_security). It does not mention alternatives, prerequisites, or exclusions, leaving the agent without context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Generate auditor-ready compliance evidence bundle for a framework
| Name | Required | Description | Default |
|---|---|---|---|
| framework | Yes | Compliance framework |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must fully disclose behavior. It does not mention side effects, output format, or any long-running nature. The agent cannot infer whether the bundle is generated as a file, returned inline, or if there are any rate limits or destructive actions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that conveys the core action and object. It is concise but could be slightly less terse. It earns its place with clear front-loading of key terms.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema and the presence of many sibling tools, the description is inadequate. It does not explain what the bundle contains, how to access the result, or any post-generation steps. For a generation tool, this is a significant gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%; the single parameter 'framework' has a clear description and enum values. The description adds no extra meaning beyond what the schema provides. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Generate auditor-ready compliance evidence bundle for a framework' clearly states the verb (generate) and resource (compliance evidence bundle). It specifies that the bundle is for a framework, and the enum parameter lists the frameworks. However, it does not differentiate from sibling tools like 'compliance_gap_analysis' or 'compliance_posture', which have distinct purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, recommended scenarios, or exclusion criteria. Given multiple compliance-related siblings, explicit usage context is needed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Generate a minimal-privilege YAML policy based on observed tool call patterns
| Name | Required | Description | Default |
|---|---|---|---|
| windowId | No | Specific observation window to use (uses latest if omitted) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose any side effects, permissions required, or whether the tool is read-only. It only states the generation action without clarifying if it modifies any state.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no wasted words. It is appropriately concise and front-loaded with the main action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the description covers the basic purpose and output format, it lacks details on prerequisites (e.g., need for prior observations), return structure, and behavior when no observations exist. Given the tool's moderate complexity, it is minimally adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the description adds no extra meaning beyond the schema's description of the optional 'windowId' parameter. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description specifies a clear verb ('Generate'), a resource ('YAML policy'), and the source ('observed tool call patterns'). It differentiates from sibling tools like 'natural_language_to_policy' which use natural language input.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for generating policies from observations but provides no explicit guidance on when to use it versus alternatives like 'natural_language_to_policy' or 'suggest_policy_improvements'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get agent reputation score — Trusted/Standard/Suspicious/Blocked tier with bypass rate and entropy
| Name | Required | Description | Default |
|---|---|---|---|
| agentId | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must disclose behavioral traits. It implies a read operation but does not state idempotency, latency, or side effects. Missing information on whether it requires authentication or has rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence with essential information, front-loaded. No redundant words. Could be slightly more structured but remains efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, no parameter descriptions, and no annotations, the description is insufficient. Terms like 'bypass rate' and 'entropy' are undefined, and the tool's complexity is not addressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Single parameter agentId has 0% schema description coverage. The description does not clarify the format, constraints, or usage of agentId. Parameter semantics are entirely opaque.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action 'Get', the resource 'agent reputation score', and lists output fields (tier, bypass rate, entropy). It distinguishes itself from sibling tools by specifying exact outputs, though it does not explicitly differentiate from similar tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., compute_trust_score, evaluate_agent_trust). The description lacks context for appropriate usage scenarios or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Analyze MCP server config and get A-F hardening grade with one-click recommendations
| Name | Required | Description | Default |
|---|---|---|---|
| serverName | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It does not disclose whether the tool modifies config, requires special permissions, or has side effects. 'One-click recommendations' hints at possible actions but is ambiguous.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence with no wasted words. Purpose is front-loaded and directly stated.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple analysis tool with one param and no output schema, the description covers basic purpose and output type. However, missing details on return structure, whether it takes action, or prerequisites for a security tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Single parameter 'serverName' with no schema description. The description implies it identifies a server, but no format, validation, or default value info. Basic meaning is conveyed, but lacks detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('analyze'), resource ('MCP server config'), and outcome ('A-F hardening grade with one-click recommendations'), distinguishing it from sibling tools like compliance_gap_analysis.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives, such as compliance_gap_analysis or scan_security. Context of tool purpose is clear but no exclusions or prerequisites mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get attack patterns observed by all active honeypots
| Name | Required | Description | Default |
|---|---|---|---|
| honeypotId | No | Filter to a specific honeypot (optional) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It states a read operation but does not disclose what happens if no honeypots are active, rate limits, authentication needs, or whether the report is aggregated or per-honeypot. The description adds minimal behavioral context beyond the tool name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no fluff. It is front-loaded and efficiently communicates the core function.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple report tool with one optional parameter and no output schema, the description is minimally adequate. However, it lacks details on return format (e.g., list of patterns, aggregated counts) and error handling, which would be helpful for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a clear parameter description. The tool description adds value by clarifying that the default scope is 'all active honeypots', which is not in the schema. This complements the filtering nature of the optional parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'attack patterns observed by all active honeypots'. It distinguishes from sibling tools like 'list_honeypots' and 'full_report' by specifying the scope of attack patterns from active honeypots.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, such as 'full_report' or 'prompt_injection_report'. No exclusion criteria or prerequisites mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List MCP servers in the local certification registry with level and expiry
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description tells that it lists items with specific attributes, which implies a read-only, non-destructive operation. However, it does not disclose any behavioral traits such as pagination behavior, authentication requirements, or side effects. Since no annotations are provided, the description carries full burden but lacks detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 13 words that directly states the tool's purpose. It is front-loaded with the verb 'List' and contains no extraneous information. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (list operation with one optional parameter and no output schema), the description covers the basic intent and returned data ('level and expiry'). However, it omits details about the 'limit' parameter's behavior and the exact format of the output, leaving gaps for the agent to infer.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one parameter 'limit' with type 'number' but no description. The tool description does not mention this parameter at all, failing to explain its purpose (e.g., maximum number of results). With 0% schema description coverage, the description must compensate but does not.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'MCP servers in the local certification registry', and specifies the returned information ('level and expiry'). It distinguishes from sibling tools like 'certify_server' and 'verify_certification' by indicating it is a listing operation. However, it does not explicitly differentiate from 'trust_registry_list', which may be a similar listing tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. The description merely implies usage for listing certified servers. There is no mention of when not to use it or what prerequisites (e.g., having a certification registry) are needed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all supported compliance frameworks
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It states 'List', indicating a read-only operation, but does not mention any side effects, authentication needs, rate limits, or other contextual behaviors beyond the obvious.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence with no wasted words. It is front-loaded and efficiently conveys the purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the description is adequate for a simple list tool, it lacks details about the return value format or what constitutes a compliance framework. The absence of an output schema and annotations places more burden on the description, which is not fully met.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and schema description coverage is 100%. With no parameters to describe, the description adds no additional meaning, but the baseline for 0 parameters is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('List') and resource ('all supported compliance frameworks'), clearly stating the tool's function. It effectively distinguishes from sibling tools like compliance_gap_analysis or compliance_posture, which perform different tasks.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use when needing a list of frameworks, but provides no explicit guidance on when to use versus alternatives or any prerequisites. Given 53 sibling tools, this is a gap.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all active and destroyed honeypots with summary
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden. It implies a read-only operation by listing, but does not explicitly confirm non-destructive behavior or disclose any side effects. Acceptable but not thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—one sentence that conveys the essential function. No unnecessary words, perfectly front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless list tool without output schema, the description is reasonably complete. It specifies the scope (active and destroyed) and includes 'with summary' hinting at return content. Could be more specific but adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters, so baseline 4 applies. The schema coverage is 100% and no parameter details are needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists all honeypots (active and destroyed) with a summary, using a specific verb and resource. It distinguishes from sibling tools like destroy_honeypot and deploy_honeypot.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as honeypot_report or other list tools. The description does not provide usage context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Convert a natural-language security goal into a draft YAML policy rule (requires approval before enforce)
| Name | Required | Description | Default |
|---|---|---|---|
| goal | Yes | Natural language policy goal | |
| availableTools | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that the output requires approval before enforcement, implying it is a non-final action, but does not state whether it is a read-only operation, if it modifies state, or any side effects. Additional context on safety or permissions would improve transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with the core purpose, no redundant words. Each part ('Convert', 'natural-language security goal', 'draft YAML policy rule', 'requires approval before enforce') adds distinct value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, and the description offers minimal detail on the return value (just 'draft YAML policy rule'). In the context of 50+ sibling tools, more guidance on when to use this vs. similar ones would enhance completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds no new meaning to the 'goal' parameter beyond the schema description (both mention 'natural language policy goal'). The 'availableTools' parameter is entirely undocumented in both schema and description. With 50% schema coverage, the description fails to compensate for the missing parameter explanation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specifically states the verb 'Convert' and the resource 'natural-language security goal' into 'draft YAML policy rule', clearly distinguishing from the reverse sibling tool 'policy_to_natural_language' and alternative 'generate_policy_from_observations'. Also notes the approval requirement, adding precision.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for converting natural language into a draft policy, but offers no explicit when-to-use, when-not-to-use, or comparison with alternatives like 'ab_test_policy' or 'generate_policy_from_observations'. The approval requirement hints at cautious use, but guidance is minimal.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Initiate an automated trust handshake with another AI agent behind Mastyf AI
| Name | Required | Description | Default |
|---|---|---|---|
| remoteAgentId | Yes | Remote agent identifier | |
| requestedTools | Yes | Tools to request access to | |
| maxSessionMinutes | No | Maximum session duration in minutes (default: 30) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must convey behavioral traits, but it only states the action without disclosing side effects, authorization needs, success/failure behavior, or persistence. This is inadequate for a trust negotiation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with key information, no unnecessary words. Perfectly concise for a straightforward initiation action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 3 parameters, no output schema, and no annotations. The description is too brief; it lacks details on return values, error conditions, or what the handshake entails, leaving agents under-informed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and each parameter has a concise schema description. The tool's description adds no further meaning beyond what is already in the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('initiate') and resource ('automated trust handshake with another AI agent behind Mastyf AI'), clearly distinguishing it from sibling tools like 'evaluate_agent_trust' and 'revoke_agent_trust'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, nor any prerequisites or exclusions. Siblings like 'evaluate_agent_trust' or 'compute_trust_score' could be confused without context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get current behavior observation status and summary
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description carries full burden. It indicates a read operation with no side effects, but lacks detail on what exactly is returned (e.g., the structure of 'status and summary'). Minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no wasted words, clearly stating the tool's action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description should provide more details about the return structure or data scope. It is vague ('status and summary') and incomplete for an agent to understand what it will receive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema is empty (0 parameters) and has 100% coverage, so no parameter description is needed. The description adds no extra meaning but is not required to; baseline of 4 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Get') and clearly identifies the resource ('current behavior observation status and summary'), distinguishing it from sibling tools like start_behavior_observation and stop_behavior_observation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus the many sibling tools, such as drift_history or agentic_status. The agent has no context to choose this over alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Explain MCP Mastyf AI policy YAML in plain English for compliance stakeholders
| Name | Required | Description | Default |
|---|---|---|---|
| yaml | No | Policy YAML text | |
| policyPath | No | Optional path to policy file |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description does not disclose any side effects or limitations, but for a straightforward translation tool that appears non-destructive, the information is adequate. No annotations are provided, so the description carries the full burden, but no misleading claims are made.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. It efficiently conveys the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has no output schema or annotations, but given its low complexity as a simple translation task, the description is nearly complete. However, it does not specify the exact output format beyond 'plain English', which is a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with both parameters having descriptions in the schema. The tool description does not add any new meaning beyond what the schema already provides, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Explain', the resource 'MCP Mastyf AI policy YAML', the output 'plain English', and the audience 'compliance stakeholders'. It effectively distinguishes from the sibling tool 'natural_language_to_policy' which performs the reverse operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
While not explicitly stating when or when not to use, the name and description imply that this tool is for converting technical policy YAML into human-readable explanations. The presence of the sibling 'natural_language_to_policy' provides context for when to use each, but no explicit alternatives or exclusions are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Generate threat forecast for all configured MCP servers with 30/90/365-day projections
| Name | Required | Description | Default |
|---|---|---|---|
| serverName | No | Filter to a specific server name. If omitted, predicts for all. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses the projection horizons and aggregate scope, but lacks details on performance impact, rate limits, whether the operation is read-only, or behavior when no servers exist.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single 13-word sentence that is front-loaded with the action and presents key information without any fluff or unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one optional parameter and no output schema, the description covers the primary function and time horizons. It is missing details on return format or error handling, but remains sufficiently complete for basic selection.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (one parameter fully described). The description adds no new semantics beyond what the schema already provides ('Filter to a specific server name... If omitted, predicts for all').
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Generate threat forecast') and the resource ('all configured MCP servers') with specific time horizons (30/90/365-day). It distinguishes itself from sibling tools like 'threat_forecast_for_server' which likely targets a single server.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for all servers, but does not explicitly state when to use this tool versus alternatives like 'threat_forecast_for_server'. No exclusions or prerequisites are mentioned, leaving the agent to infer context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get suggested preemptive policy changes based on threat forecasts
| Name | Required | Description | Default |
|---|---|---|---|
| serverName | No | Filter to a specific server |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It fails to disclose whether the tool is read-only, if it requires authentication, what side effects exist, or the format of the suggestions. The behavior is vaguely described as 'get suggested', which is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that front-loads the core purpose. Every word is functional, with no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one optional parameter, no output schema, no annotations), the description is minimally adequate. However, it lacks details about return format, whether it applies to the current server or specified server, and any prerequisite state, leaving some gaps for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single parameter 'serverName', which already has a description. The tool description adds no additional meaning beyond what the schema provides, so it meets the baseline for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('get suggested'), the object ('preemptive policy changes'), and the basis ('based on threat forecasts'). It distinguishes this tool from siblings like 'predict_threats' and 'suggest_policy_improvements' by combining both forecasting and recommendation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for proactive policy changes based on threat forecasts, but does not provide explicit when-to-use or when-not-to-use guidance, nor does it mention alternatives among the many sibling tools. Usage context is left to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get prompt injection detection statistics
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose whether the tool is read-only, historical, or real-time. Minimal transparency beyond the implied reporting function.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no fluff. Perfect conciseness for this simple tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no parameters and no output schema, the description is adequate but could mention the type of statistics (e.g., counts, trends) to differentiate from other reporting tools in the large sibling list.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Zero parameters, schema coverage 100% (empty). Baseline 4 per instructions. Description adds no param info, but none is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it 'Get prompt injection detection statistics'. The verb 'Get' and resource 'statistics' are specific. However, it doesn't specify the scope or nature of the statistics, which could be improved.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like scan_prompt_injection or full_report. With 0 parameters, it's trivial but still lacks context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Compute cyber insurance ALE (Annualized Loss Expectancy) for an MCP server
| Name | Required | Description | Default |
|---|---|---|---|
| toolCount | No | ||
| serverName | Yes | ||
| recordsAtRisk | No | ||
| networkExposure | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should disclose behavioral traits. It only states the computation but does not clarify if it is read-only, requires permissions, or side effects. 'Compute' implies non-destructive but is not explicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no fluff, but too brief for the tool's complexity. Adequate conciseness but lacks structured information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 4 parameters and no output schema, the description should provide more context. It does not explain return values, parameter roles, or usage scenarios, leaving gaps for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no explanation of parameters (serverName, toolCount, recordsAtRisk, networkExposure). The agent must infer meaning from parameter names alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific verb 'Compute' and the resource 'ALE (Annualized Loss Expectancy) for an MCP server'. It is distinct from sibling tools like compute_trust_score or evaluate_agent_trust.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives. Sibling tools include other risk-related computations, but no differentiation or prerequisites are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Query decentralized MCP server reputation (8-dimension consensus score)
| Name | Required | Description | Default |
|---|---|---|---|
| serverName | Yes | ||
| packageName | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states 'query', implying a read-only operation, but lacks details on side effects, error behavior, idempotency, or any constraints. The mention of 'decentralized' and '8-dimension' adds some context but is insufficient for behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, achieving conciseness, but at the cost of missing critical information. It is front-loaded but overly minimal for a tool with multiple parameters and many siblings.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's two parameters (one undocumented) and no output schema, the description fails to explain the 8 dimensions, usage of packageName, or return format. It is incomplete for effective agent selection.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description does not mention any parameters, while the input schema has two parameters (serverName required, packageName optional) with zero schema description coverage. The description adds no meaning beyond the schema, leaving the purpose of packageName unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool queries decentralized MCP server reputation and specifies it's an 8-dimension consensus score. The verb 'query' and resource are specific, but it does not differentiate from similar sibling tools like get_agent_reputation or compute_trust_score, making it slightly ambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. The description does not provide context for selection among many reputation-related sibling tools (e.g., get_agent_reputation, compute_trust_score).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get latest red team assessment results and recommendations
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It only says 'Get' (a read operation) and does not mention any side effects, authentication requirements, rate limits, or what constitutes 'latest'. The lack of detail is a concern for a tool with no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is front-loaded and contains no unnecessary words. It efficiently conveys the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (zero parameters, no output schema), the description is mostly complete. However, it could briefly mention the format or scope of the results (e.g., 'returns a summary of findings'). Still, it adequately covers the core functionality.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters and 100% description coverage, so the baseline is 3. The description does not add any parameter information, but none is needed since there are no parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Get' and the resource 'latest red team assessment results and recommendations'. It distinguishes the tool from siblings like 'schedule_red_team' (scheduling) and 'prompt_injection_report' (specific test), making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. For example, it does not specify that this tool should be used after running 'schedule_red_team' or that it provides a summary vs. detailed findings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Immediately terminate a trust relationship
| Name | Required | Description | Default |
|---|---|---|---|
| sessionId | Yes | Session ID to revoke |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description mentions 'immediately terminate,' implying irreversibility and speed, but lacks details on permissions required, side effects, or whether the action is destructive. No annotations are provided, so the description carries full burden and is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (4 words) but at the cost of completeness. It is front-loaded but lacks structure and contextual information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter tool with no output schema, the description provides minimal context. Given the large sibling set, more context would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers the sole parameter with a clear description. The tool description adds no further semantic value beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description states 'terminate a trust relationship,' which clearly indicates the tool's action and resource. However, it does not differentiate from sibling tools like 'evaluate_agent_trust' or 'negotiate_agent_trust,' which also operate on trust relationships.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. Given many sibling trust-related tools (e.g., evaluate_agent_trust, negotiate_agent_trust), explicit usage context is missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Revert to a previous known-good configuration snapshot
| Name | Required | Description | Default |
|---|---|---|---|
| baselineId | No | Baseline ID to rollback to | |
| serverName | Yes | Server name to rollback |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must disclose behavioral traits. Only states 'revert' implying a destructive action, but lacks details like immediate application, server impact, or rollback limits. Significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Very concise single sentence. No wasted words, but could benefit from slightly more detail without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 2 params, no output schema, and no annotations, the description is insufficient. It does not explain post-rollback behavior, error states, or return values, leaving significant gaps for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% – both parameters have descriptions. The tool description adds no new meaning beyond the schema, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action (revert) and resource (configuration snapshot). It distinguishes well from siblings like capture_baseline and harden_config by focusing on reverting to a known-good state.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use vs alternatives. Missing context about prerequisites (e.g., need a valid baselineId) or when not to use (e.g., if no snapshots exist).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Execute an incident response playbook (prompt_injection, credential_leak, shell_injection)
| Name | Required | Description | Default |
|---|---|---|---|
| trigger | Yes | ||
| playbook | Yes | ||
| severity | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations available, the description carries full responsibility for disclosing behavioral traits. It omits any mention of side effects, permissions, destruction potential, or rate limits. A user cannot infer whether execution is reversible or safe.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise but severely under-informative. It lacks structure and front-loads minimal information, failing to earn its place by omitting essential details for correct tool invocation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 3 parameters, no output schema, and no annotations, the description is grossly incomplete. It does not clarify return values, parameter relationships, or expected outcomes. The tool's context is barely conveyed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage for parameters, and the tool description adds no detail about 'trigger', 'playbook', or 'severity'. It does not explain allowed values, formats, or constraints, leaving the agent without guidance for parameter selection.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Execute an incident response playbook' with specific playbook types listed, which clarifies the basic function. However, it does not explain what execution entails (e.g., automation, alerts), and the scope could be inferred from the name. It weakly differentiates from siblings like 'prompt_injection_report' but lacks explicit distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus sibling tools such as scan_prompt_injection or generate_compliance_evidence. No context about prerequisites, alternatives, or exclusion cases is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run MCP protocol fuzzer — test defenses against malformed JSON-RPC, overflow, injection
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility for behavioral disclosure. It states 'test defenses' but does not explain what the tool actually does—e.g., whether it sends fuzzed payloads, what side effects may occur (e.g., service disruption), or if authentication is required. With zero annotations, critical behavioral traits are missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. However, it is extremely brief and could expand on key behavioral aspects without losing conciseness. It earns a 4 for efficiency but lacks completeness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that this is a security testing tool with no output schema and no annotations, the description is insufficiently complete. It does not describe what the output looks like (e.g., a report of vulnerabilities found, success/failure status) or how to interpret results. An agent would lack context to use the tool effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so schema_description_coverage is trivially 100%. The description adds no parameter meaning because there are no parameters. Per the scoring guidelines, baseline is 4 for 0 parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool runs a protocol fuzzer to test defenses against malformed JSON-RPC, overflow, and injection. The verb 'Run' and specific resource 'MCP protocol fuzzer' with explicit attack types provide strong purpose clarity. It distinguishes from siblings like 'scan_prompt_injection' or 'scan_security' by focusing on protocol-level fuzzing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance is provided. The description implies usage for testing defenses, but it does not differentiate from alternative tools like 'scan_security' or 'prompt_injection_report'. Without context on prerequisites or exclusion criteria, an agent lacks guidance on selecting this tool over others.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run a full autonomous red team assessment with attack generation and policy testing
| Name | Required | Description | Default |
|---|---|---|---|
| attackCount | No | Number of attacks to generate (default: 50) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must disclose all behavioral traits. It only states that it runs an assessment but omits critical details: whether it is synchronous or asynchronous, side effects, durability of results, permissions needed, rate limits, or any other operational context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, well-structured sentence with no filler words. All content is front-loaded with the key action. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a 'full autonomous red team assessment', the description is too sparse. It lacks context about what happens during the assessment, what the output is, whether state is modified, and any post-conditions. The simple input schema does not compensate for these missing behavioral details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema covers the single parameter 'attackCount' with a description (Number of attacks to generate, default 50). The tool description adds no further meaning beyond this, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Run' and the resource 'full autonomous red team assessment' with specific actions 'attack generation and policy testing'. However, it does not explicitly differentiate from siblings like 'schedule_red_team' or 'red_team_results', which could be inferred but not directly stated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, when not to use it, or any prerequisites. Sibling tools like 'schedule_red_team' exist but no comparison is made.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Export Software Bill of Materials for MCP server packages
| Name | Required | Description | Default |
|---|---|---|---|
| format | No | SBOM format (default: cyclonedx) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral aspects. It does not mention permissions, side effects, or whether the export is a retrieval or file generation. The minimal description leaves ambiguity about what 'export' entails.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single concise sentence with no unnecessary words. It is appropriately front-loaded with the action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (1 parameter, no output schema), the description still lacks clarity on the output format and behavior. It does not explain whether the result is a file download or inline content, which incomplete for an export tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with the 'format' parameter having a full enum and description. The tool description adds no extra meaning beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Export Software Bill of Materials for MCP server packages' clearly specifying the action (export) and resource (SBOM). It is specific enough to distinguish from siblings like 'supply_chain_status', though not explicitly differentiating.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, nor prerequisites or conditions. It lacks any usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Scan tool call arguments for prompt injection payloads targeting downstream AI agents
| Name | Required | Description | Default |
|---|---|---|---|
| toolName | Yes | Tool name being called | |
| arguments | Yes | Tool call arguments to scan | |
| serverName | No | Server name |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry full weight. It lacks important behavioral details such as whether the scan is read-only, what happens on detection (e.g., blocking, alerting), or any rate limits. The behavior is underspecified for a security tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no redundant words. Every word is necessary and front-loaded with the action. Excellent conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of nested objects (arguments) and no output schema, the description should explain what the scan returns (e.g., boolean, list of injections, severity). It does not, leaving the agent without enough context to use the tool effectively. Annotations missing compound the issue.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all three parameters. The description adds no extra meaning beyond the schema, which is adequate. However, it does not clarify how parameters like 'arguments' structure might affect scanning or optional 'serverName' usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool scans tool call arguments for prompt injection payloads. It uses a specific verb ('Scan') and resource ('tool call arguments'), and the purpose is well-defined. Although not explicitly distinguishing from siblings like scan_security or prompt_injection_report, the focus on arguments is unique enough.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when needing to check arguments for injection risks, but provides no explicit guidance on when to use this versus alternatives, prerequisites, or context. The sibling tools are varied but no exclusions or when-not scenarios are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Scan MCP tool responses for PII, credentials, sensitive paths, and data exfiltration
| Name | Required | Description | Default |
|---|---|---|---|
| toolName | No | Tool name that produced the response | |
| serverName | No | Server name | |
| responseText | Yes | The response text to scan for data leaks |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description carries the full burden. It only lists scan targets but does not disclose whether the tool is read-only, what it returns (e.g., boolean, report), or any side effects. Missing behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with verb 'Scan', no extra words. Efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a scanning tool without output schema or annotations, the description should explain what the result looks like (e.g., scan report, alert). It only lists scan categories but omits output format and any prerequisites.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description adds no parameter-specific details beyond what the schema provides (toolName, serverName, responseText). It does not clarify the role of toolName or serverName in scanning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly specifies the tool's function: scanning MCP tool responses for PII, credentials, sensitive paths, and data exfiltration. This distinguishes it from sibling tools like scan_prompt_injection which targets prompts, and scan_security which may have a broader scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. The purpose implies it's for data loss prevention on responses, but no mention of when not to use or when to prefer siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Scan MCP server configurations for security vulnerabilities (CVEs, auth, typo-squatting, secrets)
| Name | Required | Description | Default |
|---|---|---|---|
| configPath | No | Path to an MCP config file. If omitted, auto-discovers configs. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It lists vulnerability categories but does not disclose whether the scan is read-only, requires permissions, or modifies state. Missing safety profile and behavioral details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with verb first, resource, and list of vulnerability types. No extraneous words, front-loaded key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Lacks output schema and does not describe what the scan returns (e.g., severity, list of findings). For a tool that likely produces a report, missing return value information reduces completeness for agent decision-making.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 100% coverage with a single parameter 'configPath' described. The description adds value by explaining what the scan checks for (CVEs, auth, etc.), going beyond the parameter's schema description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description specifies verb 'Scan', resource 'MCP server configurations', and lists specific vulnerability types (CVEs, auth, typo-squatting, secrets). This clearly distinguishes it from sibling tools like 'scan_prompt_injection' and 'scan_response_dlp'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives. The description implies it is a general security scan, but does not state when to use other scan tools. Usage context is only implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Configure periodic autonomous red team assessments
| Name | Required | Description | Default |
|---|---|---|---|
| enabled | No | Enable or disable scheduled assessments | |
| intervalHours | No | Hours between assessments (default: 24) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It only states 'configure' but does not disclose whether it creates a new schedule, updates an existing one, or disables it. No mention of side effects, destructive potential, or required permissions. The behavior is too opaque for a configuration tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise (6 words) and front-loaded with purpose. While it could be expanded with one more sentence on usage, the brevity is not detrimental given the simple parameter set. Every word serves a purpose, but the lack of structure (no separation of behavioral notes) prevents a perfect score.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is too sparse for a configuration tool with no annotations and no output schema. It does not explain the effect of enabling/disabling, whether the schedule persists, or how to view/modify it. Given the tool's potential impact, this is incomplete and could lead to incorrect invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with both parameters (enabled, intervalHours) documented in the schema. The description adds no additional meaning beyond the schema, meeting the baseline. It does not elaborate on default values or input constraints, but the schema already covers these.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Configure periodic autonomous red team assessments' clearly specifies the action (configure) and the resource (periodic red team assessments). It distinguishes from sibling tools like 'red_team_results' (viewing results) or 'run_self_assessment' (one-time execution), making the tool's purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for setting up a recurring schedule, but it does not explicitly state when to use this tool versus alternatives (e.g., 'run_red_team' for immediate execution). There is no guidance on prerequisites or exclusion criteria, leaving room for misinterpretation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
REINFORCE — use policy gradient to select optimal fuzzer mutation strategy
| Name | Required | Description | Default |
|---|---|---|---|
| observeReward | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description does not disclose whether the tool is read-only or modifies state, what side effects it has, or any behavioral traits beyond the brief mention of using reinforcement learning.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, concise and to the point. However, it could be more structured by separating the REINFORCE aspect from the selection purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has one parameter, no output schema, and no annotations, the description fails to provide sufficient context about how to use it or what it returns. It does not explain the reward observation or the expected output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter 'observeReward' is not described in either the schema or the description. With 0% schema description coverage, the description should explain the parameter's meaning and impact, but it does not.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description identifies a specific action (select) and resource (fuzzer mutation strategy), and mentions the method (REINFORCE). It clearly states what the tool does, though the verb 'REINFORCE' could be confusing. It is sufficiently distinct from siblings like 'run_protocol_fuzzer'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. No context about prerequisites, such as needing to run a fuzzer first, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Start observing AI agent tool calls to learn usage patterns for policy generation
| Name | Required | Description | Default |
|---|---|---|---|
| windowId | No | Optional custom observation window ID |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden for behavioral disclosure. It only states that the tool starts observation, but does not disclose whether observation is persistent, how it affects performance, or if it is read-only. The word 'observe' weakly implies read-only, but this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that front-loads the key action and purpose. It is not overly verbose, though it could include more detail without being excessively long.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description should provide more context about return values or side effects. It does not explain what the tool returns (e.g., an observation ID) or how to stop the observation, leaving the agent uncertain about usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description does not add meaning beyond the schema. Schema coverage is 100%, so baseline 3 applies. The schema already documents the optional 'windowId' parameter, and the description does not elaborate on its behavior or format.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'start' and the resource 'observing AI agent tool calls', with a specific purpose 'to learn usage patterns for policy generation'. This distinguishes it from sibling tools like 'stop_behavior_observation' or 'observation_status', though it could be more explicit about the difference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. The description implies it is a precursor to policy generation, but does not mention when not to use it or provide alternative tools like 'generate_policy_from_observations'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Stop the current observation window and finalize collected data
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden. It lacks details on side effects (e.g., whether data becomes immutable, if the observation is abortable, or if permissions are needed). The description only states what it does, not its behavioral implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, direct sentence with no wasted words. It front-loads the action and outcome efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and no output schema, the description is minimal. It adequately conveys the core function but omits context like whether an observation must be active, or if finalization is irreversible. Sibling tools imply the context, but not explicitly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, and schema description coverage is 100%. The description adds no parameter info, but none is needed. Baseline of 4 is appropriate for zero-parameter tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'stop' and resource 'observation window' with action 'finalize collected data'. It distinguishes from sibling tools like 'start_behavior_observation' and 'observation_status' by specifying it ends the observation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool, such as prerequisites (e.g., an active observation) or when not to use it. The sibling tool 'start_behavior_observation' implies a sequence, but no explicit direction is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Compare observed behavior against current policy and suggest additions/removals
| Name | Required | Description | Default |
|---|---|---|---|
| existingPolicyYaml | No | Current policy YAML to diff against |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description says 'suggest additions/removals', implying it returns suggestions without modifying the policy, but does not disclose whether it is read-only, what permissions are needed, or what side effects occur. It provides some transparency but lacks detail on behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one sentence with no wasted words. It is front-loaded with the main action. Could be slightly improved by adding more context without being verbose, but it is appropriate for a relatively simple tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description lacks crucial information: no mention of how observed behavior is obtained (prerequisite of an observation session?), no output format (list of additions/removals?), and no explanation of required inputs beyond the existing policy. Given no output schema and no annotations, the description is insufficient for a complete understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with one parameter well-described in the schema. The description does not add significant meaning beyond 'compare against current policy'. It fails to explain where 'observed behavior' comes from or how it relates to the input, which is a gap despite good schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses specific verb 'Compare' and 'suggest' with resource 'policy', and outcome 'additions/removals'. It clearly distinguishes from siblings like generate_policy_from_observations (create from scratch) and tune_policy_rule (adjust single rule).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage context (when you have existing policy and observed behavior to diff against), but does not explicitly state when to use this tool versus alternatives like ab_test_policy or tune_policy_rule. No 'use this when' or 'do not use' guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Current trust graph state for all MCP server packages
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully convey behavioral traits. It only states the tool returns 'current' state, implying a read-only snapshot, but omits details like cost, side effects, data freshness, or format. Minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with no extraneous words. Efficiently front-loads the core purpose. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite a simple interface (no params, no output schema), the domain is complex. The description fails to clarify what 'trust graph state' includes (e.g., a list, summary, or status per package). Lacks essential detail for correct interpretation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0 parameters with 100% coverage, so baseline is 4. However, the description adds no further meaning about what the output represents. It simply restates the name's implication, losing one point for lack of enrichment.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the resource ('trust graph state' for 'all MCP server packages') with an implied retrieval verb. It distinguishes from sibling tools like agent_trust_status or compute_trust_score which focus on individual agents or scores.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. Given many sibling trust tools, the description should indicate this is for a global overview versus agent-specific queries. Missing context on frequency or precedence.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Detailed threat forecast for a specific server with risk factors and preemptive hardening recommendations
| Name | Required | Description | Default |
|---|---|---|---|
| serverName | Yes | Server name to generate forecast for |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It states the tool returns a forecast with risk factors and recommendations, but does not mention idempotency, side effects, permissions, rate limits, or data source. It is adequate but incomplete for full transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that front-loads the purpose and key features. Every word adds value, with no wasted text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a single parameter and no output schema, the description is fairly complete: it explains what the tool does and what outputs it provides. However, it could mention the output format or whether the forecast is live or cached.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter, serverName, has 100% schema coverage that describes it as the server name. The description does not add extra meaning beyond the schema, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it provides a detailed threat forecast for a specific server, including risk factors and hardening recommendations. It uses a specific verb ('generate forecast') and resource ('specific server'), distinguishing it from siblings like 'predict_threats' which may be more general.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. Although the description implies per-server usage, it does not mention when not to use or compare with siblings such as 'predict_threats' or 'preemptive_recommendations'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get mesh connectivity, contribution stats, and known threat feed
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavioral traits. 'Get' implies a read-only operation, but no information is given about rate limits, authentication needs, or potential side effects. The description is too brief to provide meaningful transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise phrase with no wasted words. However, it could be structured as a sentence for clarity, but overall it is appropriately sized for a parameterless tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of parameters and output schema, the description is bare minimum. It lists what data is available but does not explain the format or how the data relates to other tools. It is adequate but leaves room for improvement in providing full context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so schema description coverage is 100%. The description does not need to add parameter information, and the baseline score of 4 is appropriate since no additional param context is required.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses the verb 'Get' and lists three resources: mesh connectivity, contribution stats, and known threat feed. While it lacks specificity on what exactly these entail, it clearly indicates the tool retrieves status information, distinguishing it from action-oriented sibling tools like 'deploy_honeypot' or 'scan_security'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as 'check_health' or 'agentic_status'. There are no explicit when-not-to-use instructions or mentions of prerequisites, leaving the agent to infer usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all registered agents in the trust registry
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and description only states it lists agents; no disclosure of side effects, permissions, or safety profile beyond the verb 'list'.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no extraneous words, front-loaded with the action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, and description does not explain return format; but for a simple list tool with no params, it is minimally adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so schema already covers them; description adds no param info, but baseline for zero params is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the verb 'list' and the resource 'all registered agents in the trust registry', distinguishing it from sibling tools like list_certified_servers or list_honeypots.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives; usage is implied but not directed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Contextual Bandit (LinUCB) — select optimal policy action (enforce/relax/skip) based on context
| Name | Required | Description | Default |
|---|---|---|---|
| agentTier | Yes | ||
| serverType | Yes | ||
| ruleCategory | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavior. It mentions 'Contextual Bandit' but does not clarify whether the tool modifies state, requires prior data, or simply recommends an action. No mention of side effects or permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single 16-word sentence that is front-loaded with the algorithm name and purpose. Every word is informative with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 3 required parameters with no descriptions, no output schema, and many sibling tools, the description is insufficient. It omits return value, prerequisites (e.g., existence of policy rule), and how it relates to tools like 'generate_policy_from_observations'.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain parameters. It only says 'based on context' without mapping to serverType, agentTier, or ruleCategory. Parameter meaning is entirely left to inference from names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool uses Contextual Bandit (LinUCB) to select from three actions (enforce/relax/skip) based on context. It distinguishes itself from sibling tools like 'adapt_threshold' and 'suggest_policy_improvements' by specifying the algorithm and action space.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. The description implies policy rule tuning but does not state prerequisites, when it is appropriate, or when other tools like 'ab_test_policy' should be used instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Verify a server certification attestation (JWS) and level
| Name | Required | Description | Default |
|---|---|---|---|
| serverName | Yes | ||
| attestationJws | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description must disclose behavioral traits. It only states 'verify', implying a read operation but no details on side effects, authentication needs, or output. It does not compensate for missing annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, concise but too vague. It could be restructured to convey more information without exceeding length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema and annotations, the description is insufficient. It fails to describe return values, error cases, or prerequisites, leaving the agent with minimal understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain the two parameters (serverName, attestationJws). The agent gets no additional meaning beyond the schema structure.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Verify') and the object ('server certification attestation (JWS) and level'). It distinguishes from siblings like 'certify_server' and 'list_certified_servers' by focusing on verification. However, 'level' is ambiguous without further context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as 'certify_server' or 'verify_supply_chain'. The description lacks context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Full supply chain integrity verification with signed attestation for MCP server packages
| Name | Required | Description | Default |
|---|---|---|---|
| version | No | Package version (optional) | |
| packageName | Yes | MCP server package name to verify |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries full burden. It mentions 'signed attestation' implying a cryptographic output, but does not disclose side effects (e.g., writing records), permissions, rate limits, or what verification entails. The description lacks essential behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single 14-word sentence that is front-loaded with purpose. Every word earns its place; there is no waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no annotations or output schema, the description is insufficient for an agent to understand the full tool behavior. It does not explain the return format, what 'verification' means, or the nature of the signed attestation. The tool appears complex but the description is too vague.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (both parameters have descriptions). The description adds no additional meaning or constraints beyond the schema. Baseline 3 is appropriate as the schema already documents parameters adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Full supply chain integrity verification with signed attestation for MCP server packages'. It uses a specific verb (verify), resource (supply chain), and differentiates from sibling tools like 'supply_chain_status' or 'verify_certification'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidelines are provided. The description does not indicate when to use this tool versus alternatives (e.g., supply_chain_status, verify_certification), nor does it mention prerequisites or scenarios to avoid.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v1.0.0ab_test_policyadapt_thresholdagent_trust_statusagentic_statusaudit_costscapture_baselinecertify_servercheck_healthcheck_slacompliance_gap_analysiscompliance_posturecompute_trust_scorecontribute_threat_signaturedeclare_intentdeploy_honeypotdestroy_honeypotdetect_collusiondetect_driftdrift_historyevaluate_agent_trustfull_reportgenerate_compliance_evidencegenerate_policy_from_observationsget_agent_reputationharden_confighoneypot_reportlist_certified_serverslist_compliance_frameworkslist_honeypotsnatural_language_to_policynegotiate_agent_trustobservation_statuspolicy_to_natural_languagepredict_threatspreemptive_recommendationsprompt_injection_reportquantify_insurance_riskquery_server_reputationred_team_resultsrevoke_agent_trustrollback_server_configrun_incident_playbookrun_protocol_fuzzerrun_self_assessmentsbom_exportscan_prompt_injectionscan_response_dlpscan_securityschedule_red_teamselect_fuzz_strategystart_behavior_observationstop_behavior_observationsuggest_policy_improvementssupply_chain_statusthreat_forecast_for_serverthreat_intel_statustrust_registry_listtune_policy_ruleverify_certificationverify_supply_chainScored across 60 tools
Most tools have clearly distinct purposes with specific descriptions (e.g., compute_trust_score vs get_agent_reputation). However, the large number of tools introduces some potential confusion, such as between scan_prompt_injection and scan_response_dlp, or between multiple threat forecasting tools.
The majority of tools follow a consistent verb_noun pattern (e.g., scan_security, check_health, list_certified_servers). Minor inconsistencies exist, such as 'agentic_status' vs 'agent_trust_status' and 'run_protocol_fuzzer' vs 'select_fuzz_strategy', but overall the naming is predictable.
60 tools is excessively large for a single MCP server. While the domain (security, compliance, trust, monitoring) is broad, the number of tools suggests potential fragmentation. A more modular organization into sub-servers would improve coherence.
The tool set covers a wide range of security, compliance, and trust operations for MCP servers, including scanning, certification, threat detection, incident response, and policy management. Minor gaps exist, such as missing tools for basic CRUD operations on policies or users, but overall very comprehensive.
The WAF for agents. Pattern-based + heuristic firewall scans prompts, RAG documents, tool argume...
Pay-per-call cybersecurity for AI agents: vuln scans, threat intel, compliance, code security.
Security gateway for AI agents: policy, approval, and audited execution, no secrets shared.
Threat modeling, code/cloud/pipeline scanning, shadow-AI discovery, compliance checks and fixes.
