The PromptSpeak MCP Server provides a pre-execution governance layer for AI agents, intercepting and validating tool calls before they execute through a 9-stage pipeline.
Frame Validation: Validate agent actions (frames) at structural, semantic, chain, or full levels — individually (ps_validate) or in batch (ps_validate_batch) — before execution
Governed Execution: Run tool calls through the full pipeline (ps_execute, ps_execute_batch) with circuit breaking, drift prediction, hold checks, and security scanning; supports dry-run preview
Human-in-the-Loop Approvals: Automatically hold risky operations for manual review; list, approve (with optional argument modifications), or reject pending holds via ps_hold_list, ps_hold_approve, ps_hold_reject
Agent Lifecycle & Drift Management: Halt, resume, reset, or recalibrate agents; monitor behavioral drift scores and history; automatically halt agents exceeding drift thresholds
Delegation Management: Create parent→child agent delegations with constrained permissions and inheritance rules; revoke or list active delegations
Security Scanning & Enforcement: Scan code for SQL injection, hardcoded secrets, insecure defaults, and more — with tiered enforcement (block on critical, hold on high, warn on medium, log on low)
Runtime Configuration: Set and activate policy overlays, tune confidence thresholds, and toggle pipeline feature flags at runtime
Symbol/Entity Registry: Create, read, update, delete, and bulk-import tracked entities (companies, people, events, tasks, etc.) with versioning, human verification workflows, and alternative explanation tracking
Audit Logging: Retrieve full audit trails of all tool calls attempted by agents, including blocked and allowed actions, with filtering
System Monitoring: View system-wide agent state, hold queue statistics, security pattern configuration, and performance metrics (latency, ops/sec)
Pre-execution governance for AI agents. Blocks dangerous tool calls before they execute.
AI agents call tools (file writes, API requests, shell commands) with no validation layer between intent and execution. A prompt injection, hallucinated argument, or drifting goal can trigger irreversible actions. PromptSpeak intercepts every MCP tool call, validates it against deterministic rules, and blocks or holds risky operations for human approval — in 0.1ms, before anything executes.
You run AI agents that call tools (MCP servers, function calling, tool use) and need a governance layer between the agent and the tools.
You need human-in-the-loop approval for high-risk operations (production deployments, financial transactions, legal filings).
You want to detect behavioral drift — an agent gradually shifting away from its assigned task.
You need an audit trail of every tool call an agent attempted, whether it was allowed or blocked.
You operate in a regulated domain (legal, financial, healthcare) where agent actions must be deterministically constrained.
Related MCP server: gov-mcp
Add to ~/.claude/settings.json (or project-level .claude/settings.json):
{
"mcpServers": {
"promptspeak": {
"command": "npx",
"args": ["promptspeak-mcp-server"]
}
}
}Restart Claude Code. All 56 governance tools are immediately available.
Add to claude_desktop_config.json:
{
"mcpServers": {
"promptspeak": {
"command": "npx",
"args": ["promptspeak-mcp-server"]
}
}
}npm install promptspeak-mcp-servergit clone https://github.com/chrbailey/promptspeak-mcp-server.git
cd promptspeak-mcp-server
npm install && npm run build
npm startValidate that a PromptSpeak frame is structurally and semantically correct before using it. Frames encode governance constraints as symbol sequences — mode first, then domain, action, and entity.
// Tool call: ps_validate
{
"name": "ps_validate",
"arguments": {
"frame": "⊕◊▶α",
"validationLevel": "full"
}
}// Response
{
"valid": true,
"frame": "⊕◊▶α",
"parsedFrame": {
"mode": { "symbol": "⊕", "meaning": "strict" },
"domain": { "symbol": "◊", "meaning": "financial" },
"action": { "symbol": "▶", "meaning": "execute" },
"entity": { "symbol": "α", "meaning": "primary" }
},
"parseConfidence": 1.0,
"report": {
"valid": true,
"errors": [],
"warnings": [
{ "code": "ACTION_MISSING_DOMAIN", "message": "Consider adding domain context", "severity": "warning" }
]
},
"summary": { "errors": 0, "warnings": 1, "passed": 16 }
}Invalid frames return actionable suggestions:
// Tool call: validate a frame with conflicting modes
{
"name": "ps_validate",
"arguments": { "frame": "⊕⊖▶", "validationLevel": "semantic" }
}
// Response: blocked — strict + flexible modes conflict
{
"valid": false,
"summary": { "errors": 1, "warnings": 0, "passed": 10 },
"suggestions": ["Remove either ⊕ (strict) or ⊖ (flexible) - cannot have both"]
}When an agent attempts a risky operation (high drift score, low confidence, security finding), the action is held for human review instead of executing. This is the full hold lifecycle: list, inspect, approve or reject.
Step 1: List pending holds
// Tool call: ps_hold_list
{
"name": "ps_hold_list",
"arguments": {}
}// Response: one hold awaiting human review
{
"holds": [
{
"holdId": "hold_7k2m9x",
"agentId": "devops-agent",
"frame": "⊕◈▶α",
"tool": "deploy_to_production",
"severity": "high",
"reason": "drift_prediction",
"state": "pending",
"evidence": { "driftScore": 0.72, "predictedDrift": 0.85 }
}
],
"count": 1,
"expiredCount": 0
}Step 2: Approve with modifications (or reject)
// Tool call: ps_hold_approve — approve but downgrade to staging
{
"name": "ps_hold_approve",
"arguments": {
"holdId": "hold_7k2m9x",
"reason": "Reviewed — safe for staging, not production",
"modifiedArgs": { "environment": "staging" }
}
}// Response
{
"success": true,
"decision": {
"holdId": "hold_7k2m9x",
"state": "approved",
"decidedBy": "human",
"reason": "Reviewed — safe for staging, not production"
},
"executionResult": { "success": true }
}To reject instead:
{
"name": "ps_hold_reject",
"arguments": {
"holdId": "hold_7k2m9x",
"reason": "Drift too high — recalibrate agent first",
"haltAgent": true
}
}
// Agent is halted (circuit breaker tripped) and the operation is denied.Scan code content for security issues before an agent writes it to disk. Critical findings block execution; high-severity findings are held for human review.
// Tool call: ps_security_scan
{
"name": "ps_security_scan",
"arguments": {
"content": "const query = `SELECT * FROM users WHERE id = ${userId}`;\nconst API_KEY = 'sk-1234567890abcdef1234567890abcdef';"
}
}// Response: two findings — one critical (blocked), one critical (blocked)
{
"findings": [
{
"patternId": "sql-injection",
"severity": "critical",
"match": "SELECT * FROM users WHERE id = ${userId}",
"line": 1,
"context": "const query = `SELECT * FROM users WHERE id = ${userId}`;",
"suggestion": "Use parameterized queries or prepared statements instead of template literals"
},
{
"patternId": "hardcoded-secret",
"severity": "critical",
"match": "API_KEY = 'sk-1234567890abcdef1234567890abcdef'",
"line": 2,
"context": "const API_KEY = 'sk-1234567890abcdef1234567890abcdef';",
"suggestion": "Move secrets to environment variables or a secrets manager"
}
],
"scannedAt": "2026-03-19T12:00:00.000Z",
"contentLength": 98,
"patternsChecked": 10,
"enforcement": {
"blocked": [
{ "patternId": "sql-injection", "severity": "critical" },
{ "patternId": "hardcoded-secret", "severity": "critical" }
],
"held": [],
"warned": [],
"logged": []
}
}Use ps_security_gate instead of ps_security_scan to enforce the policy — it blocks on critical, holds high-severity for human review, and warns on medium:
// Tool call: ps_security_gate — scan AND enforce
{
"name": "ps_security_gate",
"arguments": {
"content": "app.use(cors());\napp.listen(0.0.0.0, 3000);",
"action": "write_file"
}
}
// Response: held for review (insecure defaults = high severity)
{
"decision": "held",
"reason": "Security: 1 high-severity finding(s) held for review — insecure-defaults",
"scan": { "findings": [{ "patternId": "insecure-defaults", "severity": "high", "line": 1 }] }
}Every tool call passes through this pipeline. If any stage fails, execution is blocked.
Agent calls tool
│
├─ 1. Circuit Breaker ──── Halted agents blocked instantly (no further checks)
├─ 2. Frame Validation ─── Structural, semantic, and chain rule checks
├─ 3. Drift Prediction ─── Pre-flight behavioral anomaly detection
├─ 4. Hold Check ────────── Risky operations held for human approval
├─ 5. Interceptor ───────── Final permission gate (confidence thresholds)
├─ 6. Security Scan ─────── Scans write actions for vulnerabilities (see below)
├─ 7. Tool Execution ────── Only reached if all 6 pre-checks pass
├─ 8. Post-Audit ────────── Confirms behavior matched prediction
└─ 9. Immediate Action ──── Halts agent if critical drift detected post-executionStages 1-6 are pre-execution — the tool never runs if any check fails. Stages 8-9 are post-execution — they detect drift and can halt the agent for future calls.
When an agent writes code (write_file, edit_file, create_file, patch_file), the content is scanned against 10 detection patterns before execution. Severity determines enforcement:
Severity | Enforcement | What it catches |
CRITICAL | Blocked — execution denied | SQL injection via template literals, hardcoded API keys/passwords/tokens |
HIGH | Held — queued for human review | Security-related TODOs, logging sensitive data, insecure defaults ( |
MEDIUM | Warned — logged, execution continues | Empty catch blocks, hedging comments ("probably works"), disabled tests |
LOW | Logged — no enforcement |
|
All claims below are backed by passing tests (104 tests across 5 test files):
Pattern detection works. Each of the 10 patterns is tested for true positives AND false positives. Example: api_key = "sk-1234567890abcdef" is caught; API_KEY = process.env.API_KEY is not. SQL injection catches \SELECT * FROM users WHERE id = ${id}` but not parameterized queries (db.query("SELECT * FROM users WHERE id = ?", [id])`).
Severity enforcement works. Critical findings block execution. High findings hold for review. Medium findings warn but allow. Tested end-to-end through the interceptor pipeline.
Only write actions are scanned. read_file and other non-write actions pass through without scanning, even if their arguments contain vulnerable code. Tested.
Runtime configuration works. Patterns can be enabled/disabled and severity can be changed at runtime via ps_security_config. A disabled pattern stops firing immediately. Changing a pattern from medium to critical makes it block instead of warn. Tested end-to-end.
Performance is fine. 100-line file scans complete in under 10ms. Tested.
Multiple findings in one file work. A file with 6 different vulnerability types correctly classifies each into the right severity bucket. Tested.
No hold queue integration for HIGH findings. FIXED. HIGH-severity security findings now create real holds in HoldManager via security_finding HoldReason. They appear in ps_hold_list and can be approved/rejected through the normal hold flow.
No auto-scan on ps_execute. The security scan only triggers in the interceptor's intercept() method for direct tool calls. If an agent uses ps_execute (the governed execution path), the scan runs only if the inner tool is a write action AND the content is passed as a top-level arg. Nested argument structures may bypass scanning. Why: ps_execute wraps tool calls in its own argument schema; the scanner checks proposedArgs.content, not deeply nested fields.
No file-path-based scanning. The scanner only examines content passed as arguments. It cannot scan files already on disk — it doesn't read from the filesystem. Why: The scanner is a pure function that takes a string. Adding filesystem access would change the security model.
Patterns are regex-based, not AST-aware. The patterns use regular expressions, which means they can't understand code structure. A hardcoded secret inside a test fixture or a SQL injection in a comment will still trigger. False positive rates range from 25-70% depending on the pattern (documented per-pattern). Why: AST parsing would add dependencies and complexity. Regex is fast and good enough for a governance layer that holds for human review rather than silently blocking.
Partial persistence. Holds and circuit breaker state now persist to SQLite (data/governance.db) and survive server restarts. However, pattern configuration changes (enable/disable, severity changes via ps_security_config) are still in-memory only and reset on restart. Why: Pattern config is lightweight and rarely changed; full config persistence would need a separate config store.
Tool | When to call it | What it does |
| Before executing any agent action | Validate a frame against all rules without executing |
| When checking multiple actions at once | Batch validation for efficiency |
| When an agent wants to perform a tool call | Full pipeline: validate → hold check → execute → audit |
| When previewing what would happen | Run full pipeline without executing the tool |
Tool | When to call it | What it does |
| When reviewing pending agent actions | List all operations awaiting human approval |
| When a held operation should proceed | Approve with optional modified arguments |
| When a held operation should be denied | Reject with reason |
| When tuning which operations require approval | Configure hold triggers and thresholds |
| When monitoring hold queue health | Hold queue statistics |
Tool | When to call it | What it does |
| When checking what an agent is doing | Get agent's active frame and last action |
| When monitoring overall system health | System-wide statistics |
| When an agent must be stopped immediately | Trip circuit breaker — blocks all future calls |
| When a halted agent should be allowed to continue | Reset circuit breaker |
| When clearing agent state | Full state reset |
| When investigating behavioral changes | Drift detection alert history |
Tool | When to call it | What it does |
| When an agent spawns a sub-agent | Create parent→child delegation with constrained permissions |
| When revoking a sub-agent's authority | Remove delegation |
| When auditing delegation chains | List active delegations |
Tool | When to call it | What it does |
| When changing governance rules at runtime | Set configuration key-value pairs |
| When reading current configuration | Get current config |
| When switching policy profiles | Activate a named configuration |
| When backing up configuration | Export full config as JSON |
| When restoring configuration | Import config from JSON |
| When tuning validation strictness | Set confidence thresholds |
| When checking current thresholds | Get current thresholds |
| When reconfiguring multiple thresholds | Batch threshold update |
| When toggling pipeline stages | Enable/disable specific checks |
| When checking which stages are active | Get feature flags |
Tool | When to call it | What it does |
| When registering a new entity (company, person, system) | Create symbol with type, metadata, and tags |
| When looking up an entity | Retrieve by ID |
| When entity data changes | Update metadata or tags |
| When browsing entities by type | List with optional type filter |
| When removing an entity | Delete by ID |
| When bulk-loading entities | Batch import |
| When monitoring registry health | Registry statistics |
| When displaying an entity | Format symbol for display |
| When confirming entity data is current | Mark symbol as verified |
| When auditing stale data | List symbols needing verification |
| When an entity has aliases | Add alternative identifier |
Tool | When to call it | What it does |
| When checking code for vulnerabilities | Scan content, return findings by severity |
| When enforcing security policy on writes | Scan + enforce: block/hold/warn/allow |
| When tuning detection patterns | List, enable, disable, change severity of patterns |
Tool | When to call it | What it does |
| When reviewing what happened | Full audit trail with filters |
src/
├── gatekeeper/ # 8-stage validation pipeline (core enforcement)
│ ├── index.ts # Pipeline orchestrator + agent eviction policy
│ ├── validator.ts # Frame structural/semantic/chain validation
│ ├── interceptor.ts# Permission gate with confidence thresholds
│ ├── hold-manager.ts# Human-in-the-loop hold queue
│ ├── resolver.ts # Frame resolution with operator overrides
│ └── coverage.ts # Coverage confidence calculator
├── drift/ # Behavioral drift detection
│ ├── circuit-breaker.ts # Per-agent halt/resume
│ ├── baseline.ts # Behavioral baseline comparison
│ ├── tripwire.ts # Anomaly tripwires
│ └── monitor.ts # Continuous monitoring
├── security/ # Security vulnerability scanning
│ ├── patterns.ts # 10 detection patterns (regex-based)
│ └── scanner.ts # Scanner engine + severity classification
├── persistence/ # SQLite governance persistence
│ └── database.ts # Holds, decisions, circuit breakers (WAL mode)
├── symbols/ # SQLite-backed entity registry (11 CRUD tools)
├── policies/ # Policy file loader + overlay system
├── operator/ # Operator configuration
├── tools/ # MCP tool implementations
│ ├── registry.ts # 29 core tools
│ ├── ps_hold.ts # 5 hold tools
│ └── ps_security.ts# 3 security tools
├── handlers/ # Tool dispatch + metadata registry
├── core/ # Logging, errors, result patterns
└── server.ts # MCP server entry point (stdio transport)PromptSpeak adds governance to every tool call with sub-millisecond overhead. Benchmarked on Apple M2 Pro, Node.js 22, Vitest 4.0:
Percentile | Latency |
Average | 0.164ms |
P95 | 0.368ms |
P99 | 1.183ms |
Full execution path P95 | 0.074ms |
Operation | Rate |
Circuit breaker checks (1000 concurrent) | 6,173 ops/sec |
Hold creation | 55,556 holds/sec |
Hold approval/rejection | 200,000+/sec |
Mixed operations (halt + hold + allow) | 6,818 ops/sec |
Test | Result |
1000 concurrent blocked executions | 162ms total, all blocked correctly |
100 rapid halt/resume cycles | 100% correct state transitions |
500 agents with mixed states | 250 blocked, 250 allowed — zero misclassification |
Memory under sustained load (1000 ops) | Negative delta (-11.76 MB, GC reclaimed) |
Metric | Value |
Test count | 829 tests across 33 files |
Test duration | 1.11s total |
Categories | Unit, integration, stress, security, grammar |
Node.js >= 20.0.0
TypeScript 5.3+ (build from source)
No external services required — SQLite for symbols and governance persistence
deeptrend — Structured AI trend feed for autonomous agents. Curated from 14+ sources, synthesized via LLM Counsel, published every 6h as JSON Feed, RSS, and llms.txt. Designed as a data source for agent monitoring pipelines.
https://promptspeak.admin-as-a-service.com/privacy
PromptSpeak does not collect personal data, has no telemetry, and stores all governance data locally in SQLite. See the full policy at the link above.
https://promptspeak.admin-as-a-service.com/dpa
Standard data processing terms for platform integrations (e.g., Anthropic Connectors). PromptSpeak acts as a data processor; no sub-processors, no external data transmission.
MIT
Get audit log entries.
| Name | Required | Description | Default |
|---|---|---|---|
| since | No | ||
| action | No | ||
| limit | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Get audit log entries' implies a read-only operation, but it does not specify whether this requires authentication, has rate limits, returns paginated results, or what format the entries are in. For a tool with three parameters and no output schema, this lack of detail is a significant gap, though not contradictory to annotations (since none exist).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—a single sentence with no wasted words. It is front-loaded and directly states the tool's function without unnecessary elaboration. While this conciseness contributes to clarity, it also limits detail, but in terms of structure and brevity, it is optimal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (3 parameters, no annotations, no output schema), the description is incomplete. It does not cover parameter meanings, return values, behavioral traits, or usage context. For a tool that likely involves querying logs with filters, this minimal description leaves critical gaps, making it inadequate for effective agent use without additional inference or trial-and-error.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has three parameters (since, action, limit) with 0% description coverage, meaning their purposes are undocumented. The description does not add any meaning beyond the schema—it does not explain what 'since' refers to (e.g., timestamp), what 'action' might filter, or how 'limit' affects results. With low schema coverage, the description fails to compensate, leaving parameters largely ambiguous.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get audit log entries' clearly states the verb ('Get') and resource ('audit log entries'), which is adequate for understanding the basic function. However, it lacks specificity about what 'audit log entries' entail (e.g., system events, user actions) and does not distinguish this tool from potential siblings like 'ps_security_scan' or 'ps_state_drift_history', which might also involve logging or monitoring. This makes the purpose somewhat vague but not misleading.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With sibling tools like 'ps_security_scan' or 'ps_state_drift_history' that might relate to security or state tracking, there is no indication of context, prerequisites, or exclusions. This leaves the agent without clear direction on tool selection, relying solely on the name and basic purpose.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Set multiple confidence thresholds at once.
| Name | Required | Description | Default |
|---|---|---|---|
| thresholds | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden for behavioral disclosure. It only states the action ('set') without explaining what 'set' entails—whether it's a mutation, requires permissions, has side effects, or returns any output. This leaves critical behavioral traits unspecified.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It's front-loaded and appropriately sized for the tool's apparent complexity, earning full marks for conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, no output schema, and low schema coverage (0%), the description is incomplete. It doesn't address what the tool returns, error conditions, or detailed parameter usage, making it inadequate for a mutation tool with nested parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 1 parameter with 0% description coverage, and the description adds minimal semantics: 'thresholds' is implied to be an object for multiple values. However, it doesn't explain the structure, keys, values, or constraints of 'thresholds', failing to compensate for the low schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Set multiple confidence thresholds at once' clearly states the action (set) and resource (confidence thresholds), but it's vague about what 'confidence thresholds' are and doesn't distinguish from sibling tools like 'ps_confidence_set' (likely single threshold) or 'ps_confidence_get' (read operation). It provides basic purpose but lacks specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It doesn't mention sibling tools like 'ps_confidence_set' for single thresholds or 'ps_confidence_get' for reading, nor does it specify prerequisites or contexts for bulk operations. The description offers no usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get all confidence thresholds.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states 'Get all confidence thresholds,' implying a read operation, but doesn't specify if it's safe, requires authentication, has rate limits, or what the output format might be. This is inadequate for a tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence: 'Get all confidence thresholds.' It's front-loaded with the core action and resource, with no wasted words, making it highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain what 'confidence thresholds' are, how the data is returned, or any behavioral traits, leaving significant gaps for the agent to understand the tool's full context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters, and the schema description coverage is 100%, so there's no need for parameter details in the description. The description doesn't add parameter semantics, but with no parameters, a baseline score of 4 is appropriate as it doesn't need to compensate for gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get all confidence thresholds' clearly states the verb ('Get') and resource ('confidence thresholds'), making the purpose understandable. However, it doesn't differentiate from sibling tools like 'ps_confidence_set' or 'ps_confidence_bulk_set', which likely modify thresholds rather than retrieve them, so it misses explicit sibling distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any context, prerequisites, or exclusions, such as whether it's for read-only access or how it compares to other 'ps_confidence_' tools. This leaves the agent without usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Set a confidence threshold. This is the hidden knob for operators.
| Name | Required | Description | Default |
|---|---|---|---|
| threshold | Yes | ||
| value | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions 'hidden knob for operators' which hints at operational control, but doesn't disclose behavioral traits like whether this is a persistent change, requires specific permissions, has side effects, or impacts system performance. The description is minimal and lacks critical details for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with two sentences, front-loaded with the main action. However, the second sentence 'This is the hidden knob for operators' adds some context but could be more informative. It avoids unnecessary verbosity, but under-specification limits its effectiveness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, no output schema, and 0% schema description coverage for a mutation tool with 2 parameters, the description is incomplete. It doesn't explain what the threshold affects, the implications of setting it, or what the tool returns. For a tool that likely changes system behavior, more context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It doesn't explain the 'threshold' parameter (which has an enum of values like 'parseConfidence') or the 'value' parameter (a number between 0 and 1). The description adds no meaning beyond what the schema provides, but with only 2 parameters, the baseline is adjusted to 3 due to low complexity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the tool 'Set a confidence threshold' which provides a clear verb ('Set') and resource ('confidence threshold'), but it's vague about what this threshold controls or its domain. The phrase 'hidden knob for operators' adds some context but doesn't specify the system or purpose. It doesn't distinguish from siblings like ps_confidence_get or ps_confidence_bulk_set beyond the basic action.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like ps_confidence_bulk_set or ps_confidence_get. The description implies it's for operators, but doesn't specify contexts, prerequisites, or exclusions. Usage is implied by the name and action, but lacks clear differentiation from sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Activate a registered policy overlay.
| Name | Required | Description | Default |
|---|---|---|---|
| overlayId | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions 'activate' but doesn't disclose behavioral traits such as whether this is a read-only or destructive operation, permission requirements, side effects, or what 'registered' implies. This leaves significant gaps for an agent to understand the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It's appropriately sized and front-loaded, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity implied by 'activate' (likely a mutation), no annotations, 0% schema coverage, and no output schema, the description is incomplete. It doesn't cover what activation does, expected outcomes, or error conditions, making it insufficient for safe and effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It doesn't add any meaning beyond the schema—no explanation of what 'overlayId' represents, its format, or how to obtain it. With one undocumented parameter, this is inadequate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Activate a registered policy overlay' clearly states the verb (activate) and resource (policy overlay), but it's somewhat vague about what 'activate' entails operationally. It doesn't distinguish this tool from siblings like ps_config_set or ps_config_get, which might involve policy configuration in different ways.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. With siblings like ps_config_set, ps_config_get, and ps_config_import, the description doesn't clarify if activation is for enabling a policy, applying changes, or another purpose, leaving usage context unclear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Export current configuration for backup.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden but only states the action without behavioral details. It does not disclose if this is read-only, requires permissions, affects system state, has rate limits, or outputs format (e.g., file or data). This leaves critical gaps for a tool that exports configuration.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words, clearly front-loading the purpose. It is appropriately sized for a simple tool with no parameters, making every word count.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, no output schema, and a simple tool (0 params), the description is incomplete. It lacks details on what 'export' entails (e.g., format, location, side effects) and behavioral context, which are necessary for safe and effective use, especially among siblings with config-related tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately does not discuss parameters, earning a baseline 4 for not adding unnecessary info, though it could mention output semantics (not required per rules).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Export') and resource ('current configuration'), specifying the purpose for backup. It distinguishes from siblings like ps_config_get (retrieve) and ps_config_import (import), but does not explicitly contrast them, making it a 4 rather than a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides minimal guidance by mentioning 'for backup,' implying use cases like saving settings. However, it lacks explicit when-to-use rules, alternatives (e.g., vs. ps_config_get for viewing), or prerequisites, leaving the agent with insufficient context for optimal selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get current configuration including active overlay and thresholds.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states it's a read operation ('Get'), but doesn't disclose behavioral traits such as permissions required, rate limits, whether it returns real-time or cached data, or error conditions. For a tool with zero annotation coverage, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the purpose ('Get current configuration') and adds specific details ('including active overlay and thresholds'). There is no wasted text, making it appropriately sized and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters, no output schema, and no annotations, the description is minimal but adequate for a simple read operation. It specifies what is retrieved, but lacks details on return format, error handling, or dependencies. For a tool with low complexity, it meets the minimum viable threshold but has clear gaps in completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters, and schema description coverage is 100%, so no parameter information is needed. The description adds value by specifying what configuration details are retrieved (active overlay and thresholds), which goes beyond the empty schema. Baseline for 0 params is 4, as it provides useful context without redundancy.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'current configuration', specifying what it retrieves (active overlay and thresholds). It distinguishes from siblings like ps_config_set (write) but doesn't explicitly differentiate from ps_config_export or ps_config_import, which also involve configuration data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like ps_config_export (which might retrieve configuration in a different format) or ps_state_get (which might retrieve state rather than configuration). The description implies usage for reading configuration but lacks explicit context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Import configuration from backup.
| Name | Required | Description | Default |
|---|---|---|---|
| data | Yes | ||
| expectedChecksum | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states this is an import operation, implying it's a write/mutation tool that modifies system configuration. However, it doesn't disclose critical behaviors: whether this overwrites existing config, requires specific permissions, has side effects, provides confirmation, or handles errors. For a mutation tool with zero annotation coverage, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—a single four-word sentence that gets straight to the point without unnecessary words. However, this conciseness comes at the cost of completeness; it's arguably too brief for a tool that modifies system configuration with two parameters.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given this is a configuration mutation tool with 2 parameters (0% schema coverage), no annotations, and no output schema, the description is inadequate. It doesn't explain what 'configuration' means in this context, what a 'backup' entails, the impact of the import, or what happens after execution. For a tool that could significantly alter system state, more context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate for undocumented parameters. It mentions 'configuration from backup' which hints at the 'data' parameter being backup content, but doesn't explain the format (e.g., JSON, binary), source, or constraints. It completely ignores the 'expectedChecksum' parameter—no mention of checksum validation, purpose, or format. The description adds minimal value beyond the parameter names themselves.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Import configuration from backup' clearly states the verb ('Import') and resource ('configuration from backup'), making the basic purpose understandable. However, it doesn't specify what type of configuration or system this applies to, nor does it distinguish from sibling tools like 'ps_config_export' or 'ps_config_set' beyond the obvious import/export relationship.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a backup file), when not to use it (e.g., during active operations), or how it differs from similar tools like 'ps_config_set' (which might set individual configs) or 'ps_config_activate' (which might apply configs).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Register a new policy overlay.
| Name | Required | Description | Default |
|---|---|---|---|
| overlayId | Yes | ||
| overlay | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'Register' implies a write operation, the description doesn't address critical aspects like whether this requires special permissions, what happens if an overlayId already exists, whether the operation is idempotent, or what the expected response format is. This leaves significant gaps for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that gets straight to the point with no wasted words. It's appropriately sized for what it communicates, though it communicates very little beyond the basic purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with 2 parameters (one being a nested object), 0% schema coverage, no annotations, and no output schema, the description is severely inadequate. It doesn't explain parameter meanings, behavioral implications, or what happens after registration, leaving the agent with insufficient information to use this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage for both parameters, the description provides no information about what 'overlayId' and 'overlay' represent. The description doesn't explain what constitutes a valid overlayId format, what the overlay object should contain, or any constraints on these parameters, leaving them completely undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Register') and the resource ('a new policy overlay'), providing a specific verb+resource combination. However, it doesn't differentiate this tool from its sibling tools like 'ps_config_get' or 'ps_config_import', which appear to be related configuration operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With many sibling tools like 'ps_config_get', 'ps_config_import', and 'ps_config_activate', there's no indication of when registration is appropriate versus retrieval, import, or activation of policy overlays.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Delegate a task from parent agent to child agent. Enforces inheritance rules and constraint propagation.
| Name | Required | Description | Default |
|---|---|---|---|
| parentAgentId | Yes | ||
| childAgentId | Yes | ||
| parentFrame | Yes | ||
| childFrame | Yes | ||
| task | No | ||
| inheritanceMode | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'enforces inheritance rules and constraint propagation,' which hints at some behavioral traits, but it lacks details on permissions needed, error handling, side effects, or what happens upon delegation. This is insufficient for a tool with 6 parameters and complex operations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with two sentences that are front-loaded and to the point. Every word contributes to the core idea without unnecessary elaboration, making it efficient and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (6 parameters, nested objects, no output schema, and no annotations), the description is incomplete. It doesn't cover parameter meanings, return values, error conditions, or detailed behavioral context, which are crucial for effective tool use in this multi-agent delegation scenario.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning none of the 6 parameters are documented in the schema. The description does not add any meaning beyond the schema—it doesn't explain what parameters like 'parentFrame', 'childFrame', or 'inheritanceMode' represent or how they affect delegation. This leaves parameters largely unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('delegate a task') and the entities involved ('from parent agent to child agent'), making the purpose understandable. However, it doesn't explicitly differentiate this tool from sibling tools like 'ps_delegate_list' or 'ps_delegate_revoke', which would require more specific context about what delegation entails versus listing or revoking delegations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions 'enforces inheritance rules and constraint propagation,' which implies some context for when to use it, but it doesn't provide explicit guidance on when to choose this tool over alternatives like 'ps_execute' or other delegation-related tools. No clear exclusions or prerequisites are stated, leaving usage ambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List delegations for an agent.
| Name | Required | Description | Default |
|---|---|---|---|
| agentId | Yes | ||
| role | No | ||
| status | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'list delegations', which implies a read-only operation, but does not cover aspects like authentication needs, rate limits, pagination, or what the output looks like. For a tool with zero annotation coverage, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste, front-loading the core purpose. It is appropriately sized for a simple list operation, making it highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 3 parameters with 0% schema description coverage, no annotations, and no output schema, the description is incomplete. It does not explain parameter usage, return values, or behavioral traits, which are essential for an agent to invoke the tool correctly in a complex server environment with many siblings.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description does not mention any parameters, and schema description coverage is 0%, with 3 parameters (agentId, role, status) and 2 having enums. The description adds no semantic meaning beyond the schema, but since there are parameters, it does not fully compensate for the coverage gap. Baseline 3 is appropriate as the schema provides structure, but the description offers no additional context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and the resource 'delegations for an agent', making the purpose specific and understandable. However, it does not explicitly differentiate from sibling tools like 'ps_delegate' or 'ps_delegate_revoke', which might handle delegation creation or revocation, so it misses full sibling distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as 'ps_delegate' for creating delegations or 'ps_hold_list' for listing holds. There are no explicit when/when-not statements or named alternatives, leaving usage context implied at best.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Revoke an active delegation.
| Name | Required | Description | Default |
|---|---|---|---|
| delegationId | Yes | ||
| parentAgentId | Yes | ||
| reason | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Revoke' implies a destructive mutation, but the description doesn't specify permissions required, whether the action is reversible, what happens to the delegated resources, or any rate limits. This is a significant gap for a mutation tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It's front-loaded with the core action ('revoke'), making it easy to scan and understand quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (a destructive mutation with 3 parameters), lack of annotations, 0% schema coverage, and no output schema, the description is incomplete. It doesn't cover behavioral aspects, parameter meanings, or expected outcomes, leaving critical gaps for the agent to operate safely and effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so parameters are undocumented. The description doesn't add any meaning to the parameters (delegationId, parentAgentId, reason), such as explaining what a delegationId is, how to obtain it, or what constitutes a valid reason. It fails to compensate for the schema's lack of documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('revoke') and the target ('an active delegation'), which provides a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'ps_delegate' (which likely creates delegations) or 'ps_delegate_list' (which likely lists them), leaving some ambiguity about the exact scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description offers no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an active delegation to revoke), exclusions, or how it relates to sibling tools like 'ps_delegate' or 'ps_delegate_list', leaving the agent to infer usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Execute an action under PromptSpeak frame governance. The gatekeeper validates and enforces constraints.
| Name | Required | Description | Default |
|---|---|---|---|
| agentId | Yes | Unique identifier for the executing agent | |
| frame | Yes | The governing PromptSpeak frame | |
| action | Yes | ||
| parentFrame | No | Parent frame if part of delegation chain |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It mentions validation and enforcement by a gatekeeper, hinting at security or compliance checks, but doesn't disclose critical behavioral traits like whether it's read-only or destructive, permission requirements, rate limits, error handling, or what happens upon execution. This leaves significant gaps for a tool that likely performs actions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences that are front-loaded with the core purpose. There's no wasted text, but it could be slightly more informative without losing efficiency. It's appropriately sized for the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (4 parameters, nested objects, no output schema, and no annotations), the description is incomplete. It doesn't explain what the tool returns, how the gatekeeper operates, or the implications of execution. For a tool that likely performs actions under governance, more context is needed to understand its full behavior and outcomes.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 75%, with parameters like agentId, frame, and action documented in the schema. The description adds no additional meaning about parameters beyond implying governance via 'frame' and 'gatekeeper'. Since coverage is high (>80% threshold not met but close), the baseline is 3, as the schema does most of the work without description compensation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the tool 'Execute[s] an action under PromptSpeak frame governance' with 'gatekeeper validates and enforces constraints', which gives a vague purpose. It mentions a verb ('Execute') and resource ('action'), but lacks specificity about what types of actions or constraints are involved, and doesn't differentiate from siblings like ps_execute_batch or ps_execute_dry_run.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites, context, or exclusions, such as how it differs from ps_execute_batch for bulk operations or ps_execute_dry_run for testing. The description only states what it does, not when to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Execute multiple actions under a single frame. Supports sequential or parallel execution with optional stop-on-failure.
| Name | Required | Description | Default |
|---|---|---|---|
| agentId | Yes | Unique identifier for the executing agent | |
| frame | Yes | The governing PromptSpeak frame for all actions | |
| actions | Yes | Array of actions to execute under the frame | |
| parentFrame | No | Parent frame if part of delegation chain | |
| stopOnFirstFailure | No | Stop executing remaining actions on first failure (sequential only) | |
| parallel | No | Execute all actions in parallel instead of sequentially |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses key behavioral traits: batch execution under a frame, sequential/parallel modes, and stop-on-failure option. However, it doesn't cover important aspects like error handling details, performance implications of parallel execution, or what constitutes a 'failure' (e.g., tool errors vs. validation failures).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is perfectly concise at two sentences. The first sentence states the core purpose, and the second adds crucial behavioral context about execution modes. Every word earns its place with zero redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (batch execution with multiple parameters) and lack of annotations/output schema, the description is adequate but incomplete. It covers the high-level purpose and key behavioral aspects, but doesn't address important details like return values, error formats, or performance considerations that would be needed for robust agent usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 6 parameters thoroughly. The description adds some context by mentioning 'sequential or parallel execution' (relating to the 'parallel' parameter) and 'optional stop-on-failure' (relating to 'stopOnFirstFailure'), but doesn't provide additional semantic meaning beyond what's in the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Execute multiple actions under a single frame. Supports sequential or parallel execution with optional stop-on-failure.' It specifies the verb ('execute'), resource ('multiple actions'), and distinguishes it from sibling tools like ps_execute (single action) and ps_execute_dry_run (simulated execution).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for usage: 'Execute multiple actions under a single frame' implies this is for batch operations, and 'Supports sequential or parallel execution with optional stop-on-failure' gives guidance on execution modes. However, it doesn't explicitly state when to use this vs. alternatives like ps_execute or ps_validate_batch, though the batch nature is implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Check if an action would succeed without executing. Returns decision and coverage analysis.
| Name | Required | Description | Default |
|---|---|---|---|
| agentId | Yes | ||
| frame | Yes | ||
| action | Yes | ||
| parentFrame | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It mentions the tool returns 'decision and coverage analysis', which gives some behavioral insight, but it lacks details on permissions, rate limits, side effects, or error handling. For a tool with 4 parameters and no annotations, this is insufficient transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core functionality ('Check if an action would succeed without executing') and adds the return value. There's no wasted text, making it appropriately concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (4 parameters with 0% schema coverage, no annotations, no output schema), the description is incomplete. It doesn't explain what the parameters mean, how the 'decision and coverage analysis' is structured, or any behavioral nuances. For a tool that likely involves significant logic, this leaves too many gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning none of the 4 parameters (agentId, frame, action, parentFrame) are documented in the schema. The description doesn't add any parameter-specific information beyond the general context of checking actions, failing to compensate for the schema gap. This leaves parameters largely unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: to check if an action would succeed without executing it, and it returns decision and coverage analysis. This is a specific verb ('check') with a clear resource/scope ('an action'), though it doesn't explicitly differentiate from sibling tools like 'ps_execute' or 'ps_validate'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for pre-execution validation ('without executing'), suggesting it should be used before performing actual actions. However, it doesn't provide explicit guidance on when to use this versus alternatives like 'ps_execute' (for actual execution) or 'ps_validate' (for validation), nor does it mention any prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get all feature flags.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'Get all feature flags' but doesn't clarify if this is a read-only operation, what permissions are needed, whether it returns all flags at once or paginates, or any rate limits. For a tool with zero annotation coverage, this leaves significant behavioral gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It's front-loaded with the core action and resource, making it easy to parse quickly without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is minimally adequate but lacks depth. It doesn't explain what 'feature flags' are in this context, the return format, or how it fits with sibling tools, leaving room for confusion in a complex server environment.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0 parameters with 100% coverage, so no parameters need documentation. The description doesn't add parameter details, but that's acceptable here since there are none to explain, aligning with the baseline for zero parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'all feature flags', making the purpose specific and understandable. However, it doesn't differentiate from sibling tools like 'ps_feature_set' or explain what distinguishes getting feature flags from other get operations in the system.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With siblings like 'ps_feature_set' (for setting flags) and various other get operations (e.g., 'ps_config_get', 'ps_state_get'), there's no indication of context, prerequisites, or exclusions for this specific tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Set a feature flag.
| Name | Required | Description | Default |
|---|---|---|---|
| flag | Yes | ||
| enabled | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. 'Set a feature flag' implies a mutation operation, but it lacks details on permissions, side effects, rate limits, or response format. This is inadequate for a tool that modifies system state without any structured safety hints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It's appropriately sized and front-loaded, directly stating the tool's purpose without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with 2 parameters, 0% schema coverage, no annotations, and no output schema, the description is incomplete. It doesn't address behavioral aspects, parameter details, or usage context, leaving significant gaps for an agent to operate effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It mentions 'feature flag' but doesn't explain what 'flag' and 'enabled' parameters represent, their formats, or examples. The description adds minimal value beyond the bare schema, failing to clarify parameter meanings.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Set a feature flag' clearly states the action (set) and resource (feature flag) with a specific verb. It distinguishes from sibling 'ps_feature_get' by indicating a write operation versus a read, though it doesn't explicitly differentiate from other 'set' tools like 'ps_confidence_set' or 'ps_config_set'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites, context, or exclusions, leaving the agent to infer usage from the name and description alone without explicit direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Approve a held execution request. The operation will proceed with optional modifications.
| Name | Required | Description | Default |
|---|---|---|---|
| holdId | Yes | The hold ID to approve | |
| reason | No | Reason for approval (for audit trail) | |
| modifiedFrame | No | Optional: Modified frame to use instead of original | |
| modifiedArgs | No | Optional: Modified arguments to use instead of original | |
| executeNow | No | Execute immediately after approval (default: true) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It states the operation 'will proceed' and mentions optional modifications, but fails to describe critical behaviors: whether this is a destructive/mutative action, what permissions are required, what happens to the original request, or what the response looks like. For a tool that likely changes system state, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that gets straight to the point. It's appropriately sized for the tool's complexity and front-loads the core action. No wasted words, though it could potentially benefit from a second sentence for behavioral context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations and no output schema, the description is incomplete. It doesn't explain what 'approve' actually does to system state, what happens after approval, error conditions, or return values. Given the complexity (5 parameters including nested objects) and lack of structured behavioral information, the description should provide more operational context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 5 parameters thoroughly. The description adds minimal value beyond the schema by mentioning 'optional modifications' which loosely corresponds to 'modifiedFrame' and 'modifiedArgs', but doesn't provide additional context about parameter relationships or usage patterns. Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Approve') and the resource ('a held execution request'), making the purpose immediately understandable. It distinguishes from sibling 'ps_hold_reject' by specifying approval rather than rejection, though it doesn't explicitly mention this distinction. The description is specific but could be more explicit about sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'ps_hold_reject' or 'ps_hold_list'. It mentions 'optional modifications' but doesn't clarify when modifications are appropriate or what prerequisites exist (e.g., needing a hold ID from 'ps_hold_list'). No explicit when/when-not instructions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Configure hold behavior and thresholds. Controls when operations are held for human review.
| Name | Required | Description | Default |
|---|---|---|---|
| action | Yes | Get current config or set new config | |
| config | No | Configuration to set (only for action=set) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions that the tool 'Controls when operations are held for human review,' which implies it affects system behavior, but it doesn't detail critical aspects like whether this requires admin permissions, if changes are reversible, potential side effects, or how it interacts with other tools. For a configuration tool with no annotations, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded, consisting of two sentences that directly state the tool's purpose. There is no wasted language or redundancy, making it efficient and easy to understand at a glance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of the tool (with nested parameters and no output schema), the description is minimally adequate. It covers the basic purpose but lacks details on usage, behavioral context, and output expectations. With no annotations and incomplete guidance, it meets the minimum viable standard but has clear gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with clear documentation of parameters like 'action' and 'config'. The description adds no additional parameter semantics beyond what the schema provides, such as explaining the meaning of thresholds or hold behaviors. Since the schema does the heavy lifting, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Configure hold behavior and thresholds' and 'Controls when operations are held for human review.' It specifies the verb ('configure') and resource ('hold behavior and thresholds'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'ps_hold_approve' or 'ps_hold_list', which is why it doesn't reach a score of 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools or specific contexts for usage, such as when to configure holds versus using other hold-related tools like 'ps_hold_approve'. This lack of explicit or implied usage scenarios results in a low score.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all pending holds awaiting human approval. Returns holds for risky operations that were blocked pending review.
| Name | Required | Description | Default |
|---|---|---|---|
| agentId | No | Optional: Filter holds by agent ID | |
| includeExpired | No | Include expired holds in the list (default: false) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It mentions that holds are for 'risky operations blocked pending review', which adds some context about the tool's purpose, but it lacks details on behavioral traits like authentication requirements, rate limits, pagination, or what specific data is returned in the list.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with zero waste, front-loaded with the core purpose ('List all pending holds awaiting human approval') and followed by clarifying context ('Returns holds for risky operations that were blocked pending review'). Every sentence earns its place by adding value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description provides basic purpose and context but lacks completeness for a tool that likely returns a list of holds. It does not describe the return format, error handling, or other operational details, leaving gaps that could hinder an AI agent's effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents the two parameters (agentId and includeExpired). The description does not add any parameter-specific information beyond what the schema provides, such as explaining the format of agentId or the implications of including expired holds. Baseline 3 is appropriate as the schema handles the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('List') and resource ('all pending holds awaiting human approval'), specifying that these holds are for 'risky operations that were blocked pending review'. It distinguishes from sibling tools like ps_hold_approve and ps_hold_reject by focusing on listing rather than actioning holds.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage context by mentioning 'pending human approval' and 'risky operations blocked pending review', which suggests when to use this tool (to review blocked operations). However, it does not explicitly state when not to use it or name alternatives among siblings, such as ps_hold_stats for aggregated data.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Reject a held execution request. The operation will not proceed.
| Name | Required | Description | Default |
|---|---|---|---|
| holdId | Yes | The hold ID to reject | |
| reason | No | Reason for rejection (for audit trail) | |
| haltAgent | No | Also halt the agent that made the request (default: false) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states the operation 'will not proceed,' which implies a safe, non-destructive action, but lacks details on permissions, side effects (e.g., audit trail impact), or response behavior. For a tool with potential security implications (rejecting holds), this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and outcome, making it highly concise and well-structured for quick understanding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a hold rejection tool (likely involving security or workflow decisions), no annotations, and no output schema, the description is incomplete. It lacks context on what a 'held execution request' is, how to identify one, or what happens after rejection, leaving significant gaps for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents the three parameters. The description adds no additional meaning beyond what's in the schema (e.g., it doesn't explain the implications of 'haltAgent' or format for 'reason'). Baseline 3 is appropriate as the schema handles parameter documentation adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('reject') and resource ('a held execution request'), making the purpose specific and understandable. However, it doesn't explicitly differentiate from its sibling 'ps_hold_approve', which would be the natural alternative for handling held requests.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'ps_hold_approve' or other hold-related tools. The description doesn't mention prerequisites, context for held requests, or exclusions, leaving the agent without usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get hold statistics and history.
| Name | Required | Description | Default |
|---|---|---|---|
| historyLimit | No | Number of historical decisions to return (default: 10) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It doesn't disclose behavioral traits such as whether this is a read-only operation, if it requires specific permissions, rate limits, or what the output format looks like. The description is too minimal to inform the agent adequately.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with a single sentence 'Get hold statistics and history.' It's front-loaded and wastes no words, though this conciseness comes at the cost of detail in other dimensions.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, no output schema, and a vague purpose, the description is incomplete. It doesn't explain what 'hold' means in this context, what statistics or history entail, or how the output is structured, making it insufficient for effective tool use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents the single optional parameter. The description doesn't add any parameter semantics beyond the schema, but with 0 required parameters and high coverage, the baseline is high. No compensation is needed as the schema handles it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get hold statistics and history' clearly states the action (get) and resource (hold statistics/history), but it's vague about what 'hold' refers to and doesn't differentiate from sibling tools like ps_hold_list, ps_hold_approve, or ps_hold_reject. It provides basic purpose but lacks specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. With sibling tools like ps_hold_list and ps_hold_config, the description doesn't explain if this is for aggregated data, historical analysis, or other contexts, leaving usage unclear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Configure security detection patterns. List, enable, disable, or change severity of patterns.
| Name | Required | Description | Default |
|---|---|---|---|
| action | Yes | Configuration action to perform | |
| patternId | No | Pattern ID to modify (required for enable, disable, set_severity) | |
| severity | No | New severity level (required for set_severity) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions actions (list, enable, disable, set_severity) but doesn't describe side effects, permissions required, rate limits, or what happens when patterns are modified (e.g., if changes are immediate or require restart). For a configuration tool with mutation capabilities and no annotations, this leaves significant gaps in understanding its behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose ('Configure security detection patterns') and lists the specific actions. There is no wasted language or redundancy, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (multiple mutation actions like enable/disable/set_severity), lack of annotations, and no output schema, the description is insufficient. It doesn't explain return values, error conditions, or behavioral nuances (e.g., if 'list' returns all patterns or a subset). For a configuration tool with potential side effects, more context is needed to use it effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters (action, patternId, severity) with enums and requirements. The description adds minimal value by implying the tool handles 'security detection patterns,' which aligns with the schema but doesn't provide additional syntax, examples, or constraints beyond what's in the structured data. This meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with specific verbs (configure, list, enable, disable, change severity) and identifies the resource (security detection patterns). It distinguishes this as a configuration tool for security patterns, which differentiates it from sibling tools like ps_security_gate or ps_security_scan that likely have different functions. However, it doesn't explicitly contrast with all siblings, keeping it at a 4 rather than a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, dependencies, or compare it to sibling tools like ps_security_gate or ps_security_scan. The agent must infer usage from the description alone, which only states what the tool does, not when to choose it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Scan code and enforce security policy. Blocks on critical findings, holds high-severity for review, warns on medium, logs low/info.
| Name | Required | Description | Default |
|---|---|---|---|
| content | Yes | Code content to scan | |
| action | Yes | The action being gated (e.g., "write_file", "edit_file") |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses behavioral traits like blocking on critical findings and holding high-severity issues, which is useful. However, it lacks details on permissions required, rate limits, error handling, or what 'blocks' means in practice (e.g., returns error, throws exception). For a security tool with no annotations, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose and details severity handling without waste. Every part earns its place by clarifying the tool's behavior concisely.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, no output schema, and a security tool with potential side effects (e.g., blocking actions), the description is incomplete. It doesn't cover return values, error cases, or integration context, leaving gaps for an AI agent to understand full usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters ('content' as code to scan, 'action' as the gated action). The description doesn't add meaning beyond this, such as examples of valid actions or content formats. Baseline 3 is appropriate when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose as scanning code and enforcing security policy with specific severity-based actions (blocks, holds, warns, logs). It uses specific verbs ('scan', 'enforce') and identifies the resource ('code'), though it doesn't explicitly differentiate from sibling tools like 'ps_security_scan' or 'ps_security_config'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. With siblings like 'ps_security_scan' and 'ps_security_config', the description doesn't indicate whether this is for pre-commit gating, continuous integration, or other contexts, nor does it mention prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Scan code content for security vulnerabilities. Returns findings classified by severity (critical, high, medium, low, info).
| Name | Required | Description | Default |
|---|---|---|---|
| content | Yes | Code content to scan | |
| patterns | No | Optional: Only run these specific pattern IDs |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. While it mentions the return format (findings classified by severity), it lacks critical details such as whether this is a read-only operation, potential performance impacts, rate limits, authentication requirements, or error handling. For a security scanning tool with zero annotation coverage, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is brief and front-loaded with the core purpose in the first sentence. Both sentences earn their place by stating the action and return format, though it could be slightly more structured (e.g., separating purpose from output details). No wasted words, but minor room for improvement.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of security scanning, lack of annotations, and no output schema, the description is incomplete. It doesn't cover behavioral aspects like safety, performance, or error conditions, nor does it explain the return structure beyond severity levels. For a tool with 2 parameters and no structured output, more context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already fully documents both parameters (content and patterns). The description adds no additional parameter semantics beyond what's in the schema, such as examples of pattern IDs or content format expectations. Baseline 3 is appropriate when the schema does all the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: scanning code content for security vulnerabilities and returning classified findings. It specifies both the action (scan) and resource (code content), though it doesn't explicitly differentiate from sibling tools like ps_validate or ps_security_config, which prevents a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With sibling tools like ps_validate, ps_security_config, and ps_security_gate, there's no indication of how this security scan differs or when it's the appropriate choice, leaving the agent without contextual usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get drift history for an agent.
| Name | Required | Description | Default |
|---|---|---|---|
| agentId | Yes | ||
| since | No | Unix timestamp | |
| limit | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It implies a read operation ('Get'), but doesn't disclose behavioral traits such as permissions needed, rate limits, response format, or whether it's safe or destructive. This leaves significant gaps for an agent to understand how to interact with it effectively.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, straightforward sentence with no wasted words. It's front-loaded and efficiently conveys the core purpose without unnecessary elaboration, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (a tool with 3 parameters, no annotations, and no output schema), the description is incomplete. It lacks details on behavior, parameter meanings, and return values, leaving the agent with insufficient context to use the tool correctly beyond a basic understanding of its purpose.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 33% (only 'since' has a description), and the description doesn't add any parameter details beyond the tool name. It doesn't explain what 'agentId', 'since', or 'limit' mean in context, so it doesn't compensate for the low coverage, but with 0 parameters documented in the description, the baseline is 3 as it doesn't contradict the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get drift history for an agent' clearly states the action (get) and resource (drift history for an agent), but it's vague about what 'drift history' entails and doesn't distinguish this tool from siblings like ps_state_get or ps_state_system. It provides a basic purpose but lacks specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description offers no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, context, or exclusions, and with siblings like ps_state_get, there's no indication of how this differs in usage scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get current state of an agent including drift metrics and circuit breaker status.
| Name | Required | Description | Default |
|---|---|---|---|
| agentId | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states the tool retrieves state information, implying it's a read operation, but doesn't disclose behavioral traits like whether it requires authentication, has rate limits, returns real-time or cached data, or what happens if the agent doesn't exist. The description adds minimal context beyond the basic action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core action and details. Every word earns its place by specifying what is retrieved and the key metrics included, with no wasted text. It's appropriately sized for a simple retrieval tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (state retrieval with metrics), lack of annotations, and no output schema, the description is incomplete. It doesn't explain the return format, error conditions, or how drift metrics and circuit breaker status are structured. For a tool with behavioral implications and undocumented output, more context is needed to be fully helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 1 parameter with 0% description coverage, so the description must compensate. It mentions 'agentId' implicitly by referring to 'an agent', but doesn't explain what an agent is, its format, or where to find it. The description adds marginal meaning by linking the parameter to the resource, but doesn't fully clarify semantics beyond what's obvious from the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get') and resource ('current state of an agent'), specifying what information is retrieved ('drift metrics and circuit breaker status'). It distinguishes from siblings like 'ps_state_drift_history' (historical data) and 'ps_state_system' (system-wide state), but doesn't explicitly contrast them. The purpose is specific but could be more distinct regarding scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, such as needing an active agent, or compare it to siblings like 'ps_state_system' for broader system state. Usage is implied by the action, but no explicit context or exclusions are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Immediately halt an agent by opening its circuit breaker.
| Name | Required | Description | Default |
|---|---|---|---|
| agentId | Yes | ||
| reason | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'immediately halt' implies a potentially disruptive action, it doesn't specify consequences like whether the halt is reversible, what happens to ongoing processes, or if special permissions are required. This leaves significant behavioral gaps for a tool that appears to be a critical control operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that gets straight to the point with no wasted words. It's appropriately sized for a tool with two parameters and no complex output schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool that appears to perform a potentially destructive action (halting an agent) with no annotations and no output schema, the description is insufficient. It doesn't explain what 'halt' means in practical terms, what the circuit breaker metaphor implies, or what the user should expect after invocation. The lack of parameter guidance further compounds the incompleteness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage for both parameters (agentId and reason), the description provides no information about what these parameters mean or how they should be used. The description doesn't mention parameters at all, failing to compensate for the complete lack of schema documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('immediately halt') and target ('an agent') with a specific mechanism ('by opening its circuit breaker'), providing a concrete verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like ps_state_reset or ps_state_recalibrate, which might also affect agent states.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. With siblings like ps_state_reset, ps_state_recalibrate, and ps_state_resume that might relate to agent state management, the description offers no context about appropriate use cases or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recalibrate agent drift baseline. Optionally provide a new baseline configuration.
| Name | Required | Description | Default |
|---|---|---|---|
| agentId | Yes | Agent to recalibrate | |
| newBaseline | No | Optional new baseline configuration. If omitted, recalibrates from current state. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but offers minimal behavioral insight. It states the tool recalibrates a baseline, implying a mutation, but doesn't disclose effects (e.g., whether it's destructive, requires permissions, has side effects like downtime, or impacts system state). No rate limits, error conditions, or output details are mentioned, leaving significant gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is highly concise with two sentences that directly state the purpose and parameter behavior. Every word earns its place, with no redundancy or fluff, making it easy to parse and front-loaded for clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a mutation tool (recalibrating agent drift) with no annotations and no output schema, the description is incomplete. It lacks details on behavioral traits, error handling, prerequisites, or what 'recalibrate' means in practice. While concise, it doesn't provide enough context for safe or effective use, especially compared to siblings with clearer purposes.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents parameters (agentId, newBaseline). The description adds marginal value by noting that newBaseline is optional and that omission leads to recalibration from the current state, but doesn't explain semantics beyond what's in the schema (e.g., what 'recalibrate' entails for the agent). Baseline 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Recalibrate') and the target ('agent drift baseline'), with an optional capability ('provide a new baseline configuration'). It distinguishes from siblings like ps_state_get or ps_state_reset by focusing on recalibration rather than retrieval or resetting. However, it doesn't explicitly differentiate from all siblings (e.g., ps_state_system), keeping it from a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives is provided. The description mentions an optional parameter but doesn't specify scenarios for recalibration (e.g., after drift detection, during maintenance) or contrast with siblings like ps_state_reset. Usage is implied through the action, but lacks context for decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Reset agent state. Can reset circuit breaker, drift metrics, and/or baseline.
| Name | Required | Description | Default |
|---|---|---|---|
| agentId | Yes | ||
| resetCircuitBreaker | No | ||
| resetDriftMetrics | No | ||
| resetBaseline | No | ||
| reason | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only mentions what can be reset without detailing behavioral traits. It doesn't disclose whether this is a destructive operation, requires specific permissions, has side effects on agent performance, or provides any response format. For a state-modifying tool with 5 parameters, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose and lists reset options without unnecessary elaboration. Every word serves a clear purpose, making it appropriately sized for the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (5 parameters, no annotations, no output schema, state-modifying operation), the description is incomplete. It lacks details on behavioral impact, parameter purposes beyond the booleans, and expected outcomes, leaving significant gaps for an agent to understand how to invoke it correctly in context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate but only partially does. It mentions three boolean parameters (circuit breaker, drift metrics, baseline) but omits agentId and reason, which are required. The description adds some meaning for the boolean parameters but leaves two critical parameters undocumented, failing to fully address the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('reset') and target ('agent state'), specifying what components can be reset (circuit breaker, drift metrics, baseline). It distinguishes from siblings like ps_state_get or ps_state_halt by focusing on resetting specific state elements rather than retrieving or controlling execution state.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like ps_state_recalibrate or ps_state_system is provided. The description implies usage for resetting specific state components but doesn't clarify prerequisites, timing, or exclusions, leaving the agent to infer context from sibling tool names alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Resume a halted agent.
| Name | Required | Description | Default |
|---|---|---|---|
| agentId | Yes | ||
| reason | Yes | ||
| resetMetrics | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool resumes a halted agent, implying a state change operation, but doesn't describe what 'resume' entails (e.g., restarting execution, restoring state), potential side effects, permissions required, or error conditions. This is inadequate for a mutation tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It is appropriately sized and front-loaded, directly stating the tool's purpose without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (state mutation with 3 parameters), lack of annotations, and no output schema, the description is incomplete. It doesn't cover parameter meanings, behavioral details, or return values, leaving the agent poorly equipped to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate by explaining parameters. It mentions no parameters at all, leaving 'agentId', 'reason', and 'resetMetrics' undocumented. This fails to add meaning beyond the bare schema, creating significant gaps in understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Resume') and the target ('a halted agent'), providing a specific verb+resource combination. However, it doesn't distinguish this tool from its sibling 'ps_state_halt' (which presumably halts agents) or other state management tools, missing explicit differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when an agent is halted, but provides no explicit guidance on when to use this tool versus alternatives like 'ps_state_reset' or 'ps_state_recalibrate', nor does it mention prerequisites or exclusions. This leaves the agent with minimal contextual direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get overall system state including all agents, operations, and drift alerts.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only states it 'gets' data without disclosing behavioral traits like read-only status (implied but not confirmed), potential rate limits, authentication needs, or response format. It lacks details on what 'overall system state' entails beyond listed components.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core action ('Get overall system state') and specifies included components. There's no wasted words or redundant information, making it highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter read tool with no output schema, the description is minimally adequate but incomplete. It explains what data is retrieved but not the format, scope (e.g., real-time vs. cached), or limitations. Given the lack of annotations and output schema, more context on behavior would be helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters with 100% schema description coverage, so no parameter documentation is needed. The description appropriately doesn't discuss parameters, maintaining focus on the tool's purpose without unnecessary detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'overall system state', specifying it includes 'all agents, operations, and drift alerts'. It distinguishes this tool from siblings like ps_state_get (which likely gets specific state) and ps_state_drift_history (which focuses on drift history), but doesn't explicitly contrast them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like ps_state_get or ps_state_drift_history. It doesn't mention prerequisites, timing, or exclusions, leaving the agent to infer usage from the name and description alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Add an alternative explanation to a symbol's findings.
CRITICAL for preventing false positives: When a pattern-based finding could have multiple explanations, document them here. Adding high-likelihood alternatives automatically reduces confidence in the original claim.
Examples:
"9 identical payments" → Alternative: "Monthly insurance premium financing"
"Large round numbers" → Alternative: "Negotiated contract amounts"
"Vendor with single customer" → Alternative: "Subsidiary company"
| Name | Required | Description | Default |
|---|---|---|---|
| symbolId | Yes | Symbol ID to update | |
| alternative | Yes | Description of the alternative explanation | |
| likelihood | Yes | Estimated likelihood this alternative is correct (0-1) | |
| reasoning | No | Why this alternative is plausible | |
| added_by | Yes | Who is adding this alternative |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It clearly describes the tool's effect ('automatically reduces confidence in the original claim') and provides important context about its purpose in preventing false positives. However, it doesn't mention potential side effects, permissions needed, or error conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficiently structured with a clear purpose statement, critical usage context, and concrete examples. Every sentence serves a distinct purpose: establishing the action, explaining why it matters, and illustrating proper use. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations and no output schema, the description does well by explaining the tool's purpose, when to use it, and its behavioral effect. The examples provide valuable context. However, it doesn't describe what happens on success/failure or the return format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 5 parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema, but the examples help illustrate how parameters like 'alternative' and 'reasoning' might be used in practice.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the verb ('Add') and resource ('alternative explanation to a symbol's findings'), making the purpose clear. It distinguishes from siblings like ps_symbol_create or ps_symbol_update by focusing specifically on adding alternative explanations rather than general symbol operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool: 'CRITICAL for preventing false positives: When a pattern-based finding could have multiple explanations, document them here.' It also explains the effect ('automatically reduces confidence in the original claim') and gives concrete examples of appropriate use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Create a new directive symbol in the registry.
Symbol IDs follow the pattern:
Public companies: Ξ.{TICKER}.{PERIOD} (e.g., Ξ.NVDA.Q3FY25)
People: Ξ.I.{NAME}.{CONTEXT} (e.g., Ξ.I.JENSEN_HUANG.BIO)
Events: Ξ.E.{TYPE}.{ID} (e.g., Ξ.E.EARNINGS.NVDA.20241120)
Sectors: Ξ.S.{SECTOR}.{CONTEXT} (e.g., Ξ.S.SEMICONDUCTORS.2024)
Tasks: Ξ.T.{PROJECT}.{ID} (e.g., Ξ.T.PORTFOLIO_REVIEW.001)
Knowledge: Ξ.K.{DOMAIN}.{TOPIC} (e.g., Ξ.K.CHEMISTRY.WATER)
Queries: Ξ.Q.{DATASET}.{ID} (e.g., Ξ.Q.DEEPSEARCHQA.001)
| Name | Required | Description | Default |
|---|---|---|---|
| symbolId | Yes | Unique symbol ID following namespace rules (e.g., Ξ.NVDA.Q3FY25) | |
| category | No | Category (auto-inferred from ID if not provided) | |
| who | Yes | Who needs this / who is the audience | |
| what | Yes | What is being analyzed or done | |
| why | Yes | Why this matters / purpose | |
| where | Yes | Scope (company, market, geography) | |
| when | Yes | Time context | |
| how | Yes | ||
| commanders_intent | Yes | Ultimate goal in one sentence - the north star | |
| requirements | Yes | MUST include these elements | |
| anti_requirements | No | MUST NOT include these elements | |
| key_terms | No | Terms that MUST appear in output | |
| tags | No | Freeform tags for filtering | |
| parent_symbol | No | Parent symbol ID for hierarchical symbols | |
| related_symbols | No | Related symbol IDs | |
| created_by | No | Creator identifier |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool creates a new symbol but lacks details on permissions, side effects, error handling, or response format. For a mutation tool with 16 parameters, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose in the first sentence, followed by structured examples. While the examples are detailed, they are relevant for understanding ID patterns. There is minimal waste, but it could be more concise by integrating the examples more tightly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity with 16 parameters, no annotations, and no output schema, the description is incomplete. It explains ID patterns but does not cover behavioral aspects like mutation effects, error cases, or return values, which are crucial for a creation tool in this context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is high at 94%, so the schema already documents most parameters. The description adds value by explaining the symbol ID patterns for different categories, which clarifies the 'symbolId' parameter beyond the schema's generic description. However, it does not elaborate on other parameters like 'how' or 'commanders_intent.'
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Create' and the resource 'directive symbol in the registry,' making the purpose specific and unambiguous. It distinguishes from siblings like ps_symbol_update, ps_symbol_delete, and ps_symbol_get by focusing on creation rather than modification, deletion, or retrieval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites, dependencies, or compare it to sibling tools such as ps_symbol_update or ps_symbol_import, leaving the agent without context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Delete a symbol from the registry.
| Name | Required | Description | Default |
|---|---|---|---|
| symbolId | Yes | Symbol ID to delete | |
| reason | Yes | Reason for deletion |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It states 'Delete' which implies a destructive mutation, but fails to mention critical details like whether deletion is permanent, requires specific permissions, has side effects on related data, or what happens on success/failure. For a destructive tool, this lack of transparency is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, direct sentence with zero wasted words—'Delete a symbol from the registry.' It is front-loaded and efficiently communicates the core action without unnecessary elaboration, making it easy for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's destructive nature, no annotations, and no output schema, the description is incomplete. It doesn't cover behavioral aspects like permanence or permissions, return values, or error handling. For a mutation tool with two required parameters, this minimal description leaves too many gaps for reliable agent invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with both parameters ('symbolId' and 'reason') clearly documented in the schema. The description adds no additional meaning beyond implying these parameters are used for deletion. Since the schema does the heavy lifting, the baseline score of 3 is appropriate, as the description doesn't compensate but also doesn't detract.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete') and resource ('a symbol from the registry'), making the purpose immediately understandable. It distinguishes from sibling tools like 'ps_symbol_get' or 'ps_symbol_update' by specifying deletion rather than retrieval or modification. However, it doesn't explicitly contrast with 'ps_symbol_add_alternative' or 'ps_symbol_verify', which slightly limits differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'ps_symbol_update' for modifications or 'ps_symbol_list' for viewing symbols. The description lacks context about prerequisites (e.g., needing symbol ID from a list operation) or exclusions (e.g., not for bulk deletions). This leaves the agent without explicit usage instructions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Format a symbol for inclusion in a prompt. Returns LLM-ready text.
| Name | Required | Description | Default |
|---|---|---|---|
| symbolId | Yes | Symbol ID to format | |
| format | No | Format style (default: full) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the output ('Returns LLM-ready text') but doesn't describe error handling, rate limits, authentication needs, or what happens if the symbolId is invalid. For a tool with zero annotation coverage, this leaves significant behavioral gaps, though it at least indicates the return type.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—two short sentences that are front-loaded with the core purpose and outcome. Every word earns its place, with no redundant information or fluff, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose and output but lacks details on behavioral traits, error cases, or integration context. Without annotations or output schema, more completeness would be beneficial, but it meets the minimum viable threshold.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with clear documentation for symbolId and format (including enum values and default). The description adds no additional parameter semantics beyond what the schema provides, such as examples of formatted output or details on format styles. Baseline 3 is appropriate since the schema does the heavy lifting, but no extra value is added.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Format a symbol') and the purpose ('for inclusion in a prompt'), with a specific outcome ('Returns LLM-ready text'). It distinguishes from sibling tools like ps_symbol_get or ps_symbol_list by focusing on formatting rather than retrieval or management. However, it doesn't explicitly differentiate from ps_symbol_update or ps_symbol_verify, which could involve symbol content.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention scenarios like preparing symbols for AI processing, contrast with ps_symbol_get for raw data, or specify prerequisites such as needing an existing symbol. Usage is implied by the phrase 'for inclusion in a prompt,' but lacks explicit when/when-not instructions or named alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Retrieve a directive symbol by ID. Returns the full symbol with all grounding context.
| Name | Required | Description | Default |
|---|---|---|---|
| symbolId | Yes | Symbol ID to retrieve (e.g., Ξ.NVDA.Q3FY25) | |
| version | No | Specific version to retrieve (optional, defaults to latest) | |
| include_changelog | No | Include version changelog (default: true) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions that the tool 'returns the full symbol with all grounding context,' which adds some behavioral context beyond the basic retrieval action. However, it lacks details on permissions, rate limits, error conditions, or what 'grounding context' entails, leaving significant gaps for a tool with no annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose ('Retrieve a directive symbol by ID') and adds a useful detail about the return value. There is no wasted verbiage, making it appropriately concise for a simple retrieval tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (a simple read operation with 3 parameters) and high schema coverage (100%), the description is somewhat complete. However, with no output schema and no annotations, it should ideally explain more about the return format (e.g., what 'full symbol' and 'grounding context' include) and behavioral aspects like error handling, leaving room for improvement.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description does not add any additional meaning or context beyond what the schema provides, such as explaining the format of 'symbolId' or the implications of 'include_changelog'. This meets the baseline of 3 when schema coverage is high.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'retrieve' and resource 'directive symbol by ID', making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'ps_symbol_get' vs 'ps_symbol_list' or 'ps_symbol_stats', which would require a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'ps_symbol_list' for listing symbols or 'ps_symbol_stats' for statistics. It mentions what the tool does but not when it's the appropriate choice among the many sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Bulk import symbols from external data (HuggingFace, JSON, etc.).
| Name | Required | Description | Default |
|---|---|---|---|
| source | Yes | Data source format | |
| data | Yes | The data to import (format depends on source) | |
| category | Yes | Category for imported symbols | |
| id_prefix | Yes | Prefix for generated symbol IDs (e.g., Ξ.Q.DEEPSEARCHQA) | |
| transform | No | Field mapping from source data to symbol fields | |
| defaults | No | Default values for fields not in source data |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but offers minimal behavioral insight. 'Bulk import' implies a write/mutation operation, but it doesn't disclose critical traits: whether it overwrites existing symbols, requires specific permissions, has rate limits, returns success/failure counts, or handles errors. For a complex import tool with 6 parameters, this is inadequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It's front-loaded with the core action ('bulk import symbols') and includes relevant examples. Every word earns its place, making it easy for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 parameters, nested objects, no output schema, and no annotations), the description is incomplete. It doesn't address behavioral aspects like mutation effects, error handling, or return values. For a bulk import operation that likely modifies system state, more context is needed to use it safely and effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters well. The description adds marginal value by hinting at external sources (HuggingFace, JSON, etc.), which aligns with the 'source' enum, but doesn't explain parameter interactions (e.g., how 'transform' maps to 'data'). Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('bulk import') and resource ('symbols from external data'), with specific examples of sources (HuggingFace, JSON, etc.). It distinguishes from sibling tools like ps_symbol_create (single creation) and ps_config_import (configuration import), though not explicitly named. It loses a point for not explicitly differentiating from siblings like ps_symbol_add_alternative or ps_symbol_update.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., data format requirements), when to choose this over ps_symbol_create for single symbols, or how it differs from ps_config_import. The agent must infer usage from the name and parameters alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List symbols with optional filtering.
| Name | Required | Description | Default |
|---|---|---|---|
| category | No | Filter by category | |
| tags | No | Filter by tags | |
| created_after | No | Filter by creation date (ISO 8601) | |
| created_before | No | Filter by creation date (ISO 8601) | |
| search | No | Search in symbolId and commanders_intent | |
| limit | No | Max results (default: 50) | |
| offset | No | Pagination offset |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. 'List symbols' implies a read-only operation, but the description doesn't address authentication requirements, rate limits, pagination behavior (beyond schema parameters), error handling, or what the output looks like. It mentions 'optional filtering' but doesn't explain how filters combine or their precedence.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—just 5 words—with zero wasted language. It's front-loaded with the core action ('List symbols') and efficiently notes the key capability ('with optional filtering'). Every word earns its place, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 7 parameters, no annotations, and no output schema, the description is insufficiently complete. It doesn't explain what 'symbols' represent in this system, how results are structured, or behavioral aspects like pagination defaults. The agent must rely heavily on the schema alone, missing important contextual understanding for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the schema already documents all 7 parameters thoroughly. The description adds no additional parameter semantics beyond stating 'optional filtering'—it doesn't explain what 'symbols' are, how filtering works in practice, or provide examples. The baseline score of 3 reflects adequate but minimal value added over the comprehensive schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List symbols with optional filtering' clearly states the verb ('list') and resource ('symbols'), but it's vague about what 'symbols' are in this context and doesn't distinguish this tool from sibling tools like 'ps_symbol_list_unverified' or 'ps_symbol_get'. It provides basic purpose but lacks specificity about the domain or scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With sibling tools like 'ps_symbol_list_unverified' and 'ps_symbol_get' available, there's no indication of when this filtered listing is preferred over those options, nor any mention of prerequisites or typical use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List symbols that require human review.
Returns symbols flagged for review due to:
Accusatory claims without sufficient evidence
Missing alternative explanations
High-stakes claims (fraud, violations, diagnoses)
Low confidence scores
Use this to find claims that need human validation before action.
| Name | Required | Description | Default |
|---|---|---|---|
| claim_type | No | Filter by claim type (e.g., ACCUSATORY for fraud allegations) | |
| min_confidence | No | Minimum confidence level to include | |
| max_confidence | No | Maximum confidence level to include | |
| limit | No | Maximum results to return (default: 50) | |
| offset | No | Pagination offset |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It describes the tool's function (listing symbols for review) and criteria for inclusion (e.g., 'High-stakes claims'), which adds useful context beyond basic parameters. However, it lacks details on behavioral traits like rate limits, pagination behavior (beyond the 'limit' and 'offset' parameters in the schema), error handling, or authentication requirements. For a tool with no annotations, this is adequate but leaves gaps in operational transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and concise, with zero wasted sentences. It front-loads the purpose ('List symbols that require human review'), provides specific criteria in a bullet-like list, and ends with clear usage guidance. Each sentence earns its place by adding essential context or instructions, making it efficient and easy to parse for an AI agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (5 parameters, no output schema, no annotations), the description is reasonably complete. It explains the tool's purpose, usage context, and review criteria, which compensates for the lack of output schema by clarifying what the tool returns (symbols flagged for review). However, it could be more complete by addressing potential behavioral aspects like pagination details or error scenarios, but it covers the core functionality adequately for an agent to use it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, meaning all parameters are documented in the input schema (e.g., 'claim_type' with enum values, 'min_confidence' with range). The description does not add any parameter-specific details beyond what the schema provides, such as explaining how parameters interact with the review criteria. Since the schema does the heavy lifting, the baseline score of 3 is appropriate, as the description adds no extra parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'List symbols that require human review.' It specifies the verb ('List'), resource ('symbols'), and scope ('that require human review'), distinguishing it from sibling tools like 'ps_symbol_list' (which likely lists all symbols without filtering for review status). The description also enumerates specific criteria for review (e.g., 'Accusatory claims without sufficient evidence'), making it highly specific and actionable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool: 'Use this to find claims that need human validation before action.' It provides clear context for usage (finding items requiring review) and implies an alternative by distinguishing it from tools that might list all symbols (e.g., 'ps_symbol_list'). The guidance is direct and practical, helping the agent understand its role in a workflow involving human validation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get statistics about the symbol registry.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states this is a 'get' operation (implying read-only), but doesn't disclose behavioral traits like whether it requires authentication, has rate limits, returns aggregated data, or what format the statistics come in. For a tool with zero annotation coverage, this is inadequate disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a simple tool with no parameters and gets straight to the point without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters, no annotations, and no output schema, the description is minimally complete. It tells what the tool does but lacks crucial context about what statistics are returned, their format, or behavioral constraints. For a statistical tool with no structured output documentation, more detail would be helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters with 100% schema description coverage, so the baseline is 4. The description doesn't need to explain parameters, and it correctly doesn't mention any. No additional parameter semantics are required or provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get statistics about the symbol registry' clearly states the action (get) and resource (symbol registry statistics). It distinguishes from siblings like ps_symbol_get (retrieves specific symbols) and ps_symbol_list (lists symbols), but doesn't explicitly differentiate from ps_hold_stats (statistics about holds) or other stats tools, keeping it at 4 rather than 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With siblings like ps_symbol_get, ps_symbol_list, and ps_hold_stats, there's no indication of when statistical information about the symbol registry is needed versus other symbol operations or statistical tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Update an existing symbol. Creates a new version with updated hash.
| Name | Required | Description | Default |
|---|---|---|---|
| symbolId | Yes | Symbol ID to update | |
| changes | Yes | Fields to update (who, what, why, where, when, how, commanders_intent, requirements, etc.) | |
| change_description | Yes | Description of what changed (for changelog) | |
| changed_by | No | Who made the change |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool updates an existing symbol and creates a new version, but lacks critical details: whether this is a destructive mutation, what permissions are required, if changes are reversible, how the hash is used, or what happens on failure. For a mutation tool with zero annotation coverage, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—two short sentences that directly state the tool's purpose and key behavior ('Creates a new version with updated hash'). It is front-loaded with the main action and wastes no words, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (mutation with 4 parameters, nested objects, no output schema, and no annotations), the description is incomplete. It lacks behavioral context (e.g., side effects, error handling), doesn't explain the return value or versioning implications, and provides minimal guidance for use. For a tool that modifies data and has siblings with overlapping functions, more detail is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters (symbolId, changes, change_description, changed_by). The description adds no additional meaning beyond what the schema provides—it doesn't explain the structure of 'changes' (e.g., what fields like 'who' or 'commanders_intent' represent) or provide examples. Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Update an existing symbol') and the resource ('symbol'), with the specific detail that it 'Creates a new version with updated hash.' This distinguishes it from siblings like ps_symbol_create (create new) or ps_symbol_delete (remove). However, it doesn't explicitly differentiate from ps_symbol_format or ps_symbol_verify, which may also modify symbols in some way.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., requires an existing symbol), exclusions (e.g., not for creating new symbols), or compare to siblings like ps_symbol_create (for new symbols) or ps_symbol_format (for formatting). Usage is implied by the name but not explicitly stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Record human verification of a symbol claim.
Use this tool to upgrade or dispute claims based on human review:
VERIFIED: Human expert has confirmed the claim is accurate
CORROBORATED: Additional evidence supports the claim
DISPUTED: Human reviewer found the claim to be incorrect or misleading
Important: Accusatory claims (fraud, violations) should be DISPUTED if they lack evidence or have plausible alternative explanations.
| Name | Required | Description | Default |
|---|---|---|---|
| symbolId | Yes | Symbol ID to verify | |
| new_status | Yes | New epistemic status for the claim | |
| new_confidence | No | New confidence level (0-1). Auto-calculated if not provided. | |
| evidence_added | No | List of evidence sources that support this verification | |
| reviewer | Yes | Identifier of the human reviewer | |
| notes | No | Notes explaining the verification decision |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It explains the tool's purpose and usage scenarios well, but doesn't address important behavioral aspects like whether this operation is reversible, what permissions are required, how it affects system state, or what happens to existing verification data. The description adds value but leaves significant behavioral questions unanswered.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is perfectly structured and concise. It starts with a clear purpose statement, then provides usage guidance in bullet points, and ends with an important note about accusatory claims. Every sentence earns its place, with zero wasted words or redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 6 parameters, 100% schema coverage, and no output schema, the description provides excellent context about when and how to use it. The main gap is the lack of information about what the tool returns or how the verification affects system state. Given the complexity of verification operations, some information about behavioral outcomes would be helpful, but the description covers usage context thoroughly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 6 parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema descriptions. It provides context about the status values but doesn't explain parameter interactions or usage nuances beyond the schema's baseline documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with specific verbs ('record human verification', 'upgrade or dispute claims') and identifies the resource ('symbol claim'). It distinguishes from siblings like ps_symbol_get or ps_symbol_update by focusing specifically on verification status changes rather than general retrieval or modification.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool ('to upgrade or dispute claims based on human review') and includes specific scenarios for each status (VERIFIED, CORROBORATED, DISPUTED). It also gives important exclusion criteria for accusatory claims that should be disputed under certain conditions, offering clear decision-making context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Validate a PromptSpeak frame. Returns validation report with errors, warnings, and suggestions.
| Name | Required | Description | Default |
|---|---|---|---|
| frame | Yes | The PromptSpeak frame to validate (e.g., "⊕◊▶β") | |
| parentFrame | No | Optional parent frame for chain validation | |
| validationLevel | No | Level of validation to perform | |
| strict | No | If true, warnings also cause validation failure |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the return type ('validation report with errors, warnings, and suggestions') which is helpful, but doesn't describe what happens during validation - whether it's a read-only operation, if it modifies state, what permissions are required, or any performance characteristics. For a validation tool with zero annotation coverage, this leaves significant behavioral questions unanswered.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is perfectly concise - one sentence that states the action and return value with zero wasted words. It's front-loaded with the core purpose and doesn't include unnecessary elaboration. Every word earns its place in this compact description.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a validation tool with 4 parameters, 100% schema coverage, but no annotations and no output schema, the description provides the minimum viable information. It states what the tool does and what it returns, but doesn't address behavioral aspects like whether validation is resource-intensive, if it requires specific permissions, or how the validation report is structured. The absence of an output schema means the description should ideally say more about the return format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all four parameters thoroughly. The description adds no additional parameter information beyond what's in the schema. It doesn't explain the relationship between parameters (e.g., how parentFrame interacts with validationLevel='chain') or provide examples of valid frame strings. Baseline 3 is appropriate when the schema does all the parameter documentation work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Validate a PromptSpeak frame' with a specific verb ('validate') and resource ('PromptSpeak frame'). It distinguishes from sibling tools like ps_execute or ps_symbol_get by focusing on validation rather than execution or symbol management. However, it doesn't explicitly differentiate from ps_validate_batch, which appears to be a batch version of the same function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention ps_validate_batch for batch operations, nor does it explain when validation is needed versus execution tools like ps_execute. There's no context about prerequisites, typical use cases, or integration with other tools in the system.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Validate multiple frames at once. Useful for validating delegation chains.
| Name | Required | Description | Default |
|---|---|---|---|
| frames | Yes | ||
| validationLevel | No | ||
| strict | No | ||
| stopOnFirstError | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions validation but doesn't explain what validation entails (e.g., checks performed, error handling, or output format). For a tool with 4 parameters and no annotation coverage, this is inadequate, though it hints at a use case ('delegation chains').
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is highly concise with two sentences that are front-loaded and waste-free. The first sentence states the core purpose, and the second adds contextual value without redundancy, making it efficient and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (4 parameters, no annotations, no output schema), the description is incomplete. It lacks details on validation behavior, parameter usage, error handling, and return values. While concise, it doesn't provide enough context for effective tool invocation in this scenario.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate for undocumented parameters. It only mentions 'frames' and 'delegation chains,' which partially relates to the 'frames' parameter but ignores 'validationLevel,' 'strict,' and 'stopOnFirstError.' This adds minimal meaning beyond the schema, failing to address the coverage gap adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Validate multiple frames at once.' This specifies the verb (validate) and resource (frames) with a scope (multiple/batch). However, it doesn't distinguish this tool from its sibling 'ps_validate' (single vs. batch validation), which would require explicit comparison for a score of 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides implied usage guidance with 'Useful for validating delegation chains,' suggesting a specific context where batch validation is beneficial. However, it lacks explicit when-to-use rules, alternatives (e.g., vs. ps_validate for single frames), or exclusions, falling short of higher scores.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
The tools have clear purposes within their categories, such as 'ps_confidence_get' vs. 'ps_confidence_set', but there is overlap in some areas. For example, 'ps_security_gate' and 'ps_security_scan' both involve security scanning, and 'ps_execute' and 'ps_execute_batch' share similar execution functions, which could cause confusion for an agent. However, descriptions help differentiate them to some extent.
All tool names follow a consistent 'ps_' prefix with a verb_noun pattern, such as 'ps_config_get' and 'ps_symbol_list'. There are no deviations in naming conventions, making the set predictable and easy to parse for agents.
With 45 tools, the count is excessive for a single server, making it feel heavy and potentially overwhelming. While the server covers a broad domain (PromptSpeak governance), the high number suggests poor scoping, as many tools could be consolidated or split into more focused servers.
The tool set provides comprehensive coverage for the PromptSpeak domain, including configuration, execution, delegation, security, state management, and symbol handling. It supports full CRUD operations for symbols and other resources, with no apparent gaps that would hinder agent workflows.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
Human-in-the-loop review and approval for AI agents. Audit trail, approval policies, native MCP.
Zero-secret MCP gateway for AI agents: risk-scored, audited calls with human-in-the-loop approval.
MCP enforcement layer that intercepts AI agent actions and blocks rule violations before execution.
Pre-execution governance for AI agents. Deterministic PASS/FAIL/REVIEW verdicts, replayable proof.
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/chrbailey/promptspeak-mcp-server'
If you have feedback or need assistance with the MCP directory API, please join our Discord server
