Skip to main content
Glama

switch-trust-mcp

A local MCP server (stdio) that gives a coding agent read-only access to a Switch Trust backend's AI-SPM issues, inventory, guardrails, cost optimization and remediation guidance so it can fix the flagged code.

It is a thin, authenticated REST client over the Switch Trust API — no scanner runs locally and no backend changes are required. Everything valuable (AI-reasoned findings, rule-based issues, public-asset vuln enrichment, remediation guidance) is already computed server-side and served over the existing API / static assets.

Install & run

The idiomatic way is via the published wheel with uvx (no clone, no build step):

uvx switch-trust-mcp

Related MCP server: AgentLens MCP Server

Configuration

Set two environment variables:

Variable

Meaning

SWITCH_TRUST_API_KEY

Switch Trust API key (sk_...).

SWITCH_TRUST_INSTANCE

Switch Trust instance base URL, e.g. https://app.switchagents.ai.

The instance URL is validated against a fail-closed host allowlist before any credential is attached, and must be a bare origin (no path, query or credentials). Set it to the web app URL: API calls go to the same deployment's api. host automatically (https://app.switchagents.ai → https://api.switchagents.ai).

The tenant and workspace are auto-discovered from the API key at startup — you do not configure any IDs. A key is pinned to exactly one tenant + active workspace by the backend.

The API key is only ever sent in the Authorization: ApiKey ... header and is never logged.

Wiring into a coding agent

Add to your MCP client config (Claude Code .mcp.json, Cursor, etc.):

{
  "mcpServers": {
    "switch-trust-mcp": {
      "command": "uvx",
      "args": ["switch-trust-mcp"],
      "env": {
        "SWITCH_TRUST_API_KEY": "sk_...",
        "SWITCH_TRUST_INSTANCE": "https://app.switchagents.ai"
      }
    }
  }
}

Tools

Fix-focused

  • get_context — show the resolved instance / tenant / workspace.

  • list_issues(severity?, rule_id?, category?, search?, page_size?, cursor?) — list issues; paginated, page_size capped at 100.

  • get_issue(issue_id, page_size?, cursor?) — issue detail + assembled rule-level remediation guidance + a page of affected objects; objects is paginated the same way as list_issues.

  • get_finding_detail(issue_id, object_id, detail_id?) — per-finding file path, code snippet, evidence, and inline remediation.

  • find_issues_for_file(path, max_issues?) — issues whose findings reference a given working-tree file (client-side scan; matches by shared path suffix). The scan follows pagination but is bounded — max_issues is capped at 500 and the call at 300 backend requests — and returns a scan block reporting complete, issues_scanned, requests and errors. complete: false means the caps were hit and the file may have findings this result omits; treat it as "unknown", not as "clean", and narrow the search with list_issues filters instead.

  • get_remediation_guidance(rule_name) — rule-level remediation markdown.

Inventory browse

  • list_models | list_agents(name?, severity?, supplier?, library?, page_size?, cursor?)

  • list_mcp_servers(name?, severity?, library?, page_size?, cursor?)

  • list_tools(name?, severity?, page_size?, cursor?)

  • get_model | get_agent | get_tool | get_mcp_server(asset_id) — detail plus attached issues, related assets, and code locations.

Guardrails & LLM-interaction analytics

  • list_guardrail_policies(search?, page?, page_size?) — page-numbered (not cursor-based); page_size capped at 100.

  • get_guardrail_policy(policy_id) — policy detail: name, description, user/assistant detector configuration.

  • get_guardrail_interaction_counts | get_guardrail_outcomes | get_guardrail_categories(agent_id?) — guardrail-decision dashboards across LLM interactions, optionally scoped to one agent.

  • list_llm_interactions(search?, agent_id?, page_size?, cursor?) — prompt/ response pairs with guardrail outcomes; rows are trimmed (long text truncated, per-turn events omitted) for context budget. page_size capped at 100.

  • get_interaction(interaction_id) — one interaction's full detail.

  • list_llm_sessions(agent_id, sort?, sort_dir?, page_size?, cursor?) — grouped conversation turns for one agent (required). page_size capped at 100.

  • get_llm_session(session_id) — one session's aggregate cost/tokens/turns plus the full per-turn list.

ROI / cost-optimization

  • list_roi_agents(page?, page_size?) — per-agent LLM activity summaries (latest_interaction_time, interaction_count). page_size capped at 100.

  • list_roi_interactions(agent_id, since?, until?, page?, page_size?, order?) — one agent's raw LLM interaction rows (with computed cost_usd) — the source data the cost-optimization analyses read. agent_id required; since/until are RFC3339 timestamps. page_size capped at 100.

  • list_roi_analyses(agent_id?, analysis_type?, page?, page_size?, order?) — cost-optimization analysis results (findings + estimated $ savings), latest first. analysis_type is one of tool_bloat, model_optimizer, verbosity, context_cache_waste, retry_loop_waste. page_size capped at 100.

  • get_latest_roi_analyses(agent_id, analysis_types) — the latest finished result per type for one agent, i.e. its current recommendations. Both arguments are required; a type with no finished run is simply absent from the result.

  • get_roi_analysis(analysis_id) — one analysis result's full detail.

Red-teaming / eval

  • list_evaluations(search?, approach?, page?, page_size?, order?) — evaluation definitions (probes/metrics), including red-teaming adversarial probes; approach is probe or metric. Page-numbered; page_size capped at 100.

  • get_evaluation(evaluation_id) — full definition detail, including attack_techniques / adversarial_goals / num_prompts / max_turns for adversarial-probe evaluations.

  • list_agent_evaluations(agent_id?, evaluation_id?, search?, page?, page_size?, order?) — agent↔evaluation assignments, each with latest_run_status/progress/ summary and a score_trend (last 5 scores). Page-numbered; page_size capped at 100.

  • get_agent_evaluation(agent_evaluation_id) — one assignment's full detail.

  • get_agent_evaluation_summary(agent_id) — one agent's evaluation health rollup: health score, per-category OWASP coverage, and score trend.

  • list_evaluation_runs(evaluation_id?, agent_evaluation_id?, page?, page_size?, order?) — runs; status is queued/running/finished/error. Page-numbered; page_size capped at 100.

  • get_evaluation_run(evaluation_run_id) — one run's detail.

  • list_evaluation_run_results(evaluation_run_id, page?, page_size?, order?) — one run's per-prompt results; status is passed/failed/error/ scored. There is no single-result detail endpoint, so conversation transcripts are trimmed rather than dropped: every turn is kept but each turn's text is truncated to 300 characters. Page-numbered; page_size capped at 100.

  • get_agent_endpoint(agent_id) — an agent's red-teaming target-endpoint config (endpoint_url, endpoint_auth_type, model_type, config); secrets are redacted server-side and always come back empty.

Remediation guidance

Rule-level remediation markdown has no dedicated API — it is served by the instance's web server as static assets at {instance}/assets/docs-remediations/… (the same origin and files the web UI reads). This server fetches them at runtime over HTTP through the shared client and caches them per session; nothing is bundled into the package. When a rule has no docs (or the assets origin is unreachable), fall back to the per-finding remediation text from get_finding_detail.

For offline development or tests, set SWITCH_TRUST_REMEDIATION_DIR to a local directory laid out like the assets (rule-name-to-remediation-folder-map.json + per-rule folders); the loader then reads from disk and never touches the network.

Development

  • Layout: src/switch_trust_mcp/ (src/ layout), tests under test/.

  • Editable install: pip install -e '.[dev]', then run switch-trust-mcp.

  • Docs are fetched from your instance at runtime; for offline work point SWITCH_TRUST_REMEDIATION_DIR at a local copy of the docs directory.

Tests

pytest

Tests are hermetic (httpx MockTransport for the client and the remediation-docs fetch, a fake client + SWITCH_TRUST_REMEDIATION_DIR for the tools), so they need no network or live backend.

Available Tools

37 tools
find_issues_for_fileA

Find issues whose findings reference a given working-tree file.

Convenience for the edit loop: scans up to max_issues issues and returns the findings whose file path matches path (matched by shared path suffix, so an absolute or repo-relative path both work), each with its rule, severity, path, and inline remediation text. The backend cannot filter by file, so this is a client-side scan and may be slow for large workspaces.

The scan is bounded: max_issues is capped at 500 and the whole call at 300 backend requests. Check the returned scan.complete — when it is false the caps were hit and the file may have findings this result does not list, so do not read it as "no issues here". Narrow the search with list_issues filters and follow up on specific issues instead.

NameRequiredDescriptionDefault
pathYes
max_issuesNo

TDQS

A5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must disclose behavior and it does: client-side scan because the backend cannot filter by file, potential slowness on large workspaces, hard caps on issues and backend requests, and the meaning of `scan.complete` including the explicit warning not to read an incomplete result as 'no issues here'. It also names the returned fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core purpose is front-loaded in sentence one, and each following paragraph carries a distinct, non-redundant payload (matching semantics, performance/cap limits, and the completeness caveat). Sized appropriately for a tool whose limitations matter more than its happy path.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite having no output schema and no annotations, the description covers return contents (rule, severity, path, remediation text, scan.complete), the failure/limitation mode, and the fallback strategy. Nothing an agent needs to call or interpret this tool correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description carries the burden and does so: it explains that `path` is matched by shared path suffix so absolute or repo-relative paths both work, and that `max_issues` is capped at 500 with the whole call bounded at 300 backend requests. This is real semantics far beyond the bare schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (find issues whose findings reference a given working-tree file) plus the matching scope. It is clearly distinguishable from siblings like list_issues and get_finding_detail, which operate on the issue/finding hierarchy rather than by file.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly frames the tool as a 'convenience for the edit loop,' and names the alternative path ('narrow the search with list_issues filters and follow up on specific issues instead') for the case where the bounded scan is insufficient. Both when-to-use and when-to-switch are stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_agentB

Get an agent's detail plus its issues, MCP servers, and code locations.

NameRequiredDescriptionDefault
asset_idYes

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It helpfully discloses that the response aggregates related entities (issues, MCP servers, code locations), which is real behavioral context, but it says nothing about read-only semantics, permissions, pagination, or how large the bundled payload may be.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, front-loaded sentence with no wasted words; the resource and its bundled contents are stated immediately.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and no annotations, the description must stand in for both. It compensates by naming the returned collections, but the lack of any asset_id semantics and the absence of usage context leave gaps for a detail-fetch tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for the single asset_id parameter. The description implies the ID identifies an agent, but adds no format, source, or constraint details to compensate for the schema gap, leaving the agent to infer what a valid asset_id looks like.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb ('Get') and resource ('an agent's detail') and enumerates the related data bundled in (issues, MCP servers, code locations). It is clearly distinct from the plural list_agents sibling, though it never names that alternative explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use statement, no prerequisite for asset_id, and no routing to alternatives like list_agents for enumeration or get_issue for the individual issues it returns. Usage is only loosely implied by the word 'detail'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_agent_endpointA

Get one agent's target-endpoint config for red-teaming (endpoint_url, endpoint_auth_type, model_type, config). Secrets (endpoint_credential, endpoint_headers) are redacted server-side and always come back empty.

NameRequiredDescriptionDefault
agent_idYes

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden, and it does disclose a genuinely non-obvious behavior: secrets (endpoint_credential, endpoint_headers) are redacted server-side and always return empty. That prevents an agent misinterpreting empty fields as a config error. It does not state read-only status, permission requirements, or not-found behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly written sentences with zero filler. The core purpose and the returned field list are front-loaded, and the redaction caveat follows immediately where it is most useful.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and no annotations, the description carries the disclosure load and does so well: it names the return fields and the redaction behavior. For a one-parameter getter this is nearly sufficient; only error/not-found semantics are unaddressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for the single agent_id parameter, so the description should compensate. It conveys that exactly one agent is targeted, but adds no detail on agent_id format, source, or validation; the name is largely self-explanatory, keeping this at a baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb+resource ('Get one agent's target-endpoint config') and enumerates the returned fields (endpoint_url, endpoint_auth_type, model_type, config), which distinguishes it from the generic get_agent sibling. It stops short of explicitly contrasting itself with get_agent or get_model, but the purpose is unmistakable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: an agent can infer you call this when you need the red-teaming endpoint configuration for a known agent. There is no explicit when-to-use, when-not-to-use, or named alternative (e.g., get_agent for general agent metadata).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_agent_evaluationC

Get one agent<->evaluation assignment's full detail.

NameRequiredDescriptionDefault
agent_evaluation_idYes

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden, and it says almost nothing beyond 'full detail'. It does not state whether the result is read-only, what 'full detail' contains versus the summary tool, or what happens for an unknown id.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single short sentence that leads with the verb and resource, with no filler. The '<->' notation is slightly awkward but costs nothing in length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a detail-fetching tool with no output schema, the agent cannot learn what 'full detail' returns, nor how to obtain the required id. Given the crowded sibling set of near-identical get_* tools, this leaves real gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter exists and schema description coverage is 0%, so the description is the only clue; calling the target an 'agent<->evaluation assignment' hints at what the id identifies. However, it gives no format, source, or discovery hint for agent_evaluation_id.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb (Get) and resource (agent<->evaluation assignment) with a stated scope of 'full detail', which separates it from the list variant. It does not explicitly distinguish itself from get_agent_evaluation_summary, which is the closest sibling and the likeliest confusion point.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance, no mention of the alternative get_agent_evaluation_summary, and no indication of where the id comes from (presumably list_agent_evaluations). The singular 'one' only weakly implies a lookup use case.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_agent_evaluation_summaryB

Get one agent's evaluation health rollup: health (score/previous_score), coverage (per-category pass/at-risk/not-run counts), and score trend over time. agent_id is required.

NameRequiredDescriptionDefault
agent_idYes

TDQS

B3.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, so the description carries the full burden, and it does disclose the shape of the returned data (score/previous_score, per-category pass/at-risk/not-run counts, trend over time). It is clearly a read-only retrieval, but it omits any note on authorization scope, aggregation window, or freshness.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences, front-loading the resource and the return contents before the parameter note. The trailing 'agent_id is required' sentence is redundant with the schema and slightly dilutes the front-loading.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and no annotations, the description usefully compensates by describing the returned fields (health, coverage, trend), which is the main thing an agent needs here. It falls short only on parameter sourcing detail for a single required id.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for the single agent_id parameter, and the description only restates that it is required. It adds no meaning about the id format, where to obtain it (e.g., list_agents), or whether it accepts a slug versus a UUID.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Get) and resource (one agent's evaluation health rollup), and enumerates what the rollup contains (health, coverage, trend). The word 'rollup' implicitly distinguishes it from the detail-oriented siblings get_agent_evaluation/list_agent_evaluations, but no sibling is named.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Only guidance is 'agent_id is required', which is already in the schema. There is no indication of when to use this summary rollup versus get_agent_evaluation, list_agent_evaluations, or get_agent_evaluation, all of which live in the sibling set.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_contextA

Show the resolved backend scope (instance, tenant, workspace).

Useful to confirm the API key maps to the workspace you expect before querying issues.

NameRequiredDescriptionDefault

No parameters

Output Schema

NameRequiredDescription

No output parameters

TDQS

A4.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description bears the full burden. The verb 'Show' implies a read-only operation, but the description does not explicitly confirm it is safe, non-destructive, or whether any permissions are required. It adequately conveys the basic behavior but leaves safety traits implicit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with the core purpose. Every sentence earns its place, and there is no redundant or vague language.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple zero-parameter read tool with an output schema, the description is complete. It explains what the tool returns (resolved scope) and why to call it, and the output schema covers return values in detail.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so the baseline score of 4 applies. The description need not explain parameter syntax, and the input schema is empty.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Show') and resource ('resolved backend scope'), and enumerates the scope components (instance, tenant, workspace). This clearly distinguishes it from sibling tools, which all deal with issues, guardrails, agents, or evaluations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly says when to use it: 'before querying issues' to confirm the API key maps to the expected workspace. No alternatives are named because none exist, but the context is clear. It lacks any explicit 'when not to use' guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_evaluationB

Get one evaluation definition's full detail, including its config (attack_techniques/adversarial_goals/num_prompts/max_turns for adversarial_probe evaluations).

NameRequiredDescriptionDefault
evaluation_idYes

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. It usefully discloses what the returned config contains (attack_techniques/adversarial_goals/num_prompts/max_turns for adversarial_probe evaluations), which exceeds structured data, but it says nothing about permissions, errors, or missing-definition behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single tight sentence, front-loaded with the verb and resource. It is appropriately sized and free of filler, though the parenthetical detail list slightly interrupts the flow.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, so the description should describe what is returned; it does partially by naming config fields for adversarial_probe evaluations, but leaves the rest of the definition body, error cases, and ID sourcing unaddressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and the single parameter is undocumented. 'evaluation_id' is largely self-explanatory by name, and the description ties the retrieved entity back to 'one evaluation definition', but no format, source, or ID-example is supplied.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Get') and resource ('one evaluation definition') with the scope of detail ('full detail'). It implicitly distinguishes itself from list_evaluations by fetching a single definition, but it doesn't explicitly differentiate itself from the several other evaluation-related getter siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: fetching a single definition's detail versus listing. There is no explicit when-to-use, no mention of prerequisites, and no routing away from siblings like get_agent_evaluation or get_evaluation_run.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_evaluation_runB

Get one evaluation run's detail: status, progress, summary, timestamps.

NameRequiredDescriptionDefault
evaluation_run_idYes

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It usefully discloses what the response contains (status, progress, summary, timestamps) in lieu of an output schema, but says nothing about read-only nature, not-found/error behavior, or permissions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One sentence, front-loaded with the action and resource, with no filler; the return-field list is compact. It is terse to the point of omitting usage context, but nothing in it is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter read tool with no output schema, the description covers both intent and the returned payload, which is the main thing an agent needs. Only error/not-found handling and id-source guidance are missing, which is a minor gap at this complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter evaluation_run_id has 0% schema description coverage and the description adds no meaning of its own — no format, no source of the id, no distinction between run id and evaluation id. The self-evident name is the only guide the agent gets.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb and resource ('Get one evaluation run's detail') and enumerates the returned fields (status, progress, summary, timestamps), which cleanly separates it from list_evaluation_runs and list_evaluation_run_results. It does not name those siblings explicitly, so it stops short of full differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no statement of when to use this tool versus list_evaluation_runs, list_evaluation_run_results, or get_evaluation, and no prerequisites or exclusions are given. The singular phrasing only implicitly signals 'one run by id'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_finding_detailA

Get per-finding detail: file path, code snippet, evidence, and remediation.

Provide the issue_id and object_id (from get_issue's objects). Omit detail_id to list all findings for that object, or pass one to fetch a single finding.

NameRequiredDescriptionDefault
issue_idYes
detail_idNo
object_idYes

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It does disclose the dual-mode behavior (list vs. single fetch) and names returned fields, but says nothing about pagination, result size, ordering, or permission requirements for what is effectively a read/list operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the payload contents, then follows with the required inputs and the detail_id branch. Two tight sentences with no filler; only minor trimming would improve it.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 3-parameter read tool with no annotations and no output schema, the description covers inputs, provenance, and the list-vs-single behavior adequately. It could say more about the shape of a multi-finding response, but nothing critical for correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description compensates well: issue_id and object_id are explained with their source, and detail_id's optional semantics (omit to list all, pass to select one) are made explicit. The provenance note 'from get_issue's objects' adds real meaning beyond the bare schema types.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Get per-finding detail') and enumerates the payload (file path, code snippet, evidence, remediation). It also positions itself relative to the get_issue sibling by sourcing object_id from get_issue's objects, so the agent can distinguish it without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear operative guidance: supply issue_id and object_id, omit detail_id to list all findings for the object, or pass one to fetch a single finding. It also tells the agent where object_id comes from. It does not, however, contrast against related siblings like find_issues_for_file or get_remediation_guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_guardrail_categoriesC

Guardrail-detector-category counts across LLM interactions, optionally scoped to one agent.

NameRequiredDescriptionDefault
agent_idNo

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. 'Counts' implies a read-only aggregation, but nothing is said about the time window, whether results are pre-aggregated or live, rate limits, or return shape for a tool with no output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One sentence, front-loaded with the resource and the aggregation, with the optional scoping trailing. No waste, though it is arguably under-specified rather than concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no annotations and no output schema, the description should explain the time window and the returned count structure, neither of which is covered. It is minimally adequate for a one-parameter aggregation but leaves key operational context to inference.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for the single agent_id parameter, so the description must compensate. It does add meaning by explaining the parameter is an optional scope ('optionally scoped to one agent'), clarifying it filters rather than requires, but gives no format or default behavior detail.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific resource and measurement ('guardrail-detector-category counts') plus the population it covers ('across LLM interactions'), so the agent knows this is an aggregation query. It does not distinguish itself from the near-identical sibling get_guardrail_interaction_counts, leaving ambiguity about which counting tool to pick.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance and no named alternative, despite several overlapping guardrail siblings (get_guardrail_outcomes, get_guardrail_interaction_counts). The only hint is 'optionally scoped to one agent', which describes the parameter rather than the use case.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_guardrail_interaction_countsB

Guardrail-decision counts (ok/blocked/redacted/alerted/errored) across LLM interactions, optionally scoped to one agent.

NameRequiredDescriptionDefault
agent_idNo

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations, so the description carries the full burden. It usefully discloses what is computed (per-category decision counts) rather than raw records, but says nothing about time range, default scope, aggregation window, or whether it is all-time data.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One compact sentence with the resource and metric front-loaded and the optional scope trailing. Every element earns its place; nothing is padded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description does the work of listing returned categories, which is the key missing piece. However it omits the aggregation scope (time window, session vs interaction grouping), leaving the agent unsure what set of data is being counted.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single agent_id parameter has 0% schema description coverage, so the description must compensate. 'Optionally scoped to one agent' does convey its meaning and optional nature, but adds no format, ID-type, or default behavior detail.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (counts) and resource (guardrail decisions across LLM interactions) and enumerates the counted categories (ok/blocked/redacted/alerted/errored). This separates it from list_llm_interactions and get_guardrail_outcomes without naming them explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The only usage signal is 'optionally scoped to one agent', which is a filter hint rather than guidance. It never says when to reach for this aggregate versus get_guardrail_outcomes or get_guardrail_categories, nor any prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_guardrail_outcomesA

Guardrail outcome totals across LLM interactions, optionally scoped to one agent. Same underlying data as get_guardrail_interaction_counts, without the per-category breakdown.

NameRequiredDescriptionDefault
agent_idNo

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. 'Outcome totals' implies an aggregate read-only operation and the data relationship to a sibling is disclosed, but there is no mention of return shape, time-range defaults, pagination, or whether counts are cumulative or windowed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences with zero filler; the scope statement comes first and the sibling differentiation second. Every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a low-complexity aggregate tool with no annotations and no output schema, the description is serviceable but thin: it never indicates what a 'total' looks like in the response, over what time window it is computed, or how results are ordered/limited. An agent could call it correctly but cannot predict the payload.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for the single undocumented agent_id parameter. It does explain that the parameter is optional and that supplying it scopes results to one agent, though it adds no format/details (e.g., ID style) beyond that.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb-plus-resource ('Guardrail outcome totals across LLM interactions') with an explicit scope modifier. It directly names and differentiates itself from the sibling get_guardrail_interaction_counts by stating what it omits ('without the per-category breakdown').

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The comparison to get_guardrail_interaction_counts gives clear routing context: pick this one when you don't need category-level granularity. There is no explicit 'when not to use' or prerequisite guidance, but the alternative is named and the trade-off is spelled out.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_guardrail_policyB

Get one guardrail policy's detail: name, description, and the user/assistant detector configuration.

NameRequiredDescriptionDefault
policy_idYes

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It usefully discloses the shape of the returned content (name, description, detector config), which lets an agent know this is a read. However, it says nothing about permissions, error behavior for an unknown id, or any side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence naming the action and the returned fields, with no filler. Efficient, though the field list slightly truncates what an agent might want to know.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter getter with no output schema and no annotations, the description covers the return content adequately, but it leaves the parameter's semantics and any read-safety/error expectations unstated. It is minimally sufficient rather than complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and the single parameter policy_id has no type detail beyond 'string'. The description only implies that the id identifies one policy; it adds no information about id format, source, or how to obtain a valid value, so it does not compensate for the coverage gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Get) and resource (one guardrail policy) and enumerates what is returned: name, description, user/assistant detector configuration. 'One' implicitly distinguishes it from list_guardrail_policies, but no sibling is named explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied by the singular 'one' and the required policy_id, suggesting retrieval of a single policy versus the listing sibling. There is no explicit when-to-use statement, no prerequisites, and no named alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_interactionC

Get one LLM interaction's full detail (prompt, response, guardrail events).

NameRequiredDescriptionDefault
interaction_idYes

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. It does disclose the returned payload shape (prompt, response, guardrail events), which is genuinely useful, but says nothing about permissions, error behavior for unknown IDs, or whether the read is safe/read-only.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler or repetition. It is efficient, though its brevity contributes to the missing guidance noted elsewhere.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple single-parameter read tool with no output schema and no annotations, the description is minimally adequate: it tells the agent what comes back. It omits how to obtain a valid ID and any failure semantics, which are the remaining gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is one parameter at 0% schema description coverage, so the description must compensate. It only implies that interaction_id identifies an interaction; it does not say the expected format or that IDs come from list_llm_interactions, leaving the parameter effectively undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Get') and resource ('one LLM interaction'), and enumerates what the detail contains (prompt, response, guardrail events). It implicitly contrasts with the sibling list_llm_interactions by specifying 'one', but never names an alternative, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no when-to-use guidance, no prerequisite (e.g., obtaining the interaction_id), and no exclusions relative to get_llm_session, list_llm_interactions, or get_guardrail_outcomes. Usage is only inferable from the name.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_issueA

Get full detail for one issue plus rule-level remediation guidance.

Returns the issue record, the assembled remediation guidance markdown for the issue's rule (when available), and a page of affected objects (assets). The objects list is paginated the same way as list_issues: page_size is capped at 100, and objects.cursor carries forward to fetch the next page. Use get_finding_detail for the code location of a specific occurrence.

NameRequiredDescriptionDefault
cursorNo
issue_idYes
page_sizeNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does well: it discloses the three-part return (issue record, remediation markdown, affected-objects page), notes remediation is conditional ('when available'), and documents pagination mechanics including the page_size cap of 100 and cursor carry-forward. It never says this is a pure read operation, but 'Get' plus the read-shaped payload makes that clear enough.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences, purpose front-loaded, followed by return shape, pagination behavior, and the sibling redirect. Each sentence adds distinct information; only the pagination sentence is slightly dense but it is justified by the missing schema docs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema and no annotations, yet the description enumerates the returned parts and the pagination contract, which is the critical missing structured information. What remains thin is failure/empty behavior (e.g. an issue with no rule guidance, or a invalid issue_id), but that is a minor gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate. It meaningfully explains page_size (capped at 100) and cursor (carries forward to the next page), covering two of three params; issue_id is left implicit, though its meaning is unambiguous from context. Adequate but not complete compensation for a 0%-coverage schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Get full detail for one issue') and immediately scopes the payload ('plus rule-level remediation guidance'), which separates it from list_issues and get_remediation_guidance. It also names get_finding_detail as the tool for a narrower concern, so an agent can differentiate it from siblings without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit routing rule: 'Use get_finding_detail for the code location of a specific occurrence,' and implies the singular-vs-list split against list_issues. It stops short of stating when this tool should be preferred over the standalone get_remediation_guidance sibling, which is the one ambiguous boundary left.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_latest_roi_analysesA

Get the latest finished ROI analysis result per type for one agent — the agent's current cost-optimization recommendations.

Both agent_id and analysis_types are required. Returns a dict keyed by analysis_type; a type with no finished run yet is simply absent from the result rather than erroring.

NameRequiredDescriptionDefault
agent_idYes
analysis_typesYes

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden and does well: it discloses the 'finished only' filter, the per-type keying, and the non-erroring absence semantics for missing types. It doesn't state read-only/inference-cost implications explicitly, but 'Get' plus the absent-rather-than-error rule gives solid behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences, front-loaded with the core action and scope, followed by a required-params note and the return-shape caveat. No filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description usefully explains the return shape (dict keyed by analysis_type, missing types absent). Complete enough to call correctly, though valid analysis_type values remain unstated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It confirms both params are required and clarifies that analysis_types drives the per-type keying of the result, but it does not enumerate valid analysis_type values or specify the agent_id format.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('get the latest finished ROI analysis result per type for one agent') and adds the semantic gloss 'the agent's current cost-optimization recommendations.' This distinguishes it clearly from siblings like get_roi_analysis (single analysis) and list_roi_analyses.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Says both parameters are required and explains that a type with no finished run is simply absent from the result. However, it never tells the agent when to prefer this over get_roi_analysis or list_roi_analyses, so selection guidance is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_llm_sessionB

Get one LLM session's detail: aggregate cost/tokens/turns and the full per-turn list.

NameRequiredDescriptionDefault
session_idYes

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full disclosure burden, and it does add useful content by describing the returned data (aggregate cost/tokens/turns and the per-turn list) in the absence of an output schema. However, it says nothing about permissions, error behavior, or whether the per-turn list is paginated or bounded.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with zero filler that immediately names the resource and summarizes its contents. Every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter read tool with no output schema, the description usefully summarizes what the response contains, which is exactly the burden it must carry here. It is only slightly short on the shape/limits of the per-turn list and on the session_id prerequisite.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and the single parameter session_id is never mentioned in the description. The schema only provides the name and type, so the description does not compensate for the coverage gap; the parameter's meaning is inferred solely from the tool name.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ("Get") and resource ("one LLM session's detail") and enumerates the payload (aggregate cost/tokens/turns plus the full per-turn list). It is clearly distinguishable from the sibling list_llm_sessions by the singular "one ... session" framing, though it does not name that sibling explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: "one LLM session" signals this is the single-record retrieval counterpart to list_llm_sessions, but the description never states when to prefer it over the list tool or what prerequisite (a known session_id) is needed. No exclusions or alternatives are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_mcp_serverB

Get an MCP server's detail plus its issues and code locations.

Agents using this server are embedded in asset directly (the backend has no standalone "agents" edge for the MCP-server scope).

NameRequiredDescriptionDefault
asset_idYes

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it does disclose the return shape (detail, issues, code locations), which is valuable given there is no output schema. However, it says nothing about permissions, error behavior, or cost, and the note about agents being 'embedded in asset' is a data-model aside that is more confusing than clarifying.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The first sentence is well front-loaded and earns its place. The second sentence leaks RST double-backtick markup and spends its length on an internal backend detail that does not help an agent decide or invoke, weakening the structure.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple single-parameter read tool the definition covers the essential purpose and return contents, so it is close to adequate. The unconsummated asset_id semantics and the absence of any usage contrast with the lookalike list tool leave a real gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is one required parameter at 0% schema coverage, and the description never states what asset_id identifies or its format. The oblique reference to `asset` as the MCP-server scope hints at a relationship but does not give the agent a usable definition of the identifier.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence gives a specific verb and resource ('Get an MCP server's detail') and even enumerates the payload ('plus its issues and code locations'), which cleanly separates it from the sibling list_mcp_servers. It stops short of explicitly naming the plural list tool as the alternative, so it is clear but not fully self-differentiating.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the singular 'get' against a known MCP server implies fetching one server's detail rather than browsing. There is no explicit when-to-use/when-not statement and no mention of the list_mcp_servers alternative. The second sentence addresses backend data modeling, not invocation guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_modelB

Get a model's detail plus its attached issues and code locations.

NameRequiredDescriptionDefault
asset_idYes

TDQS

B3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are supplied, so the description carries the full behavioral burden. It does disclose that the response aggregates the model detail plus attached issues and code locations, which is meaningful return-shape information with no output schema present, but it says nothing about permissions, failure modes, or whether the attached data is truncated.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single tight sentence with the verb and resource front-loaded and zero filler. Nothing is wasted, though there is little structure to speak of given the length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter read tool the description covers the essential 'what you get back', which helps since there is no output schema or annotation coverage. It falls short on how to supply asset_id and on any behavioral caveats, leaving real gaps for an unannotated tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and the description never mentions asset_id. The name alone implies an identifier, but it is unclear whether this is a model ID, a repository asset, or something else, and no format or sourcing hint is given.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Get a model's detail') and adds the scope of attached data (issues and code locations), which distinguishes it from the sibling list_models. It does not explicitly name the sibling alternative, but the resource is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance, no prerequisites, and no mention of list_models or any alternative. The agent must infer that this is the single-model counterpart to list_models purely from the names.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_remediation_guidanceA

Get the rule-level remediation guidance markdown for a rule name.

The rule_name matches the value shown on issues (see get_issue). Returns None guidance for rules without docs — use the per-finding remediation from get_finding_detail in that case.

The guidance is wrapped in the procedure for choosing and applying a remediation; get_issue returns the same docs unwrapped, for browsing.

NameRequiredDescriptionDefault
rule_nameYes

Output Schema

NameRequiredDescription

No output parameters

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full behavioral burden and does much of it: it discloses the None-guidance edge case, the wrapping-vs-unwrapping behavior relative to get_issue, and the output being markdown. It does not state read-only/permission characteristics, but 'Get' plus the output schema mitigate that gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action, then edge cases and disambiguation. Three compact sentences with no filler, though the wrapping/unwrapping sentence is slightly convoluted and could be tightened.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-param read tool with an output schema, the description supplies the sourcing of the key parameter, the empty-result edge case, and alternative-tool routing. Nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% for the single required param, so the description must compensate. It does by explaining that rule_name matches the value shown on issues and points to get_issue for sourcing it, adding genuine meaning beyond the bare 'Rule Name' title.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Get') and resource ('rule-level remediation guidance markdown for a rule name'), and distinguishes itself from siblings by contrasting with get_finding_detail and get_issue. An agent can immediately tell what this returns without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit routing: it names the fallback (get_finding_detail) for rules without docs, and explains that get_issue returns the same docs unwrapped for browsing. This is a clear when-to-use-this-vs-alternatives statement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_roi_analysisA

Get one ROI analysis result by ID (findings, estimated $ savings, status, timestamps).

NameRequiredDescriptionDefault
analysis_idYes

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. 'Get' and 'result' strongly imply a read-only lookup, and the parenthetical discloses the return fields (findings, estimated $ savings, status, timestamps), which is genuinely useful. It says nothing about behavior for an unknown ID (error vs empty), permissions, or whether the analysis data is ever mutated, leaving real gaps for an annotation-free tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One compact sentence, front-loaded with the action and resource, with the return contents injected as a parenthetical rather than a second sentence. Nothing is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, and the description compensates by naming the returned fields (findings, estimated $ savings, status, timestamps). For a single-record lookup, that is nearly everything an agent needs; only error/missing-ID behavior is absent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must carry the parameter. It does tell the agent that the ID identifies a specific ROI analysis result, but adds no format, source, or example for the analysis_id value, and the schema gives only a bare string with no title detail.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb and resource: 'Get one ROI analysis result by ID', with the parenthetical enumerating the payload contents. 'One ... by ID' implicitly separates it from listing siblings such as list_roi_analyses and get_latest_roi_analyses, though it never names them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by 'by ID' – a caller who already has an analysis_id fetches that record. There is no explicit statement of when to prefer this over list_roi_analyses or get_latest_roi_analyses, and no prerequisites or failure behavior mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_toolB

Get a tool's detail plus its agents, issues, and code locations.

NameRequiredDescriptionDefault
asset_idYes

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided so the description carries the full behavioral burden. It does disclose that the call expands into related collections (agents, issues, code locations), which is useful added context, but says nothing about permissions, read-only safety, or result limits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single compact sentence with the primary purpose front-loaded and the expanded returns trailing. No wasted words, though it is arguably under-specified rather than deliberately concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no annotations, no output schema, and an undocumented required parameter, the description covers return scope but leaves safety profile and parameter meaning unexplained. Adequate for a simple one-parameter read but with clear gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for the single parameter and the description gives no indication of what asset_id refers to (tool id? asset identifier format?). The agent must infer that the identifier is a tool ID from the tool name alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb ('Get') plus resource ('a tool's detail') and it enumerates the associated data returned (agents, issues, code locations). It is clearly distinguishable from the plural sibling list_tools, though it does not name that sibling explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied by the singular 'get' naming and the required identifier; there is no statement of when to prefer this over list_tools, list_agents, or get_agent, and no prerequisites or exclusions are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_agent_evaluationsA

List agent<->evaluation assignments with their latest-run status.

Filters: agent_id, evaluation_id, search (substring match on name). Each row includes evaluation_name, latest_run_status/progress/summary, and a score_trend (last 5 scores). Page-numbered; page_size is capped at 100. Use get_agent_evaluation for full detail, get_agent_evaluation_summary for the OWASP-coverage/health rollup across all of one agent's evaluations.

NameRequiredDescriptionDefault
pageNo
orderNo
searchNo
agent_idNo
page_sizeNo
evaluation_idNo

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and largely meets it: it discloses the row shape (evaluation_name, latest_run_status/progress/summary, score_trend of last 5), the pagination model ('Page-numbered') and a hard limit ('page_size is capped at 100'). It does not state permission/auth requirements or whether an out-of-range page errors or returns empty, which keeps it out of the top band.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the purpose sentence, then filters, then return shape, then the alternative-tool routing. Every sentence either adds a filter contract, a return contract, or a routing rule; nothing is padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema and no annotations, so the description has to supply the return contract and constraints, and it does (row fields, trend window, page cap). The only material omission for a 6-param tool is the semantics of 'order', plus the absence of any default/range hint for page_size beyond the cap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, and it does for five of six params: search is defined as substring match on name, agent_id/evaluation_id as filters, page/page_size as page-numbered with a 100 cap. The 'order' parameter is never explained (no direction, no sort field, no accepted values), leaving one blind spot.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('List agent<->evaluation assignments') plus the payload attribute ('with their latest-run status'), so the agent knows exactly what comes back. It further distinguishes itself from siblings by naming get_agent_evaluation (full detail) and get_agent_evaluation_summary (rollup), which no other tool in the list provides.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly routes the agent: 'Use get_agent_evaluation for full detail, get_agent_evaluation_summary for the OWASP-coverage/health rollup across all of one agent's evaluations.' The list-vs-detail condition is stated, not inferred, and the filter vocabulary (agent_id, evaluation_id, search) tells the agent when this tool is the right entry point.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_agentsB

List inventoried AI agents. Filters: name, severity, supplier, library.

page_size is capped at 100; page with the returned cursor for more.

NameRequiredDescriptionDefault
nameNo
cursorNo
libraryNo
severityNo
supplierNo
page_sizeNo

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry the behavioral burden, and it does disclose real operational context: page_size is capped at 100 and results are cursor-paged. It says nothing about read-only nature, permissions, or what 'inventoried' scope excludes, so coverage is partial.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences: purpose plus filters first, pagination constraint second. Front-loaded and free of padding, though the pagination note could be tightened.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a six-parameter list tool with no annotations, no output schema, and 0% schema coverage, the description covers filters and paging adequately. It leaves gaps on filter value semantics and scope of the inventory, so it is viable but not complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description has to compensate. It names the four filter parameters (name, severity, supplier, library) and explains cursor and page_size, effectively touching all six, but adds no value semantics (e.g., allowed severity values) beyond the schema's own titles.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('List inventoried AI agents'), so the agent knows this enumerates agents. It does not distinguish itself from siblings like get_agent or the similarly-named list_roi_agents, but the basic purpose is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for enumerating agents and lists the supported filters, which helps the agent construct a query. It gives no explicit when-to-use guidance or contrast with get_agent/list_roi_agents, so usage is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_evaluation_run_resultsA

List one run's per-prompt results (score, status, conversation transcript).

status is one of passed/failed/error/scored. There is no separate detail endpoint for a single result, so conversation turns are trimmed here for context budget rather than dropped: every turn is kept (turn sequence matters for multi-turn attacks) but each turn's text is truncated to 300 chars. For full-fidelity transcript review, use the product UI. Page-numbered; page_size is capped at 100.

NameRequiredDescriptionDefault
pageNo
orderNo
page_sizeNo
evaluation_run_idYes

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations the description carries the full behavioral burden, and it does substantial work: it enumerates the status values (passed/failed/error/scored), explains that turns are truncated to 300 chars rather than dropped and why, and states that paging is page-numbered with page_size capped at 100. It stops short of permission requirements, ordering behavior, or response shape, which keeps it out of the top band.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded in the first clause, followed by status values and then the truncation caveat; every sentence carries information. The truncation sentence is slightly long but the added detail (why turns are kept) is genuinely useful rather than filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a paginated list tool with no annotations and no output schema, the description covers the agent's main risks: documented status values, the truncation trade-off, the absence of a detail endpoint, and the pagination ceiling. Only the undefined `order` parameter leaves a real gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for all four parameters. It clarifies page_size (capped at 100) and that results are page-numbered, but says nothing about the `order` parameter's accepted values or about evaluation_run_id's format, leaving two parameters semantically undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('List one run's per-prompt results') and enumerates the returned fields (score, status, conversation transcript). It is clearly distinguishable from the sibling get_evaluation_run (run-level) and get_finding_detail without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a clear routing condition: there is no per-result detail endpoint, so this is the only way to get the transcript, and for full-fidelity review the agent should use the product UI instead. It does not explicitly contrast with get_evaluation_run or state when NOT to list results (e.g., vs. get_agent_evaluation_summary), but the primary use case and its limitation are well signposted.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_evaluation_runsA

List evaluation runs. Filters: evaluation_id, agent_evaluation_id.

status is one of queued/running/finished/error. Page-numbered; page_size is capped at 100. Use get_evaluation_run for one run's full detail and list_evaluation_run_results for its per-prompt results.

NameRequiredDescriptionDefault
pageNo
orderNo
page_sizeNo
evaluation_idNo
agent_evaluation_idNo

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it does add behavioral facts: the status enum values (queued/running/finished/error) and the pagination contract ('Page-numbered; page_size is capped at 100'). It leaves auth/read-only and default sort behavior implicit, which keeps it from a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four tight sentences, front-loaded with purpose, then filters, status values, pagination, and routing. Every sentence adds information and none restates the title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a paginated list tool with no annotations and no output schema, the description covers the essentials an agent needs: filters, status values, page mechanics, and sibling routing. It omits any indication of returned fields or default sort, a minor gap given no output schema is available.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate. It labels evaluation_id and agent_evaluation_id as 'Filters' and documents both pagination params (page-numbered, page_size cap), covering four of five parameters; only the 'order' parameter's semantics are left unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb+resource (list evaluation runs) and immediately distinguishes scope via its filters (evaluation_id, agent_evaluation_id) and the enumerated status values. An agent can tell it apart from get_evaluation_run and list_evaluation_run_results without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly routes the agent to alternatives with the selecting condition: 'Use get_evaluation_run for one run's full detail and list_evaluation_run_results for its per-prompt results.' The filtering and status context further clarifies when this list tool is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_evaluationsA

List evaluation definitions (probes/metrics), including red-teaming adversarial probes.

Filters: search (substring match on name), approach ('probe' or 'metric'). Attack-technique/goal detail for adversarial_probe evaluations lives in each item's config. Page-numbered; page_size is capped at 100. Use get_evaluation for full config detail.

NameRequiredDescriptionDefault
pageNo
orderNo
searchNo
approachNo
page_sizeNo

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden and does so reasonably: it discloses page-numbered pagination, a page_size cap of 100, and that attack-technique/goal detail is nested in each item's config. It doesn't mention auth requirements or the shape of returned items, keeping it from a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core purpose, then filters, then behavioral notes, then a sibling pointer. Sentences are dense and largely earn their place, with only minor redundancy around the adversarial-probe detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema and no annotations, so the description is the only guide. It covers purpose, two filters, and pagination limits, but leaves the 'order' parameter undocumented and doesn't hint at returned fields, leaving the agent partially equipped.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate. It documents 'search' (substring match on name), 'approach' (values 'probe'/'metric'), and the page_size cap, but leaves 'page' and 'order' entirely unexplained — a meaningful gap given five params.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource: 'List evaluation definitions (probes/metrics), including red-teaming adversarial probes.' It clearly conveys scope, and the closing pointer to get_evaluation signals the relationship to a sibling tool, though the differentiation is more about depth than a hard routing rule.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It names the available filters and their semantics, and explicitly says to 'Use get_evaluation for full config detail,' giving an alternative for the deeper-lookup case. It lacks explicit when-not-to-use guidance, so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_guardrail_policiesA

List guardrail policies (input/output detector configurations).

Filters: search (substring match on policy name). Page-numbered, not cursor-based — pass the next page number to page through results. page_size is capped at 100. Use get_guardrail_policy for full detail.

NameRequiredDescriptionDefault
pageNo
searchNo
page_sizeNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the behavioral burden. It discloses non-obvious traits: page-numbered rather than cursor-based pagination, page_size capped at 100, and search as a substring match on policy name. It does not state read-only status or return shape, but 'List' strongly implies a safe read.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded: purpose first, then filters, pagination mechanics, and the alternative tool. Every sentence earns its place with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple three-parameter list tool with no output schema and no annotations, the description covers purpose, filtering, pagination, cap, and the sibling retrieval tool. It could mention the return shape or required permissions, but it is otherwise complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It explains all three parameters meaningfully: search is a substring match on policy name, page is a page number to advance through results, and page_size is capped at 100. Defaults are left to the schema, which is acceptable.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('List guardrail policies') and clarifies what those policies are ('input/output detector configurations'). It also names the sibling tool get_guardrail_policy for full detail, letting an agent distinguish listing from retrieval without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly directs agents to get_guardrail_policy for full detail and explains filtering via search plus page-number pagination. It lacks an explicit when-not-to-use statement, but the alternative and paging conditions are clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_issuesA

List AI-SPM issues for the workspace.

Filters: severity (CRITICAL/HIGH/MEDIUM/LOW/INFORMATIONAL), rule_id, category (vulnerability_category), and search (substring match on rule name). Results are paginated: pass the returned cursor to fetch the next page. page_size is capped at 100. Use get_issue for full detail and remediation guidance on a specific issue.

NameRequiredDescriptionDefault
cursorNo
searchNo
rule_idNo
categoryNo
severityNo
page_sizeNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and delivers real behavioral facts: cursor-based pagination semantics, a hard page_size cap of 100, and exact filter matching behavior (substring on rule name). It omits auth requirements, default sort order, and returned fields, but the operational behavior an agent needs to page correctly is disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the one-line purpose, then a compact filter list and pagination rules; every sentence adds usable information. The terse filter-list formatting is slightly telegraphic but nothing is wasted or padded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-parameter, no-annotation, no-output-schema list tool, the description covers filtering, pagination, and the sibling hand-off well enough to invoke correctly. It stops short of describing the shape or ordering of returned issues, which is the only remaining gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate, and it does: severity enumerates the five accepted values, category is mapped to vulnerability_category, search is defined as a substring match on rule name, cursor is tied to the returned cursor for next-page fetching, and page_size gets a stated cap. All six parameters gain meaning beyond the bare schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('List') and resource ('AI-SPM issues') scoped to the workspace, and explicitly names the sibling 'get_issue' as the detail-oriented alternative, so an agent can separate the two without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly routes to get_issue 'for full detail and remediation guidance on a specific issue,' which is a clear use-vs-alternative statement. It does not state when not to use this tool (e.g., searching across workspaces), but the selection guidance is unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_llm_interactionsA

List LLM interactions (prompt/response pairs) with guardrail outcomes.

Filters: search (substring match on prompt text), agent_id. Cursor-paginated: pass the returned cursor to fetch the next page. Rows are trimmed (long prompt/response text truncated, per-turn events omitted) to protect context budget — use get_interaction for full detail on a specific interaction. page_size is capped at 100.

NameRequiredDescriptionDefault
cursorNo
searchNo
agent_idNo
page_sizeNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does substantial work: it discloses cursor pagination semantics, that rows are deliberately trimmed (truncated text, omitted per-turn events) to protect context budget, and that page_size is capped at 100. It omits read-only/auth framing, but the trimming and cap disclosures are genuinely useful behavioral context beyond the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short, front-loaded units: purpose, filters, then pagination/trimming behavior. No filler, and the most important scoping caveat (trimming) is stated before the escape hatch is offered.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a list tool with no annotations and no output schema, the description covers filters, pagination, trimming policy, page cap, and where to go for full detail. It does not sketch the shape of a returned row beyond 'trimmed prompt/response with guardrail outcomes', which is the one remaining gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, and it largely does: search is defined as substring matching on prompt text, cursor is explained as the returned value for the next page, and page_size is capped at 100. Only agent_id is left without added meaning, a minor gap given the otherwise strong compensation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (List) and resource (LLM interactions) with a clarifying gloss (prompt/response pairs) and scope (with guardrail outcomes). It routes the agent away from get_interaction, so it is distinguishable from its nearest sibling without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly tells the agent to use get_interaction for full detail on a specific interaction, which is a clear alternative-with-condition. It does not address when to prefer list_llm_sessions or list_roi_interactions, so it falls short of the full when/where-not/alternatives bar.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_llm_sessionsA

List LLM sessions (grouped conversation turns) for one agent (required).

Cursor-paginated; page_size is capped at 100. Use get_llm_session for a session's aggregate cost/tokens/turns plus the full per-turn list.

NameRequiredDescriptionDefault
sortNo
cursorNo
agent_idYes
sort_dirNo
page_sizeNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden and it delivers the key behavioral fact for a read-only list tool: results are cursor-paginated and page_size is capped at 100, a constraint the schema alone doesn't express. It omits auth/permission requirements and return shape, keeping it out of 5 territory.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with the core purpose and scope. The pagination/cap detail and the sibling routing both earn their place with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a list tool with no output schema and no annotations, the agent still lacks the shape of a returned session and the accepted sort/cursor semantics. Pagination and sibling routing are covered, but the return contract and remaining parameter semantics leave real gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% across 5 params, so the description must compensate. It does cover the required agent_id and the page_size cap (useful beyond the schema's default of 25), but sort, sort_dir, and cursor are left entirely unexplained, including no indication of valid sort fields or direction values.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (list LLM sessions) and even defines the resource ('grouped conversation turns') and its scope ('for one agent (required)'). This clearly separates it from get_llm_session, which is named in the same description.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Names the alternative get_llm_session and the condition that selects it (aggregate cost/tokens/turns plus per-turn list), so the agent knows which tool to pick. It doesn't state an explicit when-not-to-use forward (e.g., bulk export or cross-agent queries), so it falls just short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_mcp_serversB

List inventoried MCP servers. Filters: name, severity, supplier, library.

page_size is capped at 100; page with the returned cursor for more.

NameRequiredDescriptionDefault
nameNo
cursorNo
libraryNo
severityNo
supplierNo
page_sizeNo

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it does disclose the useful pagination behavior: page_size is capped at 100 and a returned cursor is used to page further. However, it omits other behavioral context such as read-only nature (only implied by "list"), authorization needs, and the shape of returned records.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences with zero waste: purpose first, then the filter set, then the pagination constraint. Every sentence carries information an agent needs, and the critical cap is stated explicitly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-parameter list tool with no annotations and no output schema, the description covers filtering and pagination but not filter semantics or the fields returned. It is minimally adequate, with clear gaps an agent must resolve by experimentation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate; it names four filter parameters (name, severity, supplier, library) and documents the page_size cap. It does not explain matching semantics (exact vs partial, case sensitivity, expected format) or the cursor, leaving the parameters only partially specified.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("List inventoried MCP servers"), and "inventoried" adds a scoping qualifier beyond the plain name. It is distinguishable from the singular sibling get_mcp_server, but the description does not explicitly name or differentiate against siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use or when-not-to-use guidance and no named alternatives; the agent must infer that this is the list counterpart to get_mcp_server. The only contextual hint is the filter list, which implies a search/list use case but doesn't state it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_modelsB

List inventoried AI models. Filters: name, severity, supplier, library.

page_size is capped at 100; page with the returned cursor for more.

NameRequiredDescriptionDefault
nameNo
cursorNo
libraryNo
severityNo
supplierNo
page_sizeNo

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden but only discloses pagination behavior (page_size capped at 100, cursor returned for continuation). It says nothing about read-only nature, auth requirements, or result shape, though for a read-list tool the omissions are moderate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short, front-loaded sentences: purpose and filters first, pagination constraint second. No filler; each sentence carries weight, though a fragment-style filter list is slightly terse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-param, no-annotation, no-output-schema read tool, the description covers purpose, filters, and pagination but omits value formats for enum-like filters and any indication of what a returned model contains. Adequate but with clear gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% across 6 params, so the description must compensate. It names all filter params (name, severity, supplier, library) and explains page_size/cursor, but adds no value semantics such as accepted severity values, matching behavior, or the page_size default.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (List) and resource (inventoried AI models), and naming convention distinguishes it from the sibling get_model for single-model retrieval. It doesn't explicitly say it is the bulk/listing counterpart, so it falls just short of full sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The listing intent and available filters imply when to reach for it over get_model, but there is no explicit when/when-not guidance or routing to get_model for a single record. Usage is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_roi_agentsB

List agents' LLM activity summaries (latest_interaction_time, interaction_count) for the workspace. Drives ROI/cost-optimization analysis: an agent with no recent activity has nothing new to analyze.

page_size is capped at 100.

NameRequiredDescriptionDefault
pageNo
page_sizeNo

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It usefully discloses the return shape (latest_interaction_time, interaction_count) and a real operational constraint (page_size capped at 100), but omits ordering, default behavior, auth requirements, and full pagination semantics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Short and front-loaded: the core purpose and return fields come first, then the analytical rationale, then the limit constraint. No wasted sentences, though each line is terse enough to leave gaps rather than being verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description correctly names the returned fields, which is helpful. However, for a paginated list tool with no annotations it fails to explain the 'page' parameter or pagination flow, leaving the calling contract incomplete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It adds meaning for page_size (the 100 cap beyond the schema's default of 25), but the 'page' parameter is entirely undocumented in both schema and description, leaving half the parameters opaque.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (List) and resource (agents' LLM activity summaries), and enumerates the returned fields (latest_interaction_time, interaction_count), which differentiates it from the plain list_agents. However, it does not explicitly distinguish itself from adjacent ROI siblings like list_roi_interactions or list_roi_analyses.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The clause 'Drives ROI/cost-optimization analysis: an agent with no recent activity has nothing new to analyze' implies the context of use, but there is no explicit when-to-use vs. when-not-to-use and no routing guidance to alternatives such as list_roi_interactions or list_agents.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_roi_analysesA

List ROI/cost-optimization analysis results (findings + estimated $ savings), latest first.

Filters: agent_id, analysis_type (tool_bloat, model_optimizer, verbosity, context_cache_waste, retry_loop_waste). page_size is capped at 100. Use get_latest_roi_analyses for just the current recommendation per type, or get_roi_analysis for one result's full detail.

NameRequiredDescriptionDefault
pageNo
orderNo
agent_idNo
page_sizeNo
analysis_typeNo

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It does disclose real traits beyond the schema: newest-first ordering, a hard page_size cap of 100, and roughly what a result contains. It stops short of stating the read-only nature explicitly, how filters combine, or what an empty/oversized page returns.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action and result shape, then filters, then the cap, then sibling routing — a logical order with no filler sentences. The parenthetical type list is dense but each token is load-bearing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read/list tool with no output schema and no annotations, the description supplies result contents, ordering, the cap, filter semantics, and alternatives, which is close to complete. The remaining gap is the meaning of the page and order parameters, which nothing describes.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate. It documents agent_id and enriches analysis_type with five concrete enum-like values (tool_bloat, model_optimizer, verbosity, context_cache_waste, retry_loop_waste) plus the page_size cap. However, page and order remain completely undocumented in both schema and description, leaving two of five parameters opaque.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (List) and resource (ROI/cost-optimization analysis results), plus the payload an agent gets back (findings + estimated $ savings) and ordering (latest first). It also names the two sibling tools it is not, so an agent can route without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly routes the agent: use get_latest_roi_analyses for only the current recommendation per type, and get_roi_analysis for a single result's full detail. The when-to-use vs. alternatives is fully spelled out with no inference required.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_roi_interactionsA

List one agent's raw LLM interaction rows (with computed cost_usd) — the source data the ROI cost-optimization analyses read.

agent_id is required. since/until are optional RFC3339 timestamps bounding interaction_time. page_size is capped at 100.

NameRequiredDescriptionDefault
pageNo
orderNo
sinceNo
untilNo
agent_idYes
page_sizeNo

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It does disclose real traits: agent_id is mandatory, page_size is hard-capped at 100, and the rows carry a computed cost_usd. It omits auth requirements, how paging/ordering behave, and what happens at the cap (truncation vs error).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences with zero filler; the identity of the resource and its relation to the ROI analyses come first, followed by constraints. Nothing repeats the schema and nothing padding is present.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a six-parameter read tool with no annotations and no output schema, the description covers the essential required filter, the time-window semantics, and the page-size cap. It still leaves the agent guessing on pagination controls (page, order) and on what a row looks like beyond the cost_usd field.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate, and it does document four of six parameters: agent_id (required), since/until (RFC3339 bounding interaction_time), and page_size (capped at 100). However, page and order are left entirely undocumented in both schema and description, so coverage is incomplete.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (List) plus resource and scope (one agent's raw LLM interaction rows, with computed cost_usd), and frames it as the underlying source data for the ROI analyses. That framing helps separate it from the nearby list_llm_interactions sibling, though it never names that sibling explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Use is implied rather than stated: the clause 'the source data the ROI cost-optimization analyses read' suggests reaching for this when you want the raw rows behind an ROI analysis. There is no explicit when-to-use rule, no prerequisite guidance, and no alternative routed to (e.g. list_llm_interactions), leaving the agent to infer.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_toolsB

List inventoried agent tools. Filters: name, severity, supplier, library.

page_size is capped at 100; page with the returned cursor for more.

NameRequiredDescriptionDefault
nameNo
cursorNo
libraryNo
severityNo
supplierNo
page_sizeNo

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It usefully discloses the page_size cap of 100 and the cursor-based paging contract, but says nothing about return shape, default result ordering, or whether omitting all filters returns everything.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences, filter surface listed first and the pagination constraint second. No filler and nothing repeated from the schema titles.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Six optional parameters with no schema descriptions, no annotations, and no output schema make the description the only source of meaning. Filters and pagination are covered, but parameter value semantics and default full-corpus behavior remain unspecified.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and the schema provides only bare titles, so the description must compensate. It maps four of the six parameters to filter concepts but gives no value formats (e.g. valid severity strings or library identifiers) and never explains what cursor accepts.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('List inventoried agent tools') that clearly separates it from the sibling get_tool and from list_agents/list_mcp_servers by naming the inventory subject. It does not explicitly name an alternative, but the list-vs-get distinction is inferable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Supplies filter names and tells the agent to page with the returned cursor, which implies the intended usage pattern. There is no explicit when-to-use-vs-alternatives guidance, e.g. when to use get_tool instead or when filters are required.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 37 tool updatesv0.2.2
    • First observedfind_issues_for_file
    • First observedget_agent
    • First observedget_agent_endpoint
    • First observedget_agent_evaluation
    • First observedget_agent_evaluation_summary
    • First observedget_context
    • First observedget_evaluation
    • First observedget_evaluation_run
    • First observedget_finding_detail
    • First observedget_guardrail_categories
    • First observedget_guardrail_interaction_counts
    • First observedget_guardrail_outcomes
    • First observedget_guardrail_policy
    • First observedget_interaction
    • First observedget_issue
    • First observedget_latest_roi_analyses
    • First observedget_llm_session
    • First observedget_mcp_server
    • First observedget_model
    • First observedget_remediation_guidance
    • First observedget_roi_analysis
    • First observedget_tool
    • First observedlist_agent_evaluations
    • First observedlist_agents
    • First observedlist_evaluation_run_results
    • First observedlist_evaluation_runs
    • First observedlist_evaluations
    • First observedlist_guardrail_policies
    • First observedlist_issues
    • First observedlist_llm_interactions
    • First observedlist_llm_sessions
    • First observedlist_mcp_servers
    • First observedlist_models
    • First observedlist_roi_agents
    • First observedlist_roi_analyses
    • First observedlist_roi_interactions
    • First observedlist_tools

TDQS

B3.4/5.0

Scored across 37 tools

Disambiguation4/5

Tools are mostly well-separated by resource and action (list vs get vs summary). The clearest overlap is get_guardrail_interaction_counts vs get_guardrail_outcomes, which the description admits share the same underlying data, and get_remediation_guidance overlaps with the remediation embedded in get_issue. These are minor and the descriptions largely disambiguate.

Naming Consistency4/5

Overwhelmingly consistent verb_noun pattern (list_*/get_*) with predictable suffixes like _detail, _summary, _runs, _results. A few deviations: find_issues_for_file breaks the list_ convention, and get_context/get_model are vaguer than their peers. Overall readable and consistent.

Tool Count3/5

37 tools is heavy, but the server spans many genuinely distinct domains (guardrails, AI-SPM issues, model/agent/tool/MCP inventory, LLM interactions, ROI, evaluations, red-team endpoints), each with roughly 3-6 tools. It is justified but sits well above the comfortable 3-15 band and risks context bloat.

Completeness4/5

The surface is broad and largely complete for a read/analytics security platform: list+get coverage for guardrails, issues, inventory, interactions, sessions, ROI, and evaluations, plus aggregation/summary endpoints. Gaps are minor (no per-result detail endpoint for evaluation results, acknowledged trimming) and agents can work around them.

Maintenance

ActivityMaintained
ResponsivenessUnresponsive

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    D
    maintenance
    Provides AI agents with secure, read-only file system access to analyze and understand project codebases, enabling multi-repository context aggregation and cross-project code tracing.
    41 PyPI
    5
    MIT
  • A
    license
    Not graded
    quality
    B
    maintenance
    Gives AI agents read-only access to any QuestDB instance, enabling schema discovery, time-series querying, and monitoring of partitions, WAL state, symbols, and ingestion health through MCP.
    Apache 2.0
  • A
    license
    Not graded
    quality
    B
    maintenance
    Enables read-only inspection of Depot CI failures, job logs, GitHub Actions jobs on Depot runners, and container builds through the Depot API, so agents can diagnose issues without installing the Depot CLI.
    47 npm
    MIT