Skip to main content
Glama

Preflight a third-party tool

evaluate_tool_risk

Make a contextual allow, review, or block decision before an AI Agent installs or invokes a third-party tool registered in 404.directory. Use this immediately before installation or first use, and again when permissions, data sensitivity, execution mode, or evidence changes. The decision cites ownership, lifecycle, verification history and freshness, compatibility, security, and observed-usage evidence; missing evidence never counts as safe. Stores a bounded receipt without prompts or payloads and returns a one-time outcome token so the Agent can later report whether it proceeded, changed tools, requested review, or aborted. Does not execute or freshly probe the target and is not a security guarantee.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
actionYesThe next action the Agent is considering: inspect, install, or invoke.
targetYes404.directory catalog tool UUID or slug.
permissionsNoPermissions or side effects needed for this action. Include every applicable value.
execution_modeNoWhether a human supervises this action or it runs unattended.supervised
data_sensitivityNoHighest sensitivity of data the Agent may expose to the tool.public

Schema Changelog

Changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. Added

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations are all false (no hints), so the description carries the burden. It discloses important behaviors: 'Does not execute or freshly probe the target', 'Stores a bounded receipt without prompts or payloads', 'returns a one-time outcome token', and 'missing evidence never counts as safe.' This adds meaningful context about side effects and limitations beyond the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is four sentences, dense with information but not bloated. It front-loads the core purpose and usage trigger, then briefly covers behavior and return value. Every sentence contributes; no filler. Slightly long but appropriately so for a decision tool with behavioral caveats.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description adequately explains the return value ('one-time outcome token') and the decision categories ('allow, review, or block'). It covers what the tool does not do (execute, probe, guarantee security) and what it stores (bounded receipt). For a 5-parameter decision tool, this covers the essential usage and behavioral context an agent needs to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds value by connecting parameters to the decision context: it states the tool should be re-run when 'permissions, data sensitivity, execution mode, or evidence changes', which directly maps to the relevant parameters. It also explains that the decision cites 'ownership, lifecycle, verification history' etc., giving purpose to the target parameter. This goes beyond simple parameter names.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('Make a contextual allow, review, or block decision') and a specific resource ('third-party tool registered in 404.directory'). It clearly distinguishes itself from sibling information-retrieval tools like get_tool or get_trust_score by emphasizing it produces a decision, not just data.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit timing: 'Use this immediately before installation or first use, and again when permissions, data sensitivity, execution mode, or evidence changes.' This is actionable and covers re-evaluation triggers. It does not name alternative tools for when not to use it, but the clear trigger conditions satisfy most usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

A4.2/5.0
Disambiguation4/5

Tools are mostly distinct with clear descriptions. Potential overlap exists among search_tools, recommend_tools, list_capabilities, and get_capability_graph, but each serves a different purpose (query vs. recommendation vs. high-level list vs. relational graph). inspect_tool_server vs. get_tool (server vs. tool) and verify_web vs. understand_webpage (reachability vs. content) are clearly separated. Minor ambiguity between list_capabilities and get_capability_graph, but descriptions clarify.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern (e.g., compare_tools, get_tool, search_tools, verify_web). No mixing of camelCase or inconsistent styles. The naming is uniform and predictable.

Tool Count5/5

12 tools is within the ideal 3-15 range and appropriate for a directory service that provides search, retrieval, comparison, recommendation, inspection, invocation, and web verification. The count is comprehensive without being overwhelming, and each tool adds distinct value.

Completeness4/5

The tool set covers core directory operations (search, get, compare, recommend) and additional utilities (inspect, invoke, trust score, capability graph). It lacks a direct 'list all tools' or 'list all servers' endpoint, but search_tools and list_capabilities can approximate this. The inclusion of web verification and understanding tools extends beyond the catalog domain, but they are useful adjuncts. Overall, the surface is well-rounded with minor gaps.