Server Quality Checklist

Tool Scores

• server_statusB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It indicates read-only via [READ] but does not describe side effects, error handling, or authentication requirements. The simple nature partially mitigates this, but more detail on behavior (e.g., what 'reachability' means) would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two lines and a param definition. It front-loads the key info [READ] and uses minimal words. Every sentence is necessary and no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple status-check tool with one optional parameter and no output schema, the description covers the core behavior and parameter. However, it lacks information about the return value format (e.g., boolean, JSON) or error scenarios. Since there is no output schema, adding a brief note about the output would make it more complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It explains the single parameter 'target' as an optional Ollama target name, with a default when omitted. This adds value beyond the schema. However, it does not clarify what the default target is, leaving some ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks Ollama reachability and version, using a [READ] prefix to indicate read-only. It distinguishes from siblings like list_models and running_models, which focus on models rather than server status. However, it could be more specific about the output.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. The parameter description is present but does not provide context for selection among siblings. A statement like 'Use this to verify server connectivity before proceeding' would help.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• anomaly_reportB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but only adds a '[READ]' tag. It does not disclose behavioral traits such as authentication needs, rate limits, data freshness, or side effects, leaving significant gaps for a tool that likely accesses sensitive metrics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, front-loading the critical '[READ]' tag followed by a clear list of metrics and a brief parameter explanation. Every sentence is purposeful and no words are wasted.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (one optional parameter, no output schema, read-only), the description is fairly complete: it states the purpose, lists reported items, and describes the parameter. However, it lacks details on output format or examples, which would improve completeness for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single optional parameter 'target' is explained with context ('Ollama target name from config; omit for the default'), which adds value beyond the input schema's minimal type definition. Schema coverage is 0%, so the description compensates well for this simple parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is a read operation ('[READ]') and lists the specific metrics reported ('shadow models, digest drift, high-risk prompts, blocked count'), making the purpose unambiguous. It distinguishes from sibling tools that also perform reads or actions by specifying a unique rollup report.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like server_status or list_models. The description does not specify scenarios, prerequisites, or exclusions, leaving the agent to infer usage without explicit context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pull_modelA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description only indicates write operation and policy check, lacking details on side effects, disk usage, or latency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise; front-loaded with action and risk, then clear arg descriptions. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Basic but misses behavioral context like download size, time, or post-pull effects; acceptable for a simple tool but could improve.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema coverage, description explains both parameters ('Model name to pull' and 'Ollama target name from config; omit for default'), adding value beyond schema types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description explicitly states 'Pull a model' with a policy refusal note, clearly distinguishing it from siblings like list_models or remove_model.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use guidance or exclusion criteria; only mentions policy rejection but no alternatives among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• policy_viewA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It only says '[READ]' and lists the content, but does not mention error conditions, caching behavior, or permission requirements. Minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with a clear '[READ]' prefix. It is concise and front-loaded, but could benefit from slightly more structure, such as listing the two components separately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read tool with no parameters and no output schema, the description provides the essential content (policy and pins). However, it lacks any hint of the response format or potential limitations, which slightly reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is 100% trivially. Per guidelines, baseline is 4. The description does not need to add parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it reads the current allow/deny policy and provenance digest pins. '[READ]' marks it as read-only, and the specific resource names distinguish it from sibling write tools like set_model_allowlist and pin_model_digest.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for viewing current policy and pins, but does not explicitly state when to use it versus alternatives like list_models or set_model_allowlist. No when-not-to-use guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• unload_modelA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description alone must disclose behavior. It states it's a write and risk level, but omits details like reversibility, impact on model availability, and side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences plus args list; no fluff. Every sentence adds value. Front-loaded with risk tag.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool with no output schema, the description covers basic usage. However, lacks context on return value, error conditions, and undo integration (siblings include undo_list).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage. The description explains 'model: Model name' and 'target: Ollama target name from config; omit for default.', adding meaning beyond type and default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Evict a model from VRAM (keep_alive:0).' This clearly defines the action (evict/unload) and resource (model), distinguishing it from siblings like remove_model (delete model) and pull_model (download).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description marks the operation as a write with medium risk but does not explicitly contrast with alternatives (e.g., vs remove_model for deletion). Usage context is implied but not detailed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• model_detailsA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the full behavioral disclosure burden. It only indicates the tool is a 'READ' operation but fails to detail side effects, authentication requirements, rate limits, or any other behavioral traits. The lack of safety or mutability context is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, using two short sentences plus parameter definitions. The '[READ]' prefix quickly signals the tool's operation, and every sentence provides essential information without redundancy. The structure is front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description should clarify the return format or structure. It mentions 'License / parameters / capabilities' but does not specify what data is returned or how to interpret it. This leaves uncertainty for the agent about the tool's output expectations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must define parameters. It adequately explains 'model' with an example ('llama3.2:3b') and 'target' with guidance ('Ollama target name from config; omit for the default'). This adds meaningful context beyond the bare schema, though it could be slightly more detailed about accepted formats or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'License / parameters / capabilities for one model' with a '[READ]' prefix, establishing a specific verb and resource. It differentiates from sibling tools like 'list_models' (all models) and 'running_models' (only running models) by focusing on a single model's details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly lists parameters with examples ('e.g. "llama3.2:3b"') and explains when to use them (omit target for default). While it lacks explicit exclusion of alternatives, the context of being a single-model detail retrieval is clear, and the sibling list provides natural comparison points.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• model_provenanceA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that the tool is a read operation and compares digests, but does not mention what happens on drift (e.g., generates a report, returns a list), any authorization needs, or side effects. It's adequate but could be more transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise, with two sentences plus arg explanation. It is front-loaded with [READ] and the core action. Every sentence is meaningful without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and lack of output schema, the description covers the primary functionality. However, it omits details about the output format (e.g., list of models with drift status) and any prerequisites (e.g., the model must be pinned). It is complete enough for a straightforward tool but has gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description explains the 'target' parameter: 'Ollama target name from config; omit for the default.' This adds meaning beyond the schema's type and default. Although context shows 0% schema description coverage, the description effectively documents the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action: 'Compare each installed model's digest against its pin; flag drift.' The [READ] prefix indicates it's a read operation. It distinguishes itself from sibling tools like list_models or running_models by focusing on integrity validation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for checking model integrity, but it does not explicitly state when to use this tool versus alternatives (e.g., when to suspect tampering or after pulling models). No when-not-to-use guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pin_model_digestA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so description carries full burden. It includes '[WRITE][risk=medium]' indicating a write operation with medium risk. However, it does not explain what pinning entails (e.g., reversibility, effect on existing pins), leaving behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is extremely concise, consisting of a single sentence plus a parameter list. The '[WRITE][risk=medium]' tag front-loads key information. No fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 2 required params and no output schema, the description explains the purpose and parameter sources but omits details on return values, error states, or prerequisites (e.g., model must exist). Adequate but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, description adds meaning: it labels the 'digest' parameter as 'the expected digest to pin (from list_models / model_details)', providing source context. The 'model' parameter is only described as 'Model name', which is minimal but acceptable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the action ('Pin'), resource ('model's expected provenance digest'), and purpose ('drift detection'). It distinguishes from sibling tools like list_models and model_details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by referencing where the digest comes from (list_models / model_details), but does not explicitly state when to use or not use this tool, nor mention alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vram_usageA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description carries full burden. Clearly states it reads VRAM and optionally flags over-budget models. Does not mention side effects, permissions, or rate limits, but for a read operation the behavior is adequately disclosed. The [READ] tag helps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise: one sentence plus two arg lines. Every sentence adds value. Front-loaded with '[READ]' for quick scanning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, so description should explain return values. Implies output includes total VRAM and flagged models, but lacks format details. Does not mention edge cases like no models loaded or invalid target. Adequate for a simple query but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must compensate. Describes budget_bytes as 'Optional VRAM budget; models over it are flagged' and target as 'Ollama target name from config; omit for the default.' Adds meaning beyond schema types but minimal detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Starts with '[READ]' clearly indicating read operation. States specific verb+resource: 'Total VRAM used by loaded models; flag over-budget.' Distinguishes from siblings like running_models (lists models) and server_status (general status) by focusing on VRAM usage and budgeting.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or alternatives. The budget_bytes parameter hints at context (monitoring VRAM budgets), but no guidance on when to use this over other monitoring tools like server_status or running_models.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_modelsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must disclose behavior. It clearly states it's a read operation and explains output includes policy annotations. No contradictions, but could mention response structure or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is very concise: three short lines covering purpose, output detail, and parameter. No wasted words, front-loaded with the read indicator.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with one optional parameter and no output schema, the description covers the main purpose and parameter. It could mention if results are paginated or include model sizes, but overall sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, description compensates by explaining the 'target' parameter: its meaning (Ollama target from config) and default behavior (local). Adds significant clarity beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists installed models with policy verdicts, distinguishing it from sibling tools like running_models. The verb and resource are specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs. alternatives like running_models or model_details. The description only explains the parameter, not usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• observe_chatA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses the tool is a write operation ([WRITE]) with medium risk, and outlines the flow (scan, policy-gate, record, run). However, it does not specify what happens when the policy blocks the chat (e.g., error vs. silent failure) or what exactly is recorded (e.g., usage log details). Additional behavioral context, such as side effects or authorization needs, would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise: a one-line purpose summary followed by a clean Args list. Every sentence adds value. No redundant or extraneous text. Front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description should hint at return values (e.g., model response, block status). It omits output expectations. Also, error handling or side effects of the recording are not mentioned. While parameters are well-covered, the overall user journey (what the agent gets back) is incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must explain parameters. It does so for all 6: model (model to run), messages (OpenAI-style format), agent/user (actor attribution), block_threshold (threshold for blocking), target (Ollama target, omit for default). This adds rich functional meaning beyond schema field names and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a clear verb+resource: 'Scan + policy-gate a chat exchange, record it, then run if allowed.' It distinctly differs from siblings like scan_prompt (scan only) and guarded_generate (generation with guard) by combining scanning, policy enforcement, recording, and execution in one step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: use when you want to run a chat through policy checks before execution. However, it does not explicitly state when not to use this tool or suggest alternatives (e.g., scan_prompt for just scanning, guarded_generate for generation without recording). Usage context is merely implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_model_allowlistA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It adds '[WRITE][risk=medium]' indicating a write operation with risk level, and explains empty list behavior. However, it does not disclose if the replace is atomic, requires permissions, or has immediate effect.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two lines with no wasted words. It front-loads the risk indication and directly states purpose, followed by a clear parameter definition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description is largely complete. It covers parameter format, replace behavior, and empty list semantics. Minor gap: lack of contrast with set_model_denylist sibling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% (no description in schema), so description must compensate. It explains that models are 'Shell-glob patterns of permitted model names' with an example, adding significant meaning beyond the schema's array type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Replace' and the resource 'model allowlist', specifying glob patterns and the behavior for empty list (allow-all). This distinguishes it from siblings like set_model_denylist.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for setting allowed models via glob patterns, but does not explicitly contrast with siblings like set_model_denylist or provide when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• running_modelsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description explicitly marks the tool as [READ], indicating read-only behavior. No annotations are provided, so the description carries the burden. It does not mention any side effects or permissions, but for a read-only tool, the behavior is sufficiently disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two lines with no superfluous text. The first line states the purpose, the second explains the parameter. Every word serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description covers the essential: what it returns and how to parameterize. It could mention the return format (list or object), but the stated info (VRAM footprint + residency expiry) is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no descriptions (0% coverage), but the description adds meaning for the only parameter: 'target: Ollama target name from config; omit for the default.' This tells the agent what the parameter is and how to use it, compensating for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it reads currently loaded models and specifies the returned data (VRAM footprint + residency expiry). This is a specific verb-resource combination, and it distinguishes from siblings like list_models and vram_usage.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for checking loaded models and mentions the target parameter from config. However, it does not provide guidance on when not to use this tool or when alternatives (like list_models) might be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_model_denylistA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full responsibility. It reveals the tool is a WRITE operation with medium risk, and that the denylist replaces the entire list with patterns that always win. However, it does not disclose persistence or effect on currently running models.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: a single line with the WRITE tag and purpose, followed by a parameter description. Every word adds value, with no redundancy. It is front-loaded with the most important information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (one parameter, no output schema), the description covers the tool's core purpose and behavior adequately. It explains the replacement action and pattern syntax, though it could hint at return values or confirm whether the change is applied globally or session-based.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description entirely compensates by explaining that 'models' are 'Shell-glob patterns of forbidden model names', providing essential semantics beyond the schema's minimal definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool replaces the model denylist, using specific verb ('Replace') and resource ('model denylist'), and distinguishes it from sibling tools like set_model_allowlist by noting 'deny patterns always win'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies that the denylist overrides the allowlist ('always win'), but does not explicitly state when to use this tool over others (e.g., set_model_allowlist) or mention prerequisites or consequences of replacement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• proxy_guidanceA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure, and it excels. It explicitly states side effects (writes nothing, starts no listener), what it returns (composed command plus caveat), and important limitations like the proxy being a 'CHOKEPOINT, not an enforcement boundary' and the fact that responses stream uninspected. It also explains the parsing behavior for unparseable bodies.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is somewhat lengthy but well-structured, front-loading with '[READ]' and then providing essential caveats and parameter details. Every sentence adds valuable context, though it could be tightened by moving some caveats to a separate section. It remains appropriately sized for the complexity of the tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for an informational tool. It covers the tool's action (composing a command), its non-action (no writes or listener), and its output (the caveat), along with critical behavioral specifics. No output schema exists, but the description sufficiently explains what the user gets. It also addresses proxy limitations and request handling, making it self-contained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no property descriptions (coverage 0%), so the description must compensate. It does so with brief but meaningful explanations for each parameter: listen_host ('Address the proxy would bind'), listen_port ('Port the proxy would bind'), and target ('Runtime target from config; omit for the default'). This adds practical meaning beyond the schema's types and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: it explains how to run the transparent capture proxy and what it does not guarantee, by composing a command and returning a caveat. It distinguishes itself from sibling tools by explicitly noting it 'WRITES NOTHING and starts no listener,' making it a guidance/informational tool rather than an action tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied rather than explicitly stated. The '[READ]' prefix and the note that it writes nothing indicate this is for guidance, not for execution, but there is no direct 'use this when' or 'use X instead' guidance. The description provides context but lacks explicit when/when-not or alternative tool references.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scan_promptA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes the scan as pure content scan, deterministic, returning findings with risk band. Lacks details on detection methods or limitations, but adequate for a simple tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is short and front-loaded with '[READ]' to indicate nature. Minor formatting oddity (dash) does not detract. Could be slightly more structured but overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple deterministic scan with one parameter, the description fully covers purpose, usage, behavior, and parameter without needing output schema. No gaps given tool complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides a brief parameter definition ('text: The text to scan'), which adds slight value beyond the schema. With 0% schema coverage, more detail would be beneficial, but the single parameter makes it acceptable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The tool name and description clearly indicate it scans text for secrets/PII/code/jailbreak. It specifies it's a deterministic read operation, distinguishing it from sibling tools that involve model calls.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states 'Use it to pre-check anything before sending it to a local model,' providing clear context. Also notes 'No Ollama call' to differentiate from generative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• remove_modelA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Given no annotations, the description fully discloses destructive nature, dry-run behavior, undo recording conditions (only if policy permits re-pull), and that dry_run carries the same verdict. This is comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with a clear opening, behavioral context, then parameter definitions. Every sentence adds value; not overly verbose for the complexity exposed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Lacks explanation of the normal (non-dry-run) return value or output schema. Since no output schema exists, the description should clarify what the user receives upon deletion (e.g., success message or model details). Otherwise, coverage is solid.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the Args section in the description defines each parameter: model as required name, dry_run as boolean for preview, target as optional default host. Fully compensates for missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Delete a local model', distinguishing it from siblings like list_models (read-only), pull_model (download), and unload_model (unload from memory). The [WRITE][risk=high] prefix reinforces the purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance to use dry_run=True for previewing, and explains the undo recording behavior and policy implications. While it doesn't contrast with alternatives like unload_model, the context is sufficient for understanding when to invoke this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• usage_eventsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that it returns a specific data structure with truncated flag, and warns about result interpretation. Since no annotations are provided, the description carries full burden. It could mention that it is a read-only operation with no side effects, but that is implied by '[READ]'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (~11 lines) and well-structured: first line states purpose, then parameter list, then return structure with a critical usage warning. Every sentence adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a query tool with 5 optional parameters and no output schema, the description fully specifies the return format and explains truncation behavior. It covers all necessary context for an AI agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain parameters. It provides brief but meaningful descriptions for all 5 parameters (e.g., 'model: Filter by model name.', 'risk_level: Filter by risk band (none/low/medium/high/critical).'). Adds meaning beyond schema types and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with '[READ]' and clearly states it queries the observed-usage log for route-through prompts and their findings. Verb 'Query' and resource 'usage log' are specific, and the sibling tools list includes other operations like scan_prompt, so purpose is distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear guidance on interpreting truncated results and warns against drawing conclusions from truncated data. However, it does not explicitly state when to use this tool versus alternatives like scan_prompt or anomaly_report.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• guarded_generateA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavioral traits: scanning for secrets/PII/code/jailbreak, policy checking, blocking based on risk band, recording to usage log, and that the raw prompt is never stored. This is comprehensive for a guarded generation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a header, a paragraph explaining the guard mechanism, and bullet points for args. It is slightly verbose but every sentence adds value, making it efficient for an agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the process and parameters well but does not explain the return value or response format. Given no output schema, a brief mention of what the tool returns (e.g., success/error, generated text) would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, but the description provides meaningful explanations for each parameter (e.g., `model` is the model to run, `block_threshold` blocks when risk band >= this), adding value beyond the schema's titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: a write operation that scans and policy-gates a prompt, records it, and runs Ollama if allowed. It distinguishes itself from siblings like `scan_prompt` which only scans without generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool: when a prompt needs to go through the guard. It implies alternatives like `scan_prompt` for just scanning, but does not explicitly state when not to use it or name all alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• undo_listA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses the read-only nature via [READ] tag, explains the truncated measurement method (fetching one extra row), details effectVerified semantics and implications, and notes that target is unused. Comprehensive coverage beyond minimal requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections: purpose, output format, behavioral notes, and parameter details. Every sentence adds value without redundancy. Front-loaded with the main purpose and key usage hint.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description fully defines the return structure and edge cases (truncated, effectVerified). References sibling tool undo_apply and provides actionable guidance. No gaps identified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates: explains limit default of 50 and cap of 500, and clarifies that target is unused but accepted for CLI uniformity. No ambiguity remains.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists recorded, not-yet-applied undo tokens in reverse chronological order. It distinguishes itself from sibling tools like undo_apply by specifying it is a read operation for inspecting available undo actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly mentions using the undoId with undo_apply, advises re-running with higher limit when truncated, and cautions about effectVerified. Lacks explicit when-not-to-use guidance, but provides sufficient context for correct usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• undo_applyA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses risk level (medium), the mechanism of dispatching an inverse tool through its own governed tool, the one-time nature of token usage, and the dry_run preview behavior. Comprehensive for a write operation with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise and well-structured: a header with risk tag, a paragraph on behavior, then a list of parameters. Every sentence adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, parameters, risk, constraints, and preview mode. Adequate for a meta-tool with no output schema; only minor missing details like error handling, but overall complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema coverage, the description adds an 'Args' section explaining each parameter's meaning and source (e.g., undo_id from undo_list). Fully compensates for missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Apply a recorded undo by dispatching its inverse tool' with a specific verb and resource. It distinguishes from siblings like undo_list by focusing on application rather than listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance: undo_id should come from undo_list, dry_run to preview without executing, token can be applied once. Implies when and how to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.