Server Quality Checklist

Tool Scores

• deactivate_categoryA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It adds useful behavioral context by noting the session scoping and the context-budget effect. However, it does not disclose behavior for edge cases like deactivating a category that is not active, or whether the operation is reversible.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no filler. The primary action and its purpose are front-loaded, and the use-case guidance is directly relevant.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter, session-scoped tool, the description covers what it does, why it matters, and when to use it. It does not mention listing available categories, but the sibling tool list_tool_categories is available in context, making this a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the schema already documents the 'category' parameter with an example. The description reinforces the meaning by referring to 'previously activated category' but adds no new parameter-level details beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Remove'), a specific resource ('previously activated category'), and the scope ('from this session'). It clearly distinguishes the tool from its sibling activate_category by describing the inverse operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides a clear use case: 'Useful when switching between different areas of the codebase.' It implies when to use the tool, though it does not explicitly name alternatives or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tool_categoriesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden, and it delivers the crucial state-dependent behavior: 'Only the categories you activate will be shown in subsequent tools/list calls.' This is exactly the kind of non-obvious behavior an agent must know. It could also state that this call is side-effect-free, but the provided disclosure is strong.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, zero waste, with the most important information front-loaded. Each sentence serves a distinct purpose: what it lists, when to call it, and a critical behavioral caveat. This is a model of concise, useful description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter list tool with no output schema, this description is complete. It covers return contents, invocation order, and a behavioral subtlety that would otherwise be invisible. An agent has everything it needs to decide when and how to call this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are 0 parameters and schema coverage is 100%, so the baseline is already a 4. The description doesn't need to add parameter semantics since there are none, and it doesn't waste words trying.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('List'), a clear resource ('all available tool categories'), and the exact fields returned (descriptions, tool counts, token cost). The distinction from siblings is mostly implied via the workflow cue 'before activating them,' but it doesn't name an alternative sibling like the calibration gold standard, so it falls just short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives explicit guidance to 'Use this first' before activating categories, which clearly signals the intended position in the workflow. It doesn't explicitly discuss when not to use this tool or contrast with search_tools, but the context is clear enough for an agent to select it appropriately.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_toolsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the transparency burden. It discloses the return shape (name + description + input schema), the relevance model ('top matching'), and the key behavior that no activation is needed. This is solid coverage for a read-only search tool, though it does not mention edge cases like no matches found.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: the search intent is in the first sentence, return behavior in the second, and the alternative activation path in the third. There is no redundant text or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter search tool with no output schema, the description gives the agent enough to know what to pass, what it will receive, and whether activation is required. It also mentions the key distinction from activate_category, making the tool self-sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents both query and limit. The description adds no additional parameter-level meaning beyond what the schema provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Search') and resource ('specific tools') and explains how the search is expressed: by describing what you want to do. It also mentions that it returns ready-to-call tool definitions, which clearly separates it from the category-oriented sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit routing for one-off use versus repeated use ('For repeated use of a group, call activate_category instead'). It implies when search_tools is appropriate rather than listing a category, but it does not explicitly contrast with list_tool_categories or get_active_tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_active_toolsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so behavioral disclosure falls entirely on the description. The word 'Show' communicates read-only introspection, and the description explicitly reveals that the tool returns session state, a count of exposed tools, and an estimated token cost. It does not explicitly say it has no side effects, but the verb and zero-parameter design make that clear enough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no filler. The first sentence front-loads exactly what the tool reports, and the second sentence explains why the agent would call it. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, no-output-schema tool, this description is fully adequate. It conveys the tool's scope, output, and a motivating use case. Nothing essential is missing for a model to invoke the tool appropriately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters and 100% schema coverage, the schema is already complete and the description has no parameter details to add. The description does contribute some useful semantics by clarifying what the response will contain, namely active categories, count of exposed tools, and token cost.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb, 'Show', and names a clear resource: the tool categories that are currently active in this session, plus a count and estimated token cost. The phrase 'currently active' and the context-budget emphasis clearly set it apart from sibling tools like list_tool_categories or search_tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description directly tells the agent when to use the tool: 'Use this to understand your current context budget.' It clearly establishes a use case, though it does not explicitly mention exclusions alternatives such as when the agent should use list_tool_categories instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• activate_categoryA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the side effect (client receives full tool definitions on next refresh), the session scope, and the token budget guard behavior. It does not mention whether activation is reversible or if there are any side effects beyond the refresh, but the key behaviors are covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place: the action and effect, the prerequisite listing tool, and the force parameter. Front-loaded with the primary action, no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 params, no output schema), the description is complete for an agent to call it correctly. It covers the action, the prerequisite, the parameter semantics, and the side effect. The only minor gap is not describing what happens after activation (e.g., whether tools are immediately available or require a refresh), but the description does mention the refresh behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents both parameters. The description adds value by explaining the purpose of force (override token budget warning) and the category format (key from list_tool_categories), which goes beyond the schema's basic descriptions. It doesn't add syntax details but the schema already provides those.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Activate all tools in a category'), the resource ('category'), and the session scope. It distinguishes itself from siblings by mentioning the activation effect and the token budget guard, which is unique to this tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells the agent to use list_tool_categories to see available categories and token costs, and explains the force parameter for overriding the budget guard. It implies when to use this tool (when you need tools from a category) and provides a clear alternative for listing categories.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.