Server Quality Checklist

Tool Scores

• get_pageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses what data is returned (content, title, URL, last modified) but omits critical behavioral aspects such as error handling for missing pages, idempotency, authentication requirements, or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—two sentences that front-load the purpose and then list parameters. Every word adds value; there is no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of an output schema and annotations, the description should provide more details about the return structure, error conditions, and page retrieval behavior. It only vaguely mentions what is returned without specifying format or edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It lists all three parameters with examples (e.g., 'wikipedia' for project, 'en' for language), adding meaning beyond the schema's bare types. However, it does not explain valid values or format requirements for parameters like project or language.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get Wikimedia page content, title, URL and last modified date.' It specifies the verb and resource, making the function obvious. However, it does not explicitly differentiate from sibling tools like 'get_featured' or 'search_titles', which share a similar domain.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description lacks context about specific use cases, prerequisites, or situations where other tools would be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_languagesB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It only states the action and parameters, omitting any details on return format, limitations, permissions, or side effects. For a read operation, this minimal disclosure is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence followed by a parameter list. It is concise, front-loaded with the purpose, and contains no unnecessary information. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 3 parameters, no output schema, and no annotations, the description covers the basic purpose and parameter listing but lacks information about the return value, error handling, and how this tool compares to siblings. It is minimally adequate but incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description adds examples for 'project' and 'language' and notes that 'title' is required. This adds some meaning beyond the bare schema, but it does not explain valid values, formats, or semantics. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get versions of a Wikimedia page in other languages.' The verb 'get' is specific, and 'versions in other languages' distinguishes it from siblings like 'get_featured' or 'search_content'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. No explicit when/when-not conditions or mentions of sibling tools. Usage is only implied by the purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_contentA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; description fails to disclose behavioral traits such as read-only nature, authentication, or rate limits. Only states basic functionality.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose, no wasted words. Efficiently communicates core information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Lacks output schema; description minimally covers return (snippets) but omits pagination, error handling, or other details. Adequate but not thorough for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description adds meaning by listing parameters with examples and range for limit, though query format is not detailed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs full-text search across Wikimedia page content and returns snippets, distinguishing it from siblings like get_page and search_titles.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like search_titles or get_page. Absence of context for usage scenarios or limitations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_on_this_dayA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility for behavioral disclosure. It correctly implies a read-only operation ('Get historical events') but omits details such as rate limits, response format, or behavior on invalid/unsupported dates. The description adds no warning or context beyond the basic operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: one sentence stating the purpose followed by a compact parameter list. Every word adds value, with no filler or repetition. The structure front-loads the purpose and then efficiently documents parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description does not explain the return format, which could impact tool selection. It also lacks context such as date range validity, error behavior, or authentication needs. For a simple historical events tool, it is minimally adequate but leaves gaps for an agent to infer expected outputs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning beyond the input schema. It specifies the date format (MM/DD) and default (today), lists all type enum values with their meanings, restricts project to 'wikipedia' only, and enumerates supported languages. Schema coverage is 0%, so the description fully compensates by detailing each parameter's allowed values and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get historical events from Wikimedia for a date.' The verb 'Get' and resource 'historical events' are specific. The name 'get_on_this_day' further clarifies the context. It distinguishes from siblings like 'get_featured' (featured content) and 'search_content' (full-text search) by focusing on historical events for a single date.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus its siblings. It does not mention prerequisites, expected date ranges, or limitations. The description only lists parameters without explaining the appropriate context for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_featuredA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. Description specifies output (featured article, most read, picture of the day) and parameter defaults, but does not disclose behavior for invalid dates, rate limits, or whether the operation is read-only. Adequate but not detailed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, concise and no wasted words. Front-loaded with main action and immediately lists parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with no output schema or annotations, description covers return items and parameters well. Missing error handling details, but completeness is high given complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 0% description coverage. Description compensates by specifying date format (YYYY/MM/DD), default values, allowed values for project ('wikipedia' only) and language (list of codes). Adds significant meaning beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool gets featured Wikimedia content for a date, listing specific items returned (featured article, most read pages, picture of the day). It distinguishes from siblings like get_page (single page) and search_content (search results) by specifying the type of content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description does not explicitly state when to use this tool versus alternatives. It implies usage for retrieving featured content by date, but no when-not-to or comparison to sibling tools like get_languages or get_on_this_day.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_titlesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes read-only behavior (searching, returning suggestions) and provides parameter details. No annotation contradictions. Lacks details on rate limits or return format, but acceptable for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first for purpose, second for parameters. No wasted words, front-loaded with verb and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Complete for a simple search tool: covers functionality, return type, and all parameters. Could explicitly differentiate from sibling tools but not essential.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description explains all four parameters, including range for limit and examples for project/language, adding value beyond schema types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool searches Wikimedia page titles starting with the query, and returns suggestions with descriptions. Distinguishes from sibling search_content by specifying title prefix matching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implied usage is clear from name and description, but no explicit guidance on when to use this tool versus alternatives like search_content for full-text search.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.