Server Quality Checklist

Tool Scores

• search_conferencesA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the tool searches an external source (WikiCFP), implying network dependency, but doesn't mention rate limits, pagination behavior, or whether queries must match exactly. The sentence is honest but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with zero waste. Every word contributes to stating the tool's purpose. It is appropriately sized for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite an output schema hint (has output schema: true) and complete schema documentation, the description is minimal. It doesn't explain search behavior (e.g., fuzzy or exact), result sorting, or what happens with invalid queries. For a search tool with siblings, additional context would help.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds no additional meaning beyond the schema; parameters are well-documented by the schema itself (e.g., year filter defaults, limit integer). The description's brief mention of 'matching a query' reinforces query usage but doesn't add new context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches WikiCFP for academic conferences matching a query, using specific verb-resource pairing ('Search WikiCFP'). It distinguishes from siblings like conference_details by focusing on search rather than detail retrieval, though it doesn't explicitly differentiate from find_conferences.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for discovering conferences via query but provides no guidance on when to use this over siblings like find_conferences or conferences_by_category. It lacks explanation of search specificity or alternatives, leaving the agent to infer context from sibling names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_conferencesA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It transparently explains that the tool searches WikiCFP, merges and deduplicates results, filters out ended conferences, and sorts by submission deadline. This is clear and actionable for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each adding necessary information: the core action, the search and merging process, and the sorting/filtering details. No fluff—every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 params, output schema present), the description adequately covers behavior and filtering. However, it could briefly mention the output schema's structure (e.g., 'returns conference names, deadlines, and links') since no output schema details are visible to the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all parameters are documented in the schema. The description adds value by explaining how 'keywords' drives the search (each keyword yields results, which are merged) and how sorting works ('sorted by submission deadline'), but does not go beyond schema for other parameters like 'year', 'limit', or 'min_deadline'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Find') and resource ('upcoming conferences matching any of the given keywords'), clearly distinguishing it from sibling tools like 'search_conferences' (which likely searches by different criteria) or 'list_categories' (which organizes by category).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit 'when to use' or 'when not to use' guidance is given, nor are siblings mentioned. The description implies usage (for keyword-based conference discovery), but the agent must infer that this tool is ideal when the user provides keywords, while 'conference_details' is for specific conferences and 'conferences_by_category' for browsing by category.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_categoriesA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden for behavioral disclosure. It explains the ordering (by number of CFPs) and the returned fields, implying a read-only operation. It does not discuss potential size limits or pagination, but for a simple enumeration the transparency is good.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two concise sentences that immediately state the core action and output details. Every sentence adds essential information without redundancy. It is front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and an output schema exists, the description provides all necessary context: the data source (WikiCFP), operational behavior (lists all, ordered), return fields, and a usage hint (URL for browsing conferences). It is fully complete for its simple scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameters, so schema coverage is effectively 100%. According to guidelines, a baseline of 3 is appropriate when schema coverage is high. The description adds value by describing the output structure, but it does not contribute parameter-level semantics since none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists all conference categories tracked by WikiCFP, ordered by number of CFPs, and enumerates output fields. This distinguishes it from sibling tools like 'conferences_by_category' which focuses on conferences within a category, so the purpose is specific and differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains what the tool does (lists categories) but does not provide explicit guidance on when to use it versus alternatives such as 'conferences_by_category' or 'search_conferences'. The usage context is implied rather than stated, and no exclusions are mentioned, making it adequate but lacking direct differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• conference_detailsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries full burden. It states the tool fetches details and includes 'dates, deadlines, website, and categories' – implying a read operation. However, it does not disclose potential errors (e.g., invalid URL, network issues) or any side effects. This is adequate for a simple fetch operation but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the primary action. Every sentence adds value: first defines the core function, second clarifies the input source and output contents. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema, so the description does not need to detail return structure. It mentions included fields (dates, deadlines, website, categories). It does not discuss error handling or prerequisites beyond the URL source. Given the simplicity and presence of output schema, this is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has a single required 'url' parameter with no description (0% coverage). The description adds meaning by specifying that the URL comes from 'search_conferences' and is a WikiCFP page URL. This helps the agent understand the expected format and source beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it fetches 'full details for a conference' from a specific WikiCFP URL. This distinguishes it from sibling tools like search_conferences (which returns URLs) and list_categories (which lists categories). The verb 'fetch' and resource 'details' are specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says 'The URL is the `url` field returned by search_conferences', which implies a workflow: first use search_conferences, then this tool. While it doesn't explicitly exclude other sources or describe when not to use it, the guidance is clear enough for an agent to understand the typical usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• conferences_by_categoryA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Given no annotations exist, the description fully covers behavioral traits: it's a paginated list operation ('one page at a time'), explains the total reachable set ('every conference in it is reachable'), and documents the page size ('about 20 per page'). This provides complete transparency for a read-only browsing tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with zero waste. The first sentence clearly states the function, and the second provides essential usage guidance about iterating pages. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema, the description doesn't need to explain return values. It has 2 simple parameters with 100% schema coverage, no annotations needed, and the description completely covers the tool's purpose, usage pattern, and behavioral scope. No gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents both parameters well (category as a string from list_categories, page as integer with default 1 and 1-based). The description adds context that page iterates to total_pages, but this is implied by pagination behavior. Baseline 3 is appropriate since schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List conferences') and explicitly names the resource ('WikiCFP category' with pagination behavior). It clearly distinguishes itself from siblings like search_conferences or find_conferences by focusing on category browsing rather than search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states this is 'the complete view of a category' and provides explicit instructions on how to use it: iterating page from 1 to total_pages. It implies this is the right tool when you want to see all conferences in a specific category, contrasting with a search tool. Sibling names provide additional context for alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.