Server Quality Checklist

Tool Scores

• read_topicA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, 'Fetch' signals a read-only operation and the description discloses the article contents included in the response. However, it does not address edge cases like missing slugs, permission requirements, or how include_sources controls the citation list beyond the schema default.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence states the action, the input key, and the expected output contents without redundancy. Every phrase adds information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter read operation, the description adequately specifies the return value ('full Markdown article' plus components), which matters because there is no output schema. It could be more complete by mentioning citation conditionality or error behavior, but nothing critical is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so both slug and include_sources are already fully documented. The description adds no parameter-level detail, but this is acceptable because the schema carries the burden.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Fetch') and resource ('full Markdown article for a topic slug'), and names the returned components: infobox, related topics, and citations. This clearly differentiates it from search/discovery/list siblings and from mutation tools like create_topic and edit_section.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'full Markdown article' implies this is the retrieval step after finding a slug, but the description does not explicitly say when to use it over search_topics or discover_topics, nor does it state exclusions. Usage is inferred rather than directly guided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_topicsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It usefully explains that the search covers article titles and text and that results are slugs and summaries, but it does not disclose result ordering, matching behavior, or explicitly confirm the operation is read-only. These are gaps, though not severe for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. The first sentence front-loads the tool's scope, and the second sentence explains the return value and the natural next step. Every phrase earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-parameter search tool with no output schema, the description adequately covers what results look like and how to retrieve full articles. It could add explicit statements about result ranking or pagination behavior, but nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents query and limit with descriptions, defaults, and bounds. The description adds that query applies to titles and text, but no additional parameter-level meaning is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb, resource, and scope: 'Keyword search over Smartipedia article titles and text.' It also states what the tool returns (slugs and summaries), which clearly distinguishes it from read_topic and other sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it via 'Keyword search,' but it does not explicitly contrast it with alternatives like discover_topics or list_missing_topics. It does provide useful follow-up guidance by telling the agent to call read_topic with a returned slug.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• preview_phraseA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are not provided, so the description carries the full burden. It usefully discloses that the operation is cheap/fast, produces a short AI explanation, and does not generate a full article. It does not disclose the return format or whether any side effects occur, but for a simple preview utility the core behavior is reasonably clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no filler. The first states what the tool does, and the second adds cost characteristics and the recommended decision use case. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a simple one-parameter preview with no output schema, and the description covers what it returns ('short AI explanation'), how it behaves ('cheap and fast'), and why to use it. It could more explicitly describe the output format or mention alternatives, but nothing essential is missing for correct selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides 100% coverage for the only parameter, text, describing it as 'the phrase or highlighted text to explain.' The description's 'any phrase' adds no new meaning beyond the schema, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Get'), a clear resource ('short AI explanation of any phrase'), and a boundary ('without generating a full article'). This clearly distinguishes preview_phrase from the creation-focused siblings and makes its purpose immediately intelligible.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to use the tool: 'use it to decide whether a topic is worth creating.' It also contrasts itself with full article generation, which implies the main alternative. However, it does not explicitly name sibling tools or state conditions where another tool should be chosen instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• discover_topicsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals the semantic/vector search nature and optional filtering, which is helpful, but does not state that the operation is read-only, describe result ranking or format, or mention any side effects. For a search tool this is acceptable but not richly transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no wasted words. The core mechanism ('semantic/vector search') is front-loaded, followed directly by actionable usage guidance. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with a fully documented schema and clear usage guidance, the description is largely complete. It could note that the operation is read-only or clarify that results are relevance-ranked, but these are minor gaps given the low-risk nature of the task and the absence of an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all six parameters are already documented with descriptions. The description adds only the generic phrase 'optional filters,' which does not meaningfully enhance the schema's parameter explanations. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'Semantic/vector search' with a specific resource (topics) and optional filters, distinguishing it from the likely keyword-based sibling search_topics. The contrast with 'keyword search' makes the tool's role immediately understandable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to choose this tool: when keyword search misses or when browsing by meaning rather than exact wording. This effectively communicates the condition for using the tool versus the keyword-search alternative, even without naming the sibling explicitly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• edit_sectionA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the behavioral burden. 'Replace' clearly signals mutation, and the expected_revision guidance adds concurrency context. However, the schema already documents the rejection-on-stale-revision behavior, so the description adds limited new behavioral disclosure beyond what the structured fields provide; it does not mention permanence or failure modes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, no filler, with the core action front-loaded and the most important usage caveat (expected_revision) placed at the end. Every sentence earns its place, and the structure makes the tool easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a straightforward edit tool, the description covers what it does, when to prefer it, and how to avoid concurrency issues, while all six parameters are documented by the schema. It does not mention the return value or what happens if the section does not exist, but that is a minor gap given the full schema coverage and simple operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description repeats the expected_revision purpose from the schema rather than adding meaningful new parameter semantics. It does not explain formatting requirements for content or section matching beyond what the schema already states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Replace one section of an article with corrected Markdown.' It clearly distinguishes itself from create_topic by noting it is preferred when fixing an error, so an agent can tell it apart from siblings without reading schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to use this tool ('when you find an error') and names the alternative it should be preferred over ('creating a duplicate topic'). It also gives concrete guidance to pass expected_revision from read_topic to prevent concurrent edits, which is a clear, actionable usage rule.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_missing_topicsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the behavioral burden. It discloses that results are limited to nonexistent searched topics and that they are ranked by demand, which is useful behavioral context. It does not disclose output shape or pagination behavior, but this is a simple read-only list operation, so the missing detail is minor.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with no filler. The first sentence defines what the tool returns and the ordering, and the second sentence explains why it matters. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional parameter and no output schema, the description is largely complete: it states the result set, the ranking, and the intended use. The only minor gap is that it does not describe what fields the returned topics include or how demand is quantified, but this is unlikely to prevent correct invocations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully documents the single 'limit' parameter with a clear description ('How many to return'), defaults, and constraints. The tool description adds no additional parameter semantics, but with 100% schema coverage the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The title and description together state a specific action on a specific resource: listing topics that people searched for but that don't exist yet. This clearly distinguishes it from siblings like search_topics, discover_topics, and create_topic, since it is specifically about missing topics ranked by demand.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool: it is 'the highest-leverage queue for deciding what to write next.' It does not explicitly name alternatives or state when not to use other sibling tools, but the intended use case is clear enough for an agent to select it appropriately.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_topicA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does well: it discloses the web search + LLM mechanism, approximate latency (~15s), the fallback to an existing article, and the daily rate limit. This gives the agent essential expectations about cost, behavior on duplicates, and operational constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise, front-loaded sentences convey the core operation, key constraints, latency, duplicate behavior, and a usage hint. Every clause earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter tool, the description covers the essential behavioral context: creation method, latency, rate limit, and duplicate handling. It does not describe the exact return value shape, but since no output schema exists, a small gap remains.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents the single 'title' parameter with 100% coverage. The description adds no additional parameter-level semantics beyond restating that the tool creates from a title, which aligns with the baseline of 3 when schema coverage is high.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Generate a new sourced encyclopedia article from a title') and distinguishes it from read/search/discover sibling tools. It also reveals the duplicate-handling behavior, making the tool's exact purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives strong usage context: use this to create articles, avoid calling when the topic already exists, and search first due to the daily rate limit. It does not explicitly name 'search_topics' as the alternative, but the instruction to search first effectively routes the agent to the right sibling tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.