Server Quality Checklist

Tool Scores

• get_topic_infoA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states 'Get detailed information' implying a read-only operation, but does not disclose behavior such as error handling if the topic does not exist, whether the match is exact or partial, or what 'detailed information' includes. The output schema may cover return structure, but the description itself adds little behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: two sentences, with the first stating the purpose and the second documenting the parameter. It is front-loaded and to the point, though the parameter documentation could be integrated more formally. Overall, it earns its place without waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, output schema exists), the description is minimally complete. It adequately states the purpose and parameter meaning. However, it lacks context about when to use this vs. sibling tools, and does not clarify expected behavior for missing topics or exact matching. The output schema likely covers return details, so the description is sufficient for a basic tool but not rich.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description explicitly explains the sole parameter: 'topic: Topic name to get info for'. This adds meaning beyond the schema, which only defines the parameter as a string with no description. Since schema description coverage is 0%, this compensation is valuable and clear for a single parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get detailed information about a research topic.' The verb 'Get' plus the resource 'research topic' makes the purpose explicit and distinguishes it from sibling tools like list_research_topics (which lists topics) and search_research_data (which searches data).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying a topic name as input, but it does not explicitly state when to use this tool versus alternatives. There is no mention of using list_research_topics for browsing all topics or search_research_data for filtering. The context is clear enough for a simple tool, but exclusions or alternatives are not provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_research_topicA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the destructive scope ('and all its data'), which is a key behavioral trait. However, it does not mention that deletion is irreversible, whether special permissions are required, or potential side effects beyond data deletion. This is a significant gap for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the primary purpose. The Args block is a standard, efficient way to document the parameter. Every sentence earns its place, and there is no redundant or verbose text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple delete tool with one parameter and an output schema, the description covers the core action and scope. It does not explain return values, but the output schema likely covers that. However, it omits a caution about irreversibility, which is especially important for deletion operations. This is a minor gap given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, 'topic', is described as 'Topic name to delete'. This adds minimal meaning beyond the schema's property name and type, providing no details about format, case sensitivity, or behavior when the topic does not exist. Since schema_description_coverage is 0%, the description does not fully compensate for the lack of schema-provided descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'Delete a research topic and all its data.' This specifies the verb (delete), the resource (research topic), and the scope (all its data). It distinguishes itself from sibling tools like save_research_data, search_research_data, list_research_topics, and get_topic_info, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when deletion of a research topic is intended, but it does not explicitly state when to use this tool versus alternatives, nor does it mention any prerequisites or exclusions. There is no reference to sibling tools or conditions under which deletion is appropriate (e.g., 'Use this to permanently remove a topic').

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• save_research_dataA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description carries the full burden of behavioral disclosure. It mentions that a separate DB is created for each topic, which adds useful context, but it does not explain what happens when a topic already exists (append vs. overwrite), whether there are any side effects, or if specific permissions are required. These are significant gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, two sentences plus an args list, with the core purpose front-loaded. Every part contributes meaning; there is no fluff or repetition of schema details. The structure is clean and scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 params, no nested objects) and the presence of an output schema, the description covers the basics well. However, it lacks a critical behavioral detail: what happens when the same topic is used again. This ambiguity could lead an agent to misuse the tool, so the description is not fully complete for this write operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema's parameter descriptions are absent (0% coverage), so the description's explicit explanation of both parameters is essential. It clarifies that 'content' is a list of texts to save and that 'topic' is used for organization and creates a separate DB, adding meaning beyond the raw schema types and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Save') and the resource ('research content to vector database') with a clear purpose ('for future retrieval'). It distinguishes itself from siblings like search_research_data (retrieval) and delete_research_topic (deletion) by focusing solely on the write operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context ('for future retrieval') but does not explicitly state when to use this tool over alternatives, nor does it mention any exclusions. The sibling tools are known from context, but the description itself offers no direct comparison, so the guidance is implicit rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_research_dataA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It does reveal that the search uses semantic similarity (not keyword matching), and the term 'saved research data' implies a read-only operation. However, it does not state whether results are ordered by relevance, whether topic is required, or any limitations. The description gives some behavioral context but not rich detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence followed by a simple list of parameter definitions. It is front-loaded with the main action and avoids redundancy or filler. Every line serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a straightforward search operation with an output schema available, so return values do not need to be explained. The description covers the main purpose, the search method, and all parameters. The lack of explicit usage scenarios is a minor gap, but overall the description is sufficient for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The Args section in the description explicitly defines each of the three parameters: query as the search query, topic as the topic database to search in, and max_results as the maximum number of results. Since the schema itself provides no descriptions for these fields (coverage 0%), this textual description adds essential meaning and compensates for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear action statement: 'Search through saved research data using semantic similarity.' It specifies the operation (search), the resource (saved research data), and the method (semantic similarity), clearly distinguishing it from sibling tools like save_research_data or delete_research_topic.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance on when to choose this tool over alternatives. The verb 'search' implies a use case for finding saved research data, but no direct comparison with sibling tools or exclusions is provided. This is implied usage rather than explicit direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_research_topicsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full transparency burden. It explicitly states it lists 'all available' topics, implying a non-destructive read-only operation. The parenthetical 'vector databases' adds useful context. While it does not mention pagination or side effects, the presence of an output schema covers return values, and the simplicity of a list operation make this sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no redundant words. It is perfectly concise and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 0 parameters and an output schema present, the description fully conveys the tool's function. It tells the agent exactly what will happen when invoked, leaving no ambiguity for a tool of this simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, so the baseline score is 4. There are no additional parameter details needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') with a clear resource ('research topics') and adds clarifying context ('vector databases'). It clearly distinguishes itself from siblings: 'save', 'search', 'delete', and 'get_topic_info', with 'all available' disambiguating from get_topic_info which likely retrieves a single topic.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance about when to use this tool versus alternatives. However, the verb 'list' and the sibling context (save, search, delete, get_topic_info) imply that this tool is for enumerating all topics before other operations, but this is only implied, not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.