Server Quality Checklist

Tool Scores

• list_sessionsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It only states 'list all' with optional filtering, but does not mention whether it is read-only, whether it returns all sessions across users, or any rate limits or side effects. The default limit of 20 is not disclosed in the description, leaving important behavioral traits unaddressed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence: 'List all research sessions with optional filtering.' It is front-loaded with the action, contains no filler, and is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with two optional parameters fully documented in the schema, and the description conveys the core purpose. However, there is no output schema and no mention of return format, pagination behavior, or what fields are included. The absence of annotations and the lack of detail in the description leave some gaps, but the schema compensates for parameter information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for both parameters (limit and status), so the baseline is 3. The description's 'optional filtering' adds no specific detail beyond what the schema provides. It does not name the filters or explain their usage, so it adds minimal value over the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists research sessions and mentions optional filtering. This distinguishes it from siblings like start_research and get_research_status, which create or retrieve individual sessions. The verb 'list' and resource 'research sessions' are specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives. There is no mention of get_research_status for single-session status, no exclusions, and no context indicating whether it should be used for active sessions only. The intended use is only implied by the name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_research_statusB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states a read-like action ('Get') but does not disclose details such as possible status values, whether the session must exist, error behavior, or any side effects. The minimal wording leaves significant behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that immediately states the purpose. It is succinct and front-loaded, with no unnecessary words or filler, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description adequately conveys the core function. However, it could be slightly more complete by hinting at what 'status' entails (e.g., running, completed, failed) or how to handle missing sessions, but the current form is sufficient for basic selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage for the single parameter (sessionId) with a clear description ('Session ID to check'). The tool description adds no additional parameter semantics beyond reiterating that it pertains to a research session, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get the current status of a research session' uses a specific verb ('Get') and a clear resource ('current status of a research session'), which distinctly identifies the tool's function. It is clearly differentiated from sibling tools like start_research or list_sessions, which have different actions or scopes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives or any prerequisites or context. It simply states the action without explaining scenarios where this tool is preferred over siblings like list_sessions or get_research_status.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_sourcesB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It mentions multi-provider search but does not explain result merging, pagination, output format, rate limits, or any side effects. The behavioral coverage is minimal and leaves the agent guessing about important runtime characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that wastes no words. It conveys the core purpose and scope efficiently, earning a top score for conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has four parameters, no annotations, and no output schema, the description is too sparse. It does not explain return behavior, result ordering, default provider behavior, or how 'minRelevance' and 'limit' influence results. The agent would need to infer too much, making the description incomplete for effective invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all four parameters. The description adds almost no extra parameter meaning beyond the phrase 'multiple providers', which is already reflected in the 'sources' parameter. Therefore, the baseline of 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Search') and resource ('sources'), and further scopes it to 'multiple providers (web, academic, GitHub, etc.)'. This distinguishes it from sibling tools like start_research or cross_verify, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (whenever a cross-provider source search is needed) but provides no explicit exclusions or alternatives. It does not mention when not to use it or compare it to siblings like cross_verify, so the guidance is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cross_verifyA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing behavioral traits. It only states that the tool returns verification confidence and conflict analysis, which implies a read-only operation, but it does not explicitly say whether it modifies any state, requires specific permissions, or has limitations (e.g., handling missing/invalid sources). The lack of explicit safety or side-effect information leaves the agent without a clear picture of consequences beyond the output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences that directly state the tool's function and output. It is efficient, front-loaded, and contains no redundant or filler content. Every word contributes to conveying the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and simple inputs, the description gives a high-level overview but lacks detail on the format or semantics of the returned 'verification confidence' and 'conflict analysis'. It also does not explain behavior when no sources are provided or how claims are processed. Given the absence of an output schema and annotations, the description is adequate but not fully complete for an agent to anticipate results or edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage with descriptions for all three parameters (topic, claims, sources). The tool description adds minimal additional meaning beyond what the schema states—it merely mentions 'claims' and 'optional source URLs' which are already in the schema. Thus, the description does not significantly enhance parameter understanding beyond the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's purpose: cross-verifying a list of claims against optional source URLs. It uses a specific verb ('cross-verify') and states the resource (claims/sources) and the deliverables (verification confidence and conflict analysis). This distinguishes it from sibling tools like search_sources or start_research, which focus on gathering or managing research.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for verifying claims and optionally using source URLs, but it does not explicitly state when to use this tool versus alternatives or mention any exclusions. There is no guidance on prerequisites (e.g., needing sources first) or when not to use it, so the usage context is only implied rather than directly instructed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• save_to_storageA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It does not mention side effects like file overwriting, permissions, or what happens if the session is invalid, nor does it explain the meaning of 'storage provider' beyond formats. This is a significant gap for a save operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that states the action, object, and target without any filler or redundancy. It earns its place and is easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and the schema fully describes all parameters, but the lack of annotations and output schema means the description should provide more context about the save behavior, such as default destination or error handling. It is minimally adequate but leaves open questions about side effects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for all three parameters, so the baseline is 3. The description adds limited extra meaning—it restates the markdown/json enum values, but does not explain the destination parameter or provide syntax. The schema already carries the semantic weight.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Save'), identifies the exact resource ('previously generated research document'), and names the target ('storage provider') with explicit formats (markdown or json). This clearly distinguishes it from the sibling research and status tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'previously generated research document' implies the tool should be used after research is complete, but there is no explicit guidance on when to prefer this tool over alternatives or any stated exclusions. The context is present but not stated as a rule.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_researchA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states that the tool performs automatic multi-source search, collection, and cross-verification, which are meaningful behaviors. However, it does not disclose the session lifecycle, whether it is asynchronous, how to track progress via get_research_status, or what happens with user-provided URLs or storage settings.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that precisely states the tool's purpose without any fluff or repetition. Every word contributes meaning, and the description is appropriately concise for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides a high-level overview but lacks information about the output of the session, how to retrieve results, and the asynchronous nature of research. Since there is no output schema, the description should explain what the caller can expect in return, but it does not. However, the complete parameter schemas partially compensate for this gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides descriptions for all four parameters, including enums and defaults, so the schema description coverage is 100%. The tool description adds no additional parameter-specific detail, which corresponds to the baseline score of 3 for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb-action ('Start a new research session') and clearly differentiates this tool from its siblings by describing the automatic multi-source search, collection, and cross-verification process. It identifies exactly what resource is being acted upon (a research session) and the high-level workflow, making the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used to begin a research session, but it provides no explicit guidance on when to choose this tool over alternatives like search_sources or cross_verify. There are no prerequisites, exclusions, or alternatives mentioned, leaving usage context to be inferred.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.