Server Quality Checklist

Tool Scores

• execute_finance_toolA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the full burden of behavioral disclosure. It fails to state whether execution is read-only or mutating, what side effects may occur, or how results are returned. The only non-obvious context is the warning about hallucination, which is more usage guidance than behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and includes a highly relevant usage directive. Every sentence adds value, with no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity as a dynamic dispatcher, the description adequately covers the key context: how to invoke it and the prerequisite discovery steps. The presence of an output schema means return values need not be explained. However, it does not address error handling or invalid inputs, which would make it fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, and the description provides minimal added meaning. It implicitly acknowledges tool_name and params ('by name with the given parameters') but does not elaborate on their formats, constraints, or the expected structure of the params object. The description does not compensate for the lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action: 'Execute a specific finance tool by name with the given parameters.' This distinguishes the tool from its siblings (list_finance_categories, search_finance_tools, get_tool_schema) by identifying it as the executor, not a discovery or schema tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit directive: 'Always call search_finance_tools or get_tool_schema first to confirm the correct tool name and parameter shape.' This clearly indicates when and how to use the tool (only after confirmation) and explains the rationale for avoiding parameter hallucination, thereby differentiating it from the discovery tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_finance_toolsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosure. It discloses that this is a read-only search operation across all 43 tools, returns only relevant tools, and includes name, description, and expected parameters. It does not discuss exact matching semantics or edge cases, but it provides substantial behavioral context for a discovery tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences with no wasted words. It is front-loaded with the core purpose, then provides usage ordering, and finishes with a rationale for why the tool avoids loading the full catalog. Every sentence contributes unique value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 2-parameter search tool with an output schema, the description covers purpose, tool categories, return content, and usage order. It does not explicitly explain both parameters in prose, but the output schema and simple schema reduce that burden, making it mostly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description only implicitly clarifies the 'query' parameter as a natural-language request. It does not mention 'top_k', its default, or how it affects the result set. The parameters are simple enough to infer, but the description adds minimal value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('find') and resource ('finance tools most relevant to a natural-language request'), clearly distinguishing this discovery tool from the execution sibling. It explicitly contrasts with execute_finance_tool by stating it returns tool metadata rather than executing, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit sequencing by saying 'Call this FIRST before execute_finance_tool,' which gives clear usage context. It also describes the broad coverage across all 43 finance tools, but it does not explicitly state when to use list_finance_categories or get_tool_schema instead, so it lacks a full alternatives exclusion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_finance_categoriesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It clearly states that the tool lists domains and provides tool counts, and implies a read-only, non-destructive action. For a simple listing tool, this is sufficient transparency; no hidden side effects or additional requirements are relevant.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that states the verb, resource, and key output aspects. It lists examples efficiently without redundancy and earns its place with high information density.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with an output schema present, the description is complete. It covers what the tool does (lists domains) and what it returns (domain names and counts), and the sibling context makes the tool's role clear. No further detail is required.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, which gives a baseline of 4. The description adds meaningful context beyond the empty schema by naming example categories and explaining the output includes tool counts. However, since there are no parameters to describe, the contribution is minimal but acceptable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and names a clear resource ('domains of finance tools'), enumerates example categories, and notes the counts per domain. This distinguishes it from siblings like search_finance_tools, get_tool_schema, and execute_finance_tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The context in which to use this tool is implied (exploration/discovery of available finance domains), but there is no explicit statement of when to prefer it over alternatives or any exclusion criteria. The sibling tools are not referenced in the description, so no when-not-to-use guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_tool_schemaA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It suggests a read-only operation via 'Get' and adds the prerequisite that the tool name comes from search_finance_tools. However, it does not explicitly state error behavior or confirm non-mutating status, leaving some opacity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the main action and integrates the usage guidance efficiently. Every word contributes to understanding the tool's purpose and workflow.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with an output schema, the description adequately covers purpose, usage, and parameter meaning. It lacks explicit error-handling details, but the simplicity and output schema mitigate the gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description adds meaning by explaining that tool_name is the name of a finance tool and should be the value returned by search_finance_tools. This provides essential context beyond the bare 'string' type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and the specific resource ('exact parameter schema') for a single named finance tool. It also differentiates from siblings by indicating it follows search_finance_tools, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use it: 'use after search_finance_tools has told you the tool name'. This provides clear sequencing and implies it is for schema retrieval rather than execution, which distinguishes it from execute_finance_tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.