Server Quality Checklist

Tool Scores

• list_companiesA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior. It mentions pagination, substring match for industry, and exact values for accounting_standard. However, it does not describe default ordering, return fields, or whether the tool is read-only. For a listing tool without output schema, more behavioral detail would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences and 35 words, extremely concise with no redundant information. It front-loads the core action ('List companies filtered by...') and includes essential details without extraneous content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 4 optional parameters, no output schema, and no annotations, the description covers filtering and pagination adequately. However, it lacks details about the return structure (e.g., fields returned) and does not clarify default behavior when no filters are applied. The universe size context is helpful but not sufficient for full completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so each parameter has a schema description. The description adds value by clarifying that industry is a 'substring match' and providing examples ('Automobiles', 'Pharmaceuticals'), and listing all accounting_standard values. This goes beyond the schema, enhancing parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists companies with filtering options. It specifies the verb 'list' and the resource 'companies'. While it does not explicitly contrast with siblings like compare_companies or search_company, the mention of 'browse the ~3,600-company universe' implies a broad browsing use case, distinguishing it from more specific tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says 'Use to browse the ~3,600-company universe,' providing some usage context. However, it does not specify when not to use this tool or mention alternatives directly. The sibling names give implicit guidance, but explicit exclusion is missing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_financialsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description effectively discloses behaviors: returns up to 17 items, in JPY, includes metadata (accounting standard, consolidation, fiscal year, source document), and lists unavailable items with a reason. It lacks mention of read-only status or rate limits but provides solid transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences convey all essential information: what is retrieved, quantity, currency, metadata, and behavior for missing items. No redundancy or filler; every sentence serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description thoroughly explains return values: financial statement components, item count, currency, metadata fields, and handling of unavailable items. It is self-contained and adequate for agent understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with the 'code' parameter already described as a 4-digit securities code. The description adds a concrete example ('7203' Toyota) but does not add significant new semantics beyond the schema, so baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves financial statements (income, balance sheet, cash-flow) with metadata for one company using a 4-digit code. It specifies up to 17 items in JPY and distinguishes from siblings by focusing on a single company's financials.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for getting a specific company's financials but does not explicitly exclude use cases or compare to siblings like compare_companies for multiple companies. Usage context is implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_companiesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description is the sole source. It states it computes derived ratios and returns metrics in JPY, but does not disclose error handling, rate limits, or behavior for missing data. Adequate but could be more transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences: first states purpose and default metrics, second adds derived ratios and currency. No wasted words, front-loaded with key info.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description covers returned metrics and currency but omits output structure (e.g., array of company objects). For a comparison tool with moderate complexity, it is mostly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema documentation covers both parameters fully (100% coverage). The description adds value by explaining default metrics, derived ratios, and currency, providing context beyond the schema's property descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool compares multiple companies side by side on selected metrics, lists default metrics (revenue, operating_income, etc.), and notes derived ratios and currency (JPY). This clearly distinguishes it from siblings (get_financials for single company, list_companies for listing, search_company for search).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly indicates when to use the tool (for comparing companies on metrics) but does not explicitly state when not to use it or provide exclusions. However, given sibling tool names, the usage context is implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_companyA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It describes a search operation returning data, which is non-destructive, but does not explicitly state read-only behavior, auth requirements, or rate limits. The description is adequate but lacks deeper behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise, consisting of two sentences with no unnecessary words. It is front-loaded with the core action and then provides usage guidance. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of output schema, the description mentions the returned fields (code, English name, industry, accounting standard), which is complete for a search tool. The parameters are well-described, and sibling tools are available in context. Minor gap: no mention of pagination or max results beyond the limit parameter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with both parameters described in the schema. The description adds value by explaining that the query can be a name in English or Japanese or a 4-digit securities code, and specifies the return fields. This goes beyond the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches listed Japanese companies by name or code, returning matching companies with specific fields. It uses a specific verb 'search' and resource 'company', and distinguishes from sibling tools like compare_companies, get_financials, and list_companies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states 'Use this to resolve a name to a code,' providing clear usage guidance. It does not explicitly list exclusions or alternatives, but the context signals for sibling tools help. Overall, the usage context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.