Server Quality Checklist

Tool Scores

• list_indexed_companiesA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It mentions it returns a list and specifies the return fields, but it does not explicitly state that it is a safe read-only operation, nor does it describe edge cases like sorting, pagination, or exact limit enforcement. Some context is added, but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, starting with a clear one-sentence purpose, followed by Args and Returns sections. Every line provides useful information without redundancy, making it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with two optional parameters and an existing output schema, the description covers the core purpose and parameter semantics. However, it omits any usage guidance and does not mention behavioral limitations, making it slightly incomplete for an agent that must rely solely on this text.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds rich meaning beyond the bare input schema. It explains that query is a partial-match filter on company name and that omitting it returns all companies, and that limit sets the maximum number of items. This is exactly what the schema lacks, making parameters self-explanatory.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves a list of indexed companies ('インデックス済み企業の一覧を取得します'), with a specific verb ('取得') and resource ('一覧'). It is distinct from sibling tools like search_filings and compare_sections by focusing on listing companies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives. It only explains the function and parameters, leaving usage context implied. No exclusions, prerequisite steps, or alternative scenarios are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_filingsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the burden of behavioral transparency. It discloses the return format (list of matched documents with company name, section, and snippet) and parameter behaviors like default limit. However, it does not mention potential caveats such as rate limits, auth requirements, or data scope limitations. The description accurately portrays a read-only search operation but could include more behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured with an Args/Returns format. Every sentence contributes useful information—the purpose statement, parameter explanations, and return description—without repetition or fluff. It is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (4 parameters, no annotations, and an output schema), the description provides a complete picture: what the tool does, all parameter semantics, and the return structure. It also includes a concrete query example. The only minor gap is a lack of error-handling or edge-case information, but this is not critical for a straightforward search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no descriptions (0% coverage), so the description fully compensates by detailing each parameter: query with an example, section with an explicit list of allowed values, company as a partial-match filter, and limit with a default value. This adds substantial meaning beyond the bare schema fields, making parameter semantics excellent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs full-text search of securities report text (有価証券報告書のテキストを全文検索します). It uses a specific verb and resource, and the name 'search_filings' aligns with its function. It distinguishes from siblings like get_filing_section and compare_sections by focusing on searching across filings, though it doesn't explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains what the tool does but does not provide explicit guidance on when to use it versus sibling tools such as get_filing_section or compare_sections. Usage context is implied: use it to find filings by content, but no 'when not to use' or alternative recommendations are given. This makes the usage guidelines adequate but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_filing_sectionA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral transparency burden. It discloses the return type (full text and metadata) and provides a helpful example for doc_id. However, it does not mention error handling behavior, whether sections may be missing, rate limits, or any side effects. It is adequate for a simple read operation but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a one-sentence summary, followed by Args and Returns sections. Every line provides necessary detail without redundancy or fluff. It is appropriately sized for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description adequately covers the input parameters and return type, and an output schema exists, so exhaustive return field documentation is unnecessary. It lacks discussion of edge cases or usage scenarios, but for a straightforward getter with two required parameters, the context is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no descriptions for its two parameters, but the description fully compensates by explaining doc_id as an EDINET document ID with a concrete example, and section as a section key with an enumerated list of valid values. This adds significant meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves the full text of a specific section from a filing, using the verb '取得します' (retrieve) and identifies the resource as a filing section. The provided section keys (risk_factors, md_and_a, etc.) further distinguish it from sibling tools like search_filings or compare_sections.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied by the purpose: one would use this when needing section text from a known filing ID. However, there is no explicit guidance about when to use this tool versus alternatives, no exclusions, and no prerequisites mentioned beyond the required parameters.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_sectionsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the behavioral transparency burden. It does so effectively by stating that both texts, character count differences, new keywords, and disappeared keywords are returned, and by noting the default section key. It doesn't discuss side effects or auth, but for a comparison/read tool this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured with separate Args/Returns sections. The example adds context without redundancy. It earns its place, though a slightly more explicit usage scenario could make it even more actionable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 params, output schema existing), the description covers purpose, parameters, and return values, making it mostly self-contained. It lacks explicit exclusion criteria or prerequisite conditions, but the provided information is sufficient for an agent to invoke it correctly in the described use case.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates by documenting all three parameters in the Args section: doc_id_1 as source, doc_id_2 as target, and section with a default of risk_factors. This adds meaningful semantic value beyond the bare schema property names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies a clear verb+resource: '2つの書類の同一セクションを比較します' (compare the same section of two documents) with a concrete example (prior vs current risk description). This clearly distinguishes it from sibling tools like get_filing_section, which retrieves a single section.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides an example of when to use: comparing the same section across two documents (e.g., prior period vs current period). However, it does not explicitly state when not to use it or mention alternatives, though the purpose itself differentiates it from related tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_index_statsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. It discloses that the tool returns statistics (e.g., indexed document count, company count, total characters), which implies a read-only operation. While it does not explicitly state side effects or access requirements, the nature of the tool is low-risk and adequately transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence followed by a brief 'Returns' block. It is front-loaded with the action, contains no redundant information, and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with an output schema, the description is complete. It provides an overview of the returned statistics (indexed document count, company count, total characters) and the output schema covers detailed field definitions. No further context is necessary for correct usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so the baseline for parameter semantics is 4. The description adds no parameter information (as none exist) and instead focuses on the return values, which is appropriate for a parameterless tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves search index statistics using a specific verb ('取得します') and a distinct resource ('検索インデックスの統計情報'). It distinguishes itself from sibling tools like search_filings, list_indexed_companies, and compare_sections, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for obtaining aggregate index statistics, but it does not explicitly state when to use this tool versus alternatives. There is no mention of such alternatives or situations where another tool would be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.