Server Quality Checklist

Tool Scores

• list_invoicesB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must disclose all behavioral traits. It only states that it fetches a list and can filter by partner name, but does not mention that it's a read-only operation, authentication requirements, pagination, or return format. Significant transparency gaps remain.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, using one sentence to convey the main purpose and a secondary capability. It is front-loaded with the key action and resource, with no wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with two optional parameters and no output schema, the description is minimally adequate but lacks context. It does not describe the return structure, default behavior, or any prerequisites, relying solely on the schema. Without annotations, an agent might need to make assumptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides full coverage for both parameters (limit with default and max, partner_name with partial match details). The description merely restates the filtering capability already documented in the schema, adding no new semantic value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves a list of invoices (請求書の一覧を取得する) and mentions filtering by partner name, which is a specific verb+resource combination. This makes it distinct from siblings like list_deals or create_invoice.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives such as list_deals or monthly_summary. It does not mention use cases, exclusions, or alternative tools, leaving the agent to infer usage solely from the tool name and description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• monthly_summaryA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It states it aggregates and displays, which implies read-only, but it does not explicitly mention no side effects, data coverage, or any limitations. There is no contradiction, but the description adds minimal behavioral context beyond the obvious function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that directly states the tool's purpose without extraneous details. It is appropriately concise for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with two optional parameters and no output schema. The description covers the main purpose but does not explain the return format or any safety context that would normally be expected given the lack of annotations. However, for a low-complexity summary tool, it is reasonably adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers both parameters (year and month) with descriptions and defaults, achieving 100% schema description coverage. The tool description adds no additional parameter semantics, so it does not exceed the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool aggregates and displays a monthly income/expense summary by account category ('月次の収支サマリーを勘定科目別に集計して表示する'). This is a specific verb+resource scope and distinguishes it from sibling tools that list individual transactions or invoices.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for obtaining monthly summaries, but it does not explicitly state when to prefer this over list_deals, list_invoices, or pending_transactions. The context is clear but there are no explicit exclusions or alternative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• authenticateA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must disclose behavioral traits itself. It only states '開始・状態確認' (start/status check) without explaining side effects, such as initiating an OAuth flow, potential redirects, token storage, or what a status check entails. There is no mention of how the output is delivered or failure modes. This leaves significant ambiguity for an authentication tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, consisting of two short clauses. It front-loads the core function and adds a usage hint. Every word is purposeful, with no padding or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is minimal and leaves critical context unexplained. For an OAuth authentication tool with no annotations, no output schema, and no parameters, the agent lacks information about what actions the tool performs, what success/failure looks like, and any prerequisites. The description does not cover the behavioral intricacies of authentication, making it incomplete for nuanced decision-making.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the schema is empty. Per the rubric, the baseline for 0 parameters is 4. The description does not add parameter details because none exist. It appropriately focuses on the tool's purpose and usage context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool's function: 'OAuth認証の開始・状態確認' (starting/checking OAuth authentication status). It identifies a specific verb and resource, and differentiates itself from sibling tools (deals, invoices, transactions) by focusing on authentication. The purpose is unambiguous and distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: '初回認証やトークン失効時に使用' (used for initial authentication or token expiration). This tells the agent when to invoke the tool. It does not explicitly name alternatives, but the sibling context makes the tool's special role evident. The guidance is sufficient though not exhaustive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_dealA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With annotations absent, the description must bear the full burden of behavioral disclosure. It does add useful context: '税区分は勘定科目のデフォルトを自動適用' (tax classification is automatically applied from the account item's default) and '口座指定で明細と自動紐付け' (if account is specified, it auto-links with statement). However, it does not disclose response format, side effects of partner auto-creation, or error cases, leaving gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise Japanese sentences, each with a distinct purpose: main function, account item specification, tax classification, and wallet linking. There is no redundancy or filler, and the main purpose is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a create operation with no output schema and no annotations, the description covers the essential behaviors: account item naming, automatic tax classification, and wallet linking. It does not mention response format, but given that all parameters are documented in the schema and the tool is not overly complex, the description is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, but the description adds cross-parameter meaning beyond the schema: it explains that account_item is specified by name, tax classification is derived from the account item's default, and wallet_name triggers auto-linking. This extra context helps the agent understand parameter interactions, not just individual descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: '取引（仕訳）を登録する' (register a transaction/journal entry). It also differentiates from siblings by describing unique behaviors like '口座指定で明細と自動紐付け' (auto-linking with statement when account is specified), which is distinct from create_invoice or list_deals.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no explicit guidance on when to use this tool versus alternatives. It does not mention create_invoice for invoice creation or list_deals for querying transactions. The only implied usage is the verb '登録する' (register), so an agent has no context for selecting between similar tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_invoiceA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It discloses helpful traits: creation results in a draft, due date is auto-computed from config rules, and multiple line items are supported. However, it does not mention return values, auth requirements, or side effects, which are gaps for a create tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences in Japanese, front-loaded with the main action and state ('Creates a draft invoice'). Every sentence provides distinct useful information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for a simple create tool with a detailed schema, but it lacks an explanation of the return value or post-creation behavior (e.g., what the draft invoice contains, how to access it later). Since there is no output schema, this gap makes the description incomplete for an agent needing to know the tool's result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining that due_date is automatically calculated (not user-specified in many cases) and that items support multiple lines. This goes beyond the schema's field-by-field descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates (作成する) an invoice (請求書) in draft state (下書き), and specifies the key identifier (customer name). This distinguishes it from siblings like list_invoices (viewing) and create_deal (different resource).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for creating draft invoices and notes that due date is auto-calculated, but it provides no explicit guidance on when to use this tool vs alternatives. No exclusions are mentioned, so usage context is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_dealsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It conveys a read-only list operation through '取得' (retrieve) and mentions filtering capabilities, but it does not disclose potential behaviors like pagination limits (though limit is in schema), ordering, or authentication requirements. This is a moderate gap for a read-only tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary purpose and followed by concrete use cases. Every sentence earns its place, with no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 optional parameters, no output schema, and no annotations, the description provides a clear overview of purpose and intended use cases. It lacks explicit details about the return structure or default behaviors, but for a list retrieval tool with schema-covered parameters, it is sufficient to understand and invoke the tool correctly in most contexts.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds minimal meaning beyond the schema: it mentions filterable by '勘定科目名や期間' (account item name and period), which mirrors the schema fields. The other parameters (type, limit) are left to the schema, so no extra value is added.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear action: '登録済みの取引（仕訳）一覧を取得する' (get a list of registered transactions), with a specific resource and scope. It also distinguishes itself from siblings by mentioning its use for monthly_summary breakdowns and duplicate checks, which makes its purpose unique.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit use cases: 'monthly_summaryの内訳確認や、仕訳の重複チェックに使う' (used for checking monthly_summary breakdowns and duplicate journal entries). This gives clear when-to-use guidance, but it does not explicitly state when not to use it or name alternative tools for other scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pending_transactionsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It explicitly indicates a read-only retrieval ('取得する') and scopes the data to 'unprocessed' items. No side effects are mentioned, and for a simple listing operation, this level of disclosure is sufficient. It does not contradict any annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that states the action and purpose, then notes filtering options. Every word is relevant, and there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list retrieval tool with a well-covered schema, the description is complete enough. It explains what is returned (a list), the scope (unprocessed), and the optional filters. The lack of an output schema does not require a detailed return-format description since '一覧' implies a list. It could have gone further by mentioning default date ranges, but those are already in the schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers all four parameters with descriptions and defaults (100% coverage), so the baseline is 3. The description's mention of filtering by account name and period merely restates what the schema already documents without adding new semantic meaning, such as format details or relationships between parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('取得する' / retrieve) and a clear resource ('未処理の口座明細一覧' / unprocessed account statements), which distinguishes it from sibling tools like list_deals and list_invoices. It also mentions filtering capabilities, making its function unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies the usage context: when you need a list of pending/unposted account statements, optionally filtered by account name and date range. It does not explicitly name alternatives or say when not to use it, but the distinct resource and purpose provide adequate guidance without confusion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.