Server Quality Checklist

Tool Scores

• search_clientsB
  3.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark this as read-only and idempotent, so the description's note that omitting search returns a list adds a real behavioral detail. It explains that the tool returns a list even when search is empty, which is not obvious from the schema. However, it does not disclose other behaviors like the meaning of default limit or archived handling, though annotations lower the burden.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences with no wasted words. It states the core purpose and one key behavior efficiently, making it easy to scan and digest. Ideal conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite an output schema, the description is incomplete for a tool with three optional parameters. It only covers the search omission behavior and provides no explanation of limit or archived filters. It also lacks any indication of when to use this tool versus other related tools, leaving the agent underinformed for decision-making.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description is the only source of parameter meaning. It only implies the role of 'search' by saying what happens when it is omitted (a list is returned), but it does not explain 'limit' (pagination) or 'archived' (include archived clients), which are non-obvious. This is a significant gap for a 3-parameter tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'searches MakeLeaps clients' with a specific verb and resource, and notes that omitting search returns a full list. This gives a clear sense of what the tool does. However, it does not explicitly distinguish itself from the sibling tool get_client, relying on the inherent contrast between 'search' and 'get'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description offers no guidance on when to use this tool versus alternatives like get_client or create_organization_client. It only describes the default behavior when search is omitted, but does not mention when to apply the archived filter or how limit works, leaving the agent without clear selection criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_documentsB
  3.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare this as a safe read-only operation, so the description does not need to repeat that. It adds valuable context by clarifying that the date parameters represent the issue date range, which is a behavioral detail not present in annotations. This enhances transparency beyond the structured data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, consisting of two short sentences. It front-loads the core purpose and adds the essential date clarification without any redundant words. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 8 parameters and an output schema, the description is minimal but functionally adequate. It specifies the document types and date range, but lacks details on other filter behaviors or how results are returned. However, the output schema and annotations fill some gaps, so it's above average but not comprehensive for such a parameter-rich tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter understanding. It only explains the date semantics ('日付は発行日の範囲を表す'), leaving other parameters like paid, sent, search, and client_mid without explicit explanation. While some names are self-explanatory, client_mid is ambiguous, and the description does not clarify filtering logic.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('条件検索' = search) and resource ('見積書または請求書' = quotes/invoices), making the tool's purpose evident. It does not explicitly differentiate from siblings like search_clients, but the document type restriction provides reasonable distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description offers no guidance on when to use this tool versus alternatives. It does not mention related tools such as get_document for retrieving a single document or create_quote/create_invoice for creation. The usage context is only implied by the tool's purpose, with no explicit exclusions or recommendations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_quoteA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=false, destructiveHint=false, and idempotentHint=true. The description adds important context that the quote is created but not sent, which is beyond annotation-provided safety profiles. This is meaningful behavioral disclosure for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, front-loaded with the core action. Every word earns its place, and it avoids redundant information already present in annotations or schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema and annotations, the description is too thin for a complex create tool with 14 parameters. It does not explain prerequisites (e.g., client_mid must exist), how fields like line_items map to MakeLeaps concepts, or any expected workflow. The no-send detail is helpful but leaves major gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 21%, with most of the 14 parameters lacking descriptions. The tool description does not compensate by explaining any parameters. For a create operation with 8 required fields, this leaves significant ambiguity for the agent on how to populate required fields correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb '作成する' (create) and the resource '見積書' (quotation) in MakeLeaps, distinguishing it from sibling tools like create_invoice. The explicit '送付は行わない' (does not send) adds a critical scoping detail.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for creating a quote without sending, but provides no explicit guidance on when to use this tool vs alternatives (e.g., create_invoice) or any prerequisites like client/template existence. The no-send clarification is a mild usage hint but not sufficient for a clear distinction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_documentA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, so the safety profile is clear. The description adds the scope (quotes/invoices) but no additional behavior such as error handling or required permissions. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, clear sentence in Japanese with no extraneous words. Front-loaded with the action and scope.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter read tool with a readOnlyHint annotation and an output schema, the description provides sufficient information about the input and purpose. The only missing piece is alternative guidance, already penalized in usage_guidelines.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only states document_mid is a string with no description. The tool description compensates by identifying it as a MakeLeaps ID, giving the parameter semantic meaning beyond type. However, it does not describe format or examples, so not a full 5.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves details of quotes or invoices by MakeLeaps ID, with a specific verb (取得する) and resource (見積書/請求書). It is distinct from sibling search_documents which would search without an ID.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs search_documents or other siblings. It implies an ID is required but does not state that search_documents should be used when the ID is unknown.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_invoiceA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover idempotency and non-destructiveness. The description adds a key behavioral trait beyond annotations: the invoice is created but not sent (送付は行わない), preventing misuse where the agent expects sending. It does not disclose return details, but the output schema likely covers that.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with two essential pieces of information: the primary action (create invoice) and a critical limitation (does not send). Every word contributes value, making it appropriately concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having 16 parameters and multiple required fields, the description is minimal and does not explain the invoice-creation workflow, required data, or any special rules. It only clarifies the non-sending aspect. The low schema description coverage and absence of parameter guidance make this incomplete for a complex operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 31%, and the description provides no parameter-level explanations. Field names like operation_key, document_template, client_contact_mid, and tax_type are left without clarification, so the agent must infer meaning from names alone. With low coverage, the description fails to compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb and resource: 'Create an invoice in MakeLeaps' (請求書を作成する). It also adds a scope limitation 'does not send' (送付は行わない), which distinguishes this tool from sending-related actions and sibling tools like create_quote.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description establishes clear context (creating an invoice) and explicitly states a when-not: 'does not send it' (送付は行わない), implying this tool should not be used when the goal is to send an invoice. However, it does not explicitly name alternatives or specify when to choose this over siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_organization_clientA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds significant behavioral detail beyond the annotations: it precisely defines the idempotency semantics (same key+same input → saved result; same key+different input → rejected). This goes beyond the generic idempotentHint annotation and clarifies the tool's behavior without contradicting any annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences, with the purpose stated first and the crucial idempotency behavior second. Every sentence carries important information, and there is no filler or repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has 10 parameters including nested objects, the description is somewhat thin. It focuses almost entirely on operation_key and does not mention the required organization_name, external_id, or address structure. The output schema and annotations provide some structural context, but for a parameter-rich creation tool, more guidance on typical usage and parameter relationships would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is very low (0% per context, or at most 2 of 10 top-level parameters have descriptions). The description fully explains operation_key, which is critical, but it does nothing to clarify the remaining parameters such as external_id, lang_code, or tax_registration_number. With low coverage, the description must compensate more broadly, and it does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with '法人の取引先を作成する' (creates a corporate client), which is a specific verb-resource pairing that clearly states the tool's function. This distinguishes it from sibling tools like create_quote and create_invoice, which create different entity types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context that this tool is for creating corporate clients and includes specific usage guidance for operation_key: it must be unique per caller, reusable, and re-executing with the same key and input returns the saved result while reusing the key with different input is rejected. However, it does not explicitly mention when to use this tool over alternatives or provide exclusion criteria, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_document_templatesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds the scoping to document types but no additional behavioral details such as pagination or return format, which is acceptable given the output schema exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the core purpose. Every word is necessary, with no redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With one clear parameter, rich annotations, and an output schema, the description adequately covers the tool's purpose for a simple list operation. It could mention the scope of templates (e.g., all templates vs. filtered), but the openWorldHint and straightforward nature make it sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter document_type is defined by an enum in the schema, and the description restates the allowed values ('quotes or invoices') in natural language. This compensates for the 0% schema description coverage, as the agent can map the parameter to the intended document types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies a clear resource (MakeLeaps templates) and action (取得=get), scoped to quotes or invoices. This distinguishes it from sibling tools, which focus on clients and document creation, not template retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for retrieving templates for quote or invoice documents, providing clear context. It does not explicitly mention alternatives or when not to use, but no sibling tool offers template retrieval, so the purpose alone signals the appropriate situation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_clientA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the description doesn't need to cover safety. It adds useful context by specifying that both client and default contact details are returned, but doesn't disclose edge cases like not-found behavior. This matches the get_calls calibration baseline.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no redundancy. Every word contributes value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple get-by-ID tool with rich annotations and an output schema, the description is complete. It covers the resource scope and parameter semantics, and nothing else is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage for client_mid, but the description compensates by calling it the MakeLeaps ID (MakeLeaps IDを指定して). This gives the parameter its necessary meaning, and with only one parameter, this is sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (取得する), the resource (取引先とデフォルト連絡先), and the exact scope (MakeLeaps IDを指定して). It distinguishes itself from siblings like search_clients (which searches) and create_organization_client (which creates).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you have a MakeLeaps ID, providing clear context. It doesn't explicitly mention alternatives or when-not-to-use, but sibling names like search_clients make the distinction obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.