Server Quality Checklist

Tool Scores

• get_harvest_file_download_urlC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool returns a 'ダウンロード用URL（リダイレクトURL）' (download URL/redirect URL), which hints at a read-only operation that might involve redirection. However, it lacks details on authentication needs, rate limits, error handling, or whether the URL is ephemeral. For a tool with no annotations, this is insufficient behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and concise, with a clear purpose statement followed by separate 'Args' and 'Returns' sections. Each sentence adds value: the purpose, parameter list, and return value. There's no redundant information, and it's front-loaded with the main action. Minor improvement could include more detail in parameter explanations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 2 parameters with 0% schema coverage and an output schema (implied by 'Returns'), the description is moderately complete. It covers the purpose, parameters, and return value, but lacks usage guidelines, behavioral details (e.g., auth, errors), and deeper parameter semantics. The output schema likely defines the URL structure, so the description doesn't need to elaborate on returns, but overall context is incomplete for safe invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description includes an 'Args' section that lists parameters ('scope' and 'path') with brief Japanese explanations, adding meaning beyond the input schema (which has 0% description coverage). However, the explanations are minimal ('スコープ' for scope, 'ファイルパス' for path) and don't detail format, examples, or constraints (e.g., scope values like 'private' or 'operators/{operator_id}'). With low schema coverage, this partially compensates but remains vague.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Harvest Filesのファイルダウンロード用URLを取得します' (Get download URL for Harvest Files). It specifies the verb ('取得します' - get/obtain) and resource ('ダウンロード用URL' - download URL), making the action clear. However, it doesn't explicitly differentiate from sibling tools like 'get_harvest_file_info' or 'list_harvest_files', which likely provide metadata rather than download URLs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'get_harvest_file_info' (which might provide file details) or 'list_harvest_files' (which might list available files), nor does it specify prerequisites or contexts for usage. The agent must infer usage from the purpose alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_harvest_filesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions pagination ('ページング用キー') and returns a list, but doesn't disclose important behavioral traits: whether this is a read-only operation, if it requires authentication, rate limits, error conditions, or what the list structure looks like. For a tool with 4 parameters and no annotation coverage, this is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Args, Returns) and uses bullet points effectively. It's appropriately sized - every sentence adds value. The main purpose statement is front-loaded, followed by parameter explanations. No wasted words, though the Japanese-only text might limit accessibility.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 4 parameters, 0% schema coverage, no annotations, but an output schema exists, the description is moderately complete. It explains what the tool does and what parameters mean, but lacks behavioral context (permissions, errors, etc.) and usage guidance. The output schema handles return values, so that burden is lifted. For a list operation, this is minimally adequate but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It provides Japanese descriptions for each parameter (scope, path, limit, last_evaluated_key) that add meaning beyond the bare schema, explaining what each parameter represents. However, it doesn't provide format details (e.g., path syntax, limit range) or clarify the relationship between scope and path. Given the coverage gap, this is minimal compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Harvest Filesのファイル・ディレクトリ一覧を取得します' (Get a list of files/directories from Harvest Files). It specifies the verb ('取得します' - get/retrieve) and resource ('ファイル・ディレクトリ一覧' - file/directory list). However, it doesn't explicitly differentiate from sibling tools like get_harvest_file_info or get_harvest_file_download_url, which likely serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With siblings like get_harvest_file_info (likely for metadata) and get_harvest_file_download_url (likely for downloading), there's no indication of when list_harvest_files is appropriate versus those other tools. The description only states what it does, not when to choose it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_harvest_file_infoB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves storage usage (e.g., usage amount, file count), implying a read-only operation, but doesn't clarify permissions, rate limits, or side effects. It adds some context by specifying what information is returned, but lacks details on response format or potential errors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the main purpose stated first, followed by structured sections for Args and Returns. Each sentence earns its place by providing essential information without redundancy. However, the Japanese text might require translation for some agents, slightly affecting clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no nested objects) and the presence of an output schema (which handles return values), the description is reasonably complete. It covers the purpose, parameter semantics, and return content adequately. However, it lacks usage guidelines and detailed behavioral context, which are minor gaps for a simple read tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful semantics for the single parameter 'scope' by explaining it as 'スコープ（private または operators/{operator_id}）' (scope: private or operators/{operator_id}), which clarifies its purpose and possible values. Since schema description coverage is 0% and there's only one parameter, this adequately compensates, though it doesn't detail default behavior or constraints beyond the listed options.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Harvest Filesのストレージ使用状況を取得します' (Get Harvest Files storage usage status). It specifies the verb '取得します' (get) and resource 'ストレージ使用状況' (storage usage status), making the purpose unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'list_harvest_files' or 'get_harvest_data', which might also retrieve Harvest-related information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'list_harvest_files' or 'get_harvest_data', nor does it specify prerequisites, exclusions, or contextual triggers for usage. The agent must infer usage based on the purpose alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_harvest_dataA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds some context beyond the schema: it specifies the maximum limit (1000), explains pagination behavior with 'last_evaluated_key', and describes the return format. However, it doesn't cover important aspects like rate limits, authentication requirements, error conditions, or whether this is a read-only operation (though implied by 'get').

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose statement followed by organized parameter and return sections. Every sentence earns its place by providing essential information. It could be slightly more concise by combining some explanations, but the structure is effective and front-loaded with the main purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (6 parameters, pagination, time filtering) and no annotations, the description does well. It explains all parameters thoroughly and mentions the return format. With an output schema present, it doesn't need to detail return values. The main gap is lack of sibling tool differentiation, but otherwise it's reasonably complete for this data retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must fully compensate. It successfully documents all 6 parameters with clear explanations: 'imsi' as SIM identifier, time parameters as UNIX timestamps in milliseconds, 'sort' with enum values explained, 'limit' with maximum constraint, and 'last_evaluated_key' for pagination. This adds substantial meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '特定SIMのHarvest Dataを取得します' (Retrieve Harvest Data for a specific SIM). It specifies the verb '取得します' (retrieve) and resource 'Harvest Data', but doesn't explicitly differentiate it from sibling tools like 'get_harvest_data_by_resource' or 'list_harvest_files', which likely have different filtering or output approaches.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description doesn't mention sibling tools or explain scenarios where this tool is preferred over others like 'get_harvest_data_by_resource' or 'list_harvest_files'. Usage is implied only through the parameter descriptions, not explicit contextual guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_harvest_data_by_resourceA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It mentions pagination behavior ('ページング用キー') and a maximum limit constraint ('最大1000'), which are useful behavioral details. However, it doesn't cover important aspects like authentication requirements, rate limits, error conditions, or whether this is a read-only operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (Args, Returns) and front-loaded the core purpose. Every sentence adds value, though the Japanese-only presentation might be less accessible in multilingual contexts. The structure is efficient with no wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (7 parameters, pagination, time-based filtering) and the presence of an output schema (which handles return values), the description is quite complete. It explains all parameters thoroughly and mentions the pagination behavior. The main gap is lack of sibling tool differentiation and some behavioral context like authentication.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage and 7 parameters, the description provides excellent parameter semantics. It clearly explains each parameter's purpose, format constraints (UNIX timestamp in milliseconds), enum values for 'sort', maximum limit, and the pagination mechanism. This fully compensates for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'リソースタイプとIDでHarvest Dataを取得します' (Get Harvest Data by resource type and ID). It specifies the verb '取得します' (get/retrieve) and the resource 'Harvest Data', but doesn't explicitly differentiate from sibling tools like 'get_harvest_data' which likely has a different filtering approach.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With sibling tools like 'get_harvest_data', 'get_harvest_file_download_url', 'get_harvest_file_info', and 'list_harvest_files', there's no indication of when this resource-based retrieval is preferred over other methods.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.