Server Quality Checklist

Tool Scores

• generate_blank_templateA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the output format and contents, but does not mention whether the tool overwrites existing files at the default path, which is a key behavioral trait for a file generation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two sentences plus a parameter note, front-loading the core purpose and providing necessary details without extraneous text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one optional parameter) and the presence of an output schema, the description covers the essential aspects: what it generates, file contents, and parameter behavior. Minor gaps exist (e.g., overwrite) but overall sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter (output_path) has 0% schema description coverage, but the description explains its purpose, default behavior, and return value clearly, adding significant value beyond the schema's minimal definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool generates a blank Excel template for social security calculation, specifying file format (xlsx), contents (instructions, dropdowns, example row), and distinguishes from sibling tools (inspect, normalize, standardize) which handle existing data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The context is clear that this tool is for generating a blank template, implying it should be used when starting from scratch rather than processing existing files. However, no explicit guidance on when not to use it or alternatives is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inspect_payrollA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the transparency burden. It discloses the tool's outputs and handling of unmapped files, but does not explicitly state side effects, auth needs, or confirm read-only nature, leaving some ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loading the action and outputs, with no fluff. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers key outputs and fallback behavior, sufficient for the tool's complexity. Given an output schema exists, it does not need to detail return values, but could mention the output schema structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, the description clarifies 'folder' as an absolute path, which adds meaning beyond the name. The single parameter is adequately explained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool inspects payroll files in a folder and returns headers, sample rows, field mapping, and issues. It also ties into the workflow with sibling tool normalize_payroll, establishing its distinct role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing to inspect payroll files and map columns, and mentions fallback behavior for unmapped files. It does not explicitly state when not to use it, but the context with siblings is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• normalize_payrollA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It describes the transformation (merge by person cross-entity/month), output types, and the overrides mechanism. However, it does not mention side effects (e.g., does it modify original files?) or permissions required, but the disclosed behavior is sufficient for most use cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a compact paragraph that covers purpose, parameters, and output. It is front-loaded with the main function. The overrides_json description is detailed but necessary. Slightly verbose for the parameter descriptions, but overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 parameters, one required, no enums, output schema exists), the description provides sufficient context: what it does, how to use each parameter, and what the return JSON contains. It lacks details on error scenarios or file format expectations, but is adequate for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description must compensate. It does so thoroughly: defines folder, output_dir with default, and overrides_json with a detailed format example including the structure of column_map and standard field names. This adds significant meaning beyond the schema's property names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'normalize' and the resource 'payroll files in a folder', specifies the outputs (standard template xlsx and report md), and distinguishes from siblings like generate_blank_template by focusing on processing existing files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains what the tool does but does not explicitly state when to use it over siblings or when not to use it. It includes hints about optional overrides for problematic files, but lacks exclusion criteria or alternative tool references.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• standard_columnsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It correctly indicates this is a read-only retrieval tool returning definitions and options. No side effects are described, which is appropriate. The output schema likely covers return format, so no contradiction or gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences that get straight to the point, front-loaded with purpose and usage timing. Every word adds value, no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters, presence of output schema, and the tool's simple nature as a static reference, the description is complete. It covers purpose, usage context, and key content.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters in the schema. The rubric sets baseline 4 for 0 parameters. The description adds value by explaining what the tool returns, but no parameter-specific info is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns definitions of 10 columns, identity type options, and key concepts like 应发≠实发, specifying it is a reference for column mapping. This distinguishes it from sibling tools that generate templates, inspect payroll, or normalize payroll.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says '模型在做列映射前应先读这个' (the model should read this before doing column mapping), providing clear guidance on when to use this tool relative to other operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.