Server Quality Checklist

Tool Scores

• generate_reportB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry behavioral disclosure. It does reveal two important traits: the operation is queued asynchronously and repeated calls are idempotent. Still, it does not mention potential side effects, required permissions, failure behavior, or whether the report must already exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one compact sentence that leads with the core action and then adds the idempotency caveat. There is no filler or redundant restating of the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with an output schema, the description conveys the core behavior and idempotency. However, it lacks explicit usage context relative to sibling tools and does not clarify the relationship between report_id and a 'confirmed' report, leaving meaningful gaps for an agent deciding when and how to call it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not elaborate on report_id. The only parameter is minimally labeled 'Report Id', but the description neither explains what value it expects, how it maps to a confirmed report, nor any format constraints. It fails to compensate for the schema's lack of guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action: placing a confirmed report into a background queue, with an idempotency guarantee. This distinguishes it from the sibling tools, though it does not explicitly name the alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase '已确认的报告' implies the tool is for reports that have already been confirmed, giving some contextual guidance. However, there is no explicit statement of when to use this tool over siblings like create_report_plan or update_report_outline, nor any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_report_outlineA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral transparency burden. It does disclose two important behaviors: the operation is a full replacement of the outline, and it can optionally set the report into a generation-ready state. However, it does not mention permissions, reversibility, or what happens to existing report content or state when confirm is set.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, tightly written sentence that leads with the core action ('全量替换报告大纲') and follows with the optional state confirmation. Every word contributes meaning, and there is no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main action and the confirm behavior, and the input schema together with the output schema provide structural detail. Still, there is no guidance on when this tool should be selected over create_report_plan, and the role of `topic` remains unexplained. For a destructive replace operation, this is adequate but not fully self-sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for missing parameter docs. It does clarify that `sections` represent the full replacement outline and that `confirm` relates to making the outline generation-ready. However, the optional `topic` parameter is not mentioned at all, and `report_id` is only inferable from the tool name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('全量替换' = fully replace) and a specific resource ('报告大纲' = report outline), and it adds a distinct state-transition behavior ('可将其确认为可生成状态'). This makes it clearly distinguishable from siblings like get_report, create_report_plan, and generate_report.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus create_report_plan or generate_report. There is no mention that the report must already have an outline, no mention of prerequisites, and no excluded cases. Usage context is only implied by the word 'replace'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_report_planA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral disclosure burden. It explicitly reveals two key behaviors: the tool creates a persistent side-effect task and runs asynchronously due to model/search service dependencies. This is meaningful, though it does not cover failure modes or how to track completion.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences. The first sentence states the core purpose and the second adds the critical async caveat. There is no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the essential what and why-async, and an output schema exists so return values are not missing. However, without annotations and with no usage guidance or parameter semantics, the agent lacks workflow context such as how to check the persistent task later or when to use sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the tool description does not mention 'topic' or 'options' at all. The nested ReportOptions type has some schema-level explanation (non-sensitive config, extra=forbid), but the description adds no parameter meaning, leaving the agent to infer from names alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action: '创建持久化任务并生成可编辑大纲' (create a persistent task and generate an editable outline). It distinguishes this tool from siblings like generate_report by focusing on the outline, not the final report.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context by explaining the tool is asynchronous because it calls model and search services, which implies when it should be used. However, it does not explicitly mention alternatives or when not to use it, such as preferring generate_report for final output or update_report_outline for editing an existing outline.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_reportA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full behavioral burden. It discloses that the tool reads task progress, that the default response excludes content and sources, and that including them can result in a large return payload. This is adequate for a simple read operation, though it omits details like error conditions or authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with the main verb and resource front-loaded, followed by relevant parameter guidance. Every word earns its place, and there is no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity read tool with only three simple parameters and an output schema, the description provides enough context to call the tool correctly. The only minor gap is a slight ambiguity around what 任务进度 means exactly, but the reference to 正文 and 来源 helps clarify the tool's scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does so by mapping 正文 and 来源 to include_content and include_sources, and by indicating the default behavior through the phrase 避免默认返回过大. The report_id parameter is not described, but it is required and self-evident from the parameter name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action, 读取 (read), and resource, 任务进度 (task progress), which clearly identifies this as a retrieval tool. It distinguishes itself from siblings like create_report_plan, update_report_outline, and generate_report by being the only read-oriented operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies this tool should be used to read task progress and optionally include content/sources. It does not explicitly name alternatives or exclusion conditions, but the sibling tools are all mutating or generation-oriented, making the proper usage context obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.