Server Quality Checklist

Tool Scores

• get_statusB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It describes what the tool returns (conversation state, iteration count, configuration) and lists specific return fields, which adds useful context about the tool's behavior. However, it doesn't mention whether this is a read-only operation, if it requires authentication, or any rate limits—important details for a status-checking tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first line states the purpose clearly, followed by details on return content. The use of a separator (---) and bullet points for return fields improves readability. However, the inclusion of both Chinese and English text slightly reduces efficiency, and some redundancy exists (e.g., stating return content in two ways).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no annotations, but has an output schema), the description is reasonably complete. It explains what the tool does and details the return values, which compensates for the lack of annotations. Since an output schema exists, the description doesn't need to fully explain return values, but it still provides a helpful overview. For a status-retrieval tool, this is adequate though not exhaustive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage (empty schema), so the baseline is 4 as per the rules for zero parameters. The description appropriately doesn't discuss parameters since none exist, and it focuses on the return values instead, which is correct given the context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '获取当前状态' (get current status) and specifies it returns conversation state, iteration count, and configuration information. This is a specific verb+resource combination that distinguishes it from sibling tools like consult_aurai, report_progress, and sync_context, which appear to perform different functions. However, it doesn't explicitly contrast with siblings beyond implying different functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any prerequisites, appropriate contexts, or comparisons with sibling tools like consult_aurai or report_progress. The agent must infer usage from the purpose alone, which is insufficient for optimal tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• report_progressA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's purpose and workflow (reporting progress and requesting guidance), though it doesn't specify technical details like response format, rate limits, or authentication requirements. However, it clearly communicates the tool's interactive nature and expected usage pattern.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized. It opens with a clear purpose statement, provides usage guidelines, and includes a formatted section with usage scenarios and parameters. Every sentence serves a purpose with no redundancy or wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 parameters, interactive workflow) and the presence of an output schema (which handles return values), the description is largely complete. It covers purpose, usage context, and parameters adequately. The main gap is lack of behavioral details like error handling or response structure, but the output schema mitigates this.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description lists parameters in a section but doesn't add meaningful semantic context beyond what's in the schema (e.g., explaining how 'result' influences guidance or what constitutes good 'feedback'). This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('报告执行进度' - report execution progress, '请求下一步指导' - request next-step guidance) and distinguishes it from siblings like consult_aurai (consultation), get_status (status retrieval), and sync_context (context synchronization). It explicitly defines the tool's role in reporting results after executing superior AI suggestions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidelines: '在执行了上级AI的建议后，调用此工具报告结果，获取下一步指导' (After executing superior AI suggestions, call this tool to report results and get next-step guidance). It clearly defines when to use this tool (post-execution reporting) versus alternatives like consult_aurai (for consultation before execution) or get_status (for status checking without guidance).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sync_contextA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: file type restrictions (.txt and .md only), token optimization (caching for fields >800 tokens), and the three operation modes (full_sync, incremental, clear) with their purposes. However, it doesn't mention potential side effects like whether files are stored persistently, if there are rate limits, or authentication requirements, leaving some gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with sections for typical scenarios, notes, and parameter explanations, but it is overly verbose. The extensive code examples and scenario details could be condensed; not every sentence earns its place as some repetition occurs (e.g., file restrictions mentioned multiple times). It's front-loaded with the core purpose, but the length may reduce scanability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (mutation with file handling) and rich schema (100% coverage, output schema exists), the description is highly complete. It covers purpose, usage guidelines, behavioral traits, parameter semantics with examples, and operational details. The output schema handles return values, so the description appropriately focuses on input and behavior, leaving no significant gaps for agent understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters. The description adds significant value beyond the schema by explaining the rationale behind file restrictions (to avoid truncation), providing concrete usage examples with code snippets, and detailing token optimization behavior. However, it doesn't fully explain the semantics of 'project_info' beyond stating it can contain arbitrary fields, missing guidance on typical or required fields.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '同步代码上下文（支持上传 .md 和 .txt 文件，避免内容被截断）' - to sync code context by uploading .md and .txt files to avoid truncation. It specifies the verb (sync/upload), resource (code context via files), and distinguishes from siblings by focusing on file-based context management rather than consultation, status checking, or progress reporting.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool: '在第一次调用或上下文发生重大变化时使用' (use on first call or when context changes significantly). It includes detailed scenarios (article review, code upload, project initialization) with concrete examples and contrasts with alternatives by noting that code should be uploaded here instead of placed in 'context' or 'answers_to_questions' fields of other tools to avoid truncation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• consult_auraiA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure and excels. It describes the multi-round interaction protocol (status='need_info' triggers follow-up calls), honest principle requirements (no fabrication), automatic history clearing when resolved=true, consequences of is_new_question (clears all prior context), and response formats. It adds rich context beyond what the input schema provides.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is comprehensive but overly long and not front-loaded. While it contains valuable information, it includes extensive formatting (markdown, code blocks, emojis) and repetitive sections (e.g., multiple warnings about truncation, redundant explanations of is_new_question). Some content could be condensed without losing clarity, making it less efficient than ideal.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multi-round interaction, 7 parameters), no annotations, and the presence of an output schema, the description is exceptionally complete. It covers purpose, usage, behavioral protocols, parameter guidance, sibling tool relationships, and response handling. The output schema existence means return values needn't be explained, and the description fully compensates for the lack of annotations with detailed operational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the baseline is 3. The description adds significant value by explaining parameter usage in context: it specifies which parameters are required for first calls (problem_type, error_message, context, code_snippet) vs. follow-up calls (answers_to_questions), provides examples for code_snippet/context handling with sync_context, and clarifies the impact of is_new_question. However, it doesn't add deep semantic nuance beyond the schema's descriptions for all parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool's purpose: '请求上级AI的指导' (request guidance from a higher-level AI) and '当本地AI遇到编程问题时调用此工具获取上级AI的指导建议' (call this tool when the local AI encounters programming problems to get guidance from a higher-level AI). It clearly distinguishes from siblings by explaining this is the '核心工具' (core tool) for obtaining AI guidance, while sibling tools handle context synchronization, progress reporting, and status checking.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides extensive usage guidelines, including when to use this tool ('当本地AI遇到编程问题时'), when to use sibling tools instead (e.g., use sync_context for uploading files to avoid truncation, report_progress after executing suggestions), and explicit alternatives. It also details when to set parameters like is_new_question and provides scenarios for manual vs. automatic context clearing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.