Server Quality Checklist

Tool Scores

• get_context_suggestionsC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the tool provides suggestions based on input and file history, but doesn't describe what the suggestions look like, whether they're ranked, how many are returned, or any rate limits or authentication requirements. For a suggestion tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief (two short sentences) and gets straight to the point without unnecessary elaboration. However, it could be more front-loaded with the most critical information about what type of suggestions are provided.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a suggestion-generation tool with 3 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what format the suggestions take, what they're useful for, or how they relate to the sibling tools. The agent lacks crucial context about what this tool actually returns and when it should be preferred over alternatives.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters with Chinese descriptions. The description mentions '基于当前输入和文件推荐历史记录' (based on current input and file recommendation history), which aligns with currentInput and currentFiles parameters but doesn't add meaningful semantic context beyond what the schema provides. The project parameter isn't mentioned in the description at all.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool '获取相关上下文建议' (get relevant context suggestions) and mentions it's based on current input and file recommendation history. This provides a general purpose but lacks specificity about what type of suggestions are returned or what 'context' means. It doesn't clearly differentiate from sibling tools like search_conversations or log_conversation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of when this tool is appropriate versus using search_conversations or other siblings, nor any prerequisites or constraints for usage. The agent must infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_projectsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool lists projects but doesn't cover critical aspects like whether it's read-only (implied by 'list'), pagination behavior, rate limits, authentication needs, or what happens if no projects exist. The mention of 'optional statistics' hints at output behavior but lacks detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('List all projects') and adds a useful detail ('with optional statistics'). There is zero waste, and every word earns its place, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description is incomplete. It doesn't explain the return format (e.g., list structure, fields), error conditions, or behavioral traits like pagination. The mention of statistics is vague, and without output schema, the agent lacks guidance on what to expect from the tool's execution.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'includeStats' fully documented in the schema. The description adds minimal value by mentioning 'optional statistics', which aligns with the schema but doesn't provide additional context like what statistics are included or why to use them. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('projects'), making the purpose immediately understandable. It distinguishes the action from siblings like 'search_conversations' by focusing on listing rather than searching. However, it doesn't specify the scope (e.g., all projects accessible to the user) or differentiate from 'get_context_suggestions' in terms of resource type.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search_conversations' or 'get_context_suggestions'. It mentions 'optional statistics' but doesn't explain when to include them or any prerequisites. Usage context is implied at best, with no explicit when/when-not instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_conversationsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While it lists search capabilities, it doesn't describe important behavioral aspects: whether this is a read-only operation, what permissions are needed, how results are returned (format, pagination), whether there are rate limits, or what happens with no matches. For a search tool with 6 parameters and no annotations, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with a clear purpose statement followed by bullet points of capabilities. Every sentence earns its place, though the bullet points could be more efficiently integrated into the main description. The structure is front-loaded with the core purpose first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with 6 parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain what the search returns (conversation objects? summaries? IDs?), how results are ordered, whether there's pagination, or error conditions. The lack of output schema means the description should compensate by explaining return values, which it doesn't.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 6 parameters thoroughly. The description's bullet points (关键词搜索, 文件名模式搜索, 时间范围筛选, 标签过滤) map to 4 of the 6 parameters but don't add meaningful semantic context beyond what the schema provides. The baseline of 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches historical conversation records, which is a specific verb+resource combination. However, it doesn't distinguish this tool from potential sibling tools like 'get_context_suggestions' or 'log_conversation' - we can't tell if those tools also search conversations or perform different functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of sibling tools like 'get_context_suggestions' or 'log_conversation', nor any context about when search is appropriate versus listing or logging conversations. The bullet points describe capabilities but not usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• log_conversationA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly indicates this is a logging/write operation ('记录'), but doesn't specify persistence characteristics, potential side effects, error conditions, or what happens after logging. The description adds some behavioral context through the usage guidelines, but doesn't fully describe the tool's behavior beyond the basic logging action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose statement followed by organized usage guidelines. It's appropriately sized for a 7-parameter tool with detailed requirements. Every sentence serves a purpose, though the Chinese text could potentially be more concise in some sections. The information is front-loaded with the core purpose first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a logging tool with 7 parameters, no annotations, and no output schema, the description provides good parameter guidance but lacks information about what happens after logging. It doesn't specify return values, success/failure indicators, or how the logged data can be retrieved later. The usage guidelines are comprehensive for input preparation, but the overall context of the logging operation is incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds value by providing usage guidelines that explain the purpose and format expectations for each parameter (e.g., '3-5句话' for aiSummary, specific format for fileOperations). This goes beyond the schema's basic descriptions and provides practical guidance for parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: '记录AI对话 - 所有会话都要记录' (Log AI conversations - all conversations must be logged). This is a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'search_conversations' or explain how this logging tool relates to those search/retrieval tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidelines with a '使用规范' section that details when and how to use this tool. It specifies required parameters and their purposes, and the statement '所有会话都要记录' (all conversations must be logged) gives clear context about when this tool should be invoked. No explicit alternatives are mentioned, but the guidelines are comprehensive for this specific logging operation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.