Server Quality Checklist

Tool Scores

• generate_imageA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are not provided, so the description carries the burden of disclosing behavior. It does mention the side effect: 'The image will be saved to the current working directory.' This is valuable. However, it lacks details about file overwriting, image format, or any potential errors, leaving some ambiguity about the tool's runtime behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured. It starts with the purpose, followed by usage, side effect, and an args list. Every sentence contributes useful information. The args list is slightly redundant with the schema, but since the schema lacks descriptions, the inclusion is justified and earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters, no nested objects) and the existence of an output schema, the description covers the essential aspects: what it does, when to use it, how it behaves (saving to CWD), and parameter meanings. It does not describe return values, but the output schema handles that. Minor omissions like file overwrite behavior prevent a higher score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no descriptions for its parameters (0% schema coverage), so the description's args section is essential. It explains 'prompt' and 'filename', and also adds the default value for filename. This fully compensates for the missing schema descriptions and goes beyond the bare schema by clarifying the meaning of each argument.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Generate an image from a text description' and specifies the model 'Gemini (nano banana)'. This distinguishes it from sibling tools like audio transcription or knowledge base operations, which serve entirely different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to use the tool: 'Use this when the user needs illustrations, diagrams, icons, or any visual content.' This gives clear context. It does not mention when not to use it, but there are no obvious alternative image tools among the siblings, so this is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• register_voiceprintA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosure. It does reveal that 'The voiceprint will be saved and used to identify this person in future meeting transcriptions,' which is important side-effect information. However, it omits potential failure modes, the effect of duplicate registrations, or any prerequisites, leaving gaps for a write operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is succinct and well-structured. It leads with the primary purpose, adds a vital audio requirement, and then organizes parameter details in a clear Args list. No redundant or vague sentences; every line provides useful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with only two parameters and an output schema, the description is largely complete. It explains the purpose, the audio input characteristics, and the meaning of each parameter. Minor omissions like file format validation or error conditions are acceptable given the simplicity. The presence of an output schema reduces the need to document return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no descriptions (0% coverage), so the description must compensate. It does so with an Args section explaining both parameters: 'name: The person's name (will be used as label in transcriptions)' and 'file_path: Absolute path to an audio file with the person's voice.' This adds meaningful context beyond mere strings, but doesn't fully elaborate format or validation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Register a person's voiceprint from an audio file.' This uses a specific verb and resource, and distinguishes it from sibling tools like transcribe_audio or knowledge management tools. The scope is well-defined.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit context on how to use the tool, such as 'The audio should contain only one person speaking clearly for 20-30 seconds.' This gives clear guidance for proper usage. However, it does not explicitly mention alternatives or when not to use the tool, keying it just below a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_knowledgeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden and discloses key behaviors: semantic similarity, optional type filtering, partial matching for company/person, and top_k default. It does not discuss permissions or edge cases, but for a search tool this is substantial and non-contradictory.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with a clear purpose sentence and followed by a compact Args list. Each parameter line adds necessary detail, given the schema's empty descriptions. Slightly longer than minimal, but every line earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with an output schema, the description covers all invocation-relevant aspects: required query, optional filters, partial-match behavior, and default top_k. It does not explain result ranking, but that is likely handled by the output schema, so completeness is solid.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero parameter descriptions, but the description compensates fully by documenting every parameter, including the allowed values for 'type', partial match semantics for 'company' and 'person', and the default for 'top_k'. This goes well beyond bare parameter names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Search the knowledge base using semantic similarity', which is a specific verb, resource, and method. It clearly differentiates from sibling write tools like index_knowledge and upsert_knowledge.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for semantic retrieval but does not explicitly state when to use it vs alternatives, nor are there exclusions or when-not-to-use conditions. Given sibling write tools, the use case is understandable but not directly articulated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• index_knowledgeA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses the scope (specific directories), destination (ChromaDB), and the fact that it rebuilds the full index. However, it does not explicitly state whether this is destructive to the existing index or if it is a long-running operation, though 'rebuild' implies recreation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each adding value: purpose, scope, and usage trigger. It is front-loaded with the main action and has zero redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool, the description is quite complete: it specifies input files, output destination, and when to call it. An output schema exists, so further return-value detail is unnecessary. However, it could have mentioned potential side effects like index replacement, but 'rebuild' covers this implicitly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  This tool has zero parameters, and schema coverage is vacuously 100%, so a baseline of 4 is appropriate. The description adds context about what is scanned and where data is indexed, which is useful beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: rebuilding the vector index from markdown files, listing specific directories scanned. It distinguishes itself from siblings like upsert_knowledge (which likely handles individual entries) and search_knowledge (querying).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly provides when to use: 'Call this after adding or updating knowledge base files.' It does not mention alternatives or exclusions, but the context is clear enough for the agent to decide between this and sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• transcribe_audioA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses local processing, pyannote-based speaker detection, voiceprint matching, and the exact pattern for unknown speaker labels ('未知说话人1'). This adds meaningful context beyond the bare input schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a clear opening sentence, a compact usage list, and a focused Args section. Every part earns its place, though slightly verbose in the use-case enumeration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present, return values need no explanation. The description covers purpose, usage, parameters, and a key behavioral trait (speaker labeling). Could mention file format support, but for a two-parameter tool this is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description's Args section fully compensates. It clarifies file_path as an absolute path and explains model_size with default and rationale (large-v3 for best Chinese support), providing meaning entirely missing from the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Transcribe an audio/video file to text using local Whisper' with a specific verb and resource, and then explains automatic speaker detection and voiceprint matching. This clearly distinguishes it from sibling tools like search_knowledge or generate_image.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  An explicit 'Use this when...' section lists concrete use cases such as meeting minutes and interview transcription. While it doesn't explicitly name alternative tools or exclusions, the clear context suffices given the sibling set.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_pptA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the internal pipeline (Plans an outline using Gemini, generates slide images, outputs a PDF) and parameter constraints such as 'max 3000 chars used' for context_text. This adds behavioral context beyond the schema, though it doesn't detail side effects, failure modes, or permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured. It front-loads the purpose, adds workflow context, then provides usage guidance, and ends with a clear Args list. Every sentence contributes value without unnecessary fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core functionality, workflow, usage, and parameter details. Since an output schema exists, return value explanation is not required. It misses potential edge-case behavior (e.g., error handling, file output location), but for a generation tool this is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides only parameter titles with no descriptions. The description compensates thoroughly by defining each parameter: topic with examples, context_text with a length limit, and page_count with a default. This adds substantial meaning beyond the schema structure.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and object: 'Generate a complete PPT presentation from a topic and optional context.' It further details the workflow (plans outline via Gemini, generates slide images, outputs PDF), which clearly distinguishes it from sibling tools like generate_image. The purpose is unambiguous and action-oriented.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states 'Use this when the user wants to create a PPT/presentation.' It also mentions the /ppt skill integration, providing context on automated invocation. It lacks explicit exclusions or a comparison to alternatives, but the trigger condition is clear and sufficient for an agent to select this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• upsert_knowledgeA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses several behavioral traits: it compares MD5 hashes, detects additions/modifications/deletions, and is faster for small changes. It does not mention side effects like permanent removal from index, but the mention of deletions covers that. It could be more explicit about index mutation behavior, but overall it provides substantial transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each earning its place: the first defines the core action, the second explains the mechanism, and the third gives performance guidance. No fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With zero parameters and an output schema present, the description provides sufficient context: what it does, how it works, and when to use it. It clearly differentiates from the full-index sibling, and the output format is handled by the schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4 per the rubric. The description does not need to explain parameters, and the schema is empty. No additional semantics are required or provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Incrementally update the vector index' with a specific mechanism (MD5 hashes) and scope (only changed files). It distinguishes itself from sibling index_knowledge by emphasizing incremental updates.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly provides usage guidance: 'Much faster than index_knowledge() when only a few files changed' implies using this tool for small changes and index_knowledge for full re-indexing. This addresses when to use the tool versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.