Server Quality Checklist

Tool Scores

• update_knowledgeA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It discloses that the tool is deterministic, does not interpret natural language, targets a specific heading, and appends a new section if the heading is missing. This covers key operational behavior, though it does not mention permissions, error handling, or reversibility—acceptable for a low-risk knowledge document update.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, efficient and front-loaded with the core action. The key behavioral constraints (deterministic, no NL interpretation, heading behavior) are stated without unnecessary fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple update tool with 4 parameters, no output schema, and no annotations, the description covers the main operational aspects: target document, heading behavior, append/replace semantics, and the caller's responsibility. It lacks explicit mention of edge cases like handling invalid paths or partial failures, but these are not critical for a knowledge base update.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents each parameter. The description adds value beyond the schema by explaining the heading-targeting behavior (exact text, and appending a new section if not found) and by clarifying that the caller must decide between append and replace, which maps to the mode parameter but is not fully explicit in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb (update), resource (existing knowledge document), and a precise mechanism (targeting a Markdown heading). It explicitly distinguishes itself from natural-language change requests and implies it is not for creation, which differentiates it from siblings like create_knowledge and capture_knowledge.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies use for updating existing documents and clarifies it is deterministic, but it does not explicitly name alternatives or state when not to use it (e.g., for creating new docs, use create_knowledge). The behavioral note about appending a new section when heading is missing provides some context, but sibling differentiation is left implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_knowledgeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well by disclosing the fail-if-exists behavior, path sanitization, and containment within the knowledge directory. It does not mention the success return value or authorization requirements, but those are secondary for a create operation with this level of detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no filler: the main action is front-loaded, the failure condition follows immediately, and the alternative tool is named without extra prose. Every sentence contributes essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for a simple create tool: it covers purpose, path behavior, failure mode, and the key alternative. The only minor gap is not describing what a successful response looks like, but the absence of an output schema makes that less critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents all three parameters. The description adds useful context for topic by explaining the knowledge/<topic>.md mapping and sanitization, but it does not significantly expand on title or content beyond what the schema already says.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names the specific action (create), the resource (Markdown knowledge document), and the exact storage location (knowledge/<topic>.md). It also explicitly contrasts itself with update_knowledge by stating the failure condition when the document already exists.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly says to use update_knowledge instead when a document already exists, which gives an explicit create-vs-update routing rule. It does not explicitly contrast itself with capture_knowledge, but the failure-on-existing condition makes the intended use case clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_knowledgeA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly indicates it is a read operation ('Retrieve'), which implies no side effects. However, it does not disclose details like error handling (e.g., behavior when topic is not found), authentication requirements, or rate limits. For a simple read tool this is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise, with two sentences that front-load the primary purpose and then provide illustrative examples. Every word earns its place, with no filler or redundant content. This is ideal for efficient agent parsing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one parameter and no output schema, the description covers the essential aspects: what it does (retrieve full content) and what inputs it accepts. It does not specify the return format (e.g., markdown text) or any failure modes, but given the simplicity of the tool, this is not a significant gap. The description is sufficient for an agent to know how to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Since schema coverage is 100%, the parameter is already well-documented in the schema with an example. The description adds value by explaining that loose references are accepted and provides additional format examples ('knowledge/api/apim.md'), which the schema does not cover. This goes beyond mere repetition and helps the agent understand the flexibility of the input.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the verb 'Retrieve' and the resource 'full content of a specific knowledge document', making the purpose unambiguous. It also distinguishes from siblings by focusing on retrieval of a specific document by topic/path, as opposed to search or list operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool - when you have a specific topic or path - and gives concrete examples of accepted loose references. It does not explicitly name sibling tools or state when not to use it, but the implication is clear that this is for known references rather than discovery.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• capture_knowledgeA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description bears the full burden of behavioral disclosure, and it succeeds: it states that it appends a timestamped entry to a specific file (inbox/knowledge-inbox.md) and that the entry is for later triage into knowledge/. It also clarifies that it does not require knowing where the information belongs, setting correct expectations. It could mention potential effects (e.g., file creation or append behavior) but the essential behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence followed by an example and a usage directive. It front-loads the core purpose, provides a concrete illustration, and adds the usage trigger without any filler. Every element earns its place, and the whole fits comfortably in a readable length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with only two parameters, one required, no output schema, and no nested structures, the description is fully sufficient. It covers what the tool does, where it writes, when to use it, and the key benefit (low friction). Nothing an agent needs to correctly invoke this tool is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the schema already documents both parameters at 100% coverage, the description adds meaningful semantic context. It explains that content is a durable engineering fact or convention, gives an example, and clarifies that suggestedTopic is optional because the tool does not require knowing the target location. This goes beyond mere schema repetition and helps an agent understand how both fields interact.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('capture'), a target resource (durable engineering knowledge appended to inbox/knowledge-inbox.md), and a clear distinction from direct knowledge creation. It includes a concrete example that helps an agent immediately recognize what counts as appropriate input, and the sibling list (create_knowledge) reinforces the uniqueness of this tool as an inbox capture.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to use the tool: 'Use when the user explicitly asks to remember, capture, or add durable engineering knowledge.' It also explains the benefit ('Does not require knowing where the information belongs'), which implicitly signals that create_knowledge is the alternative when topic placement is known. However, it does not explicitly list when-not-to-use conditions or name sibling alternatives, so it falls just short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_knowledge_topicsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden. It implies a read-only operation (listing without side effects) and even mentions checking existence before creation, but it does not explicitly state 'read-only' or 'non-mutating'. It is clear enough, though not fully explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences and tightly packed with purpose, output details, and usage guidance. No unnecessary words or redundancy; it is efficiently structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the operation (no parameters, no output schema), the description provides sufficient context: what it lists, what it omits, and when to use it. It covers the essential aspects for an agent to decide and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is empty, so there are no parameters to describe. The description adds no parameter-related information, but since schema coverage is 100% (empty), a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists knowledge documents with their path, title, and tags, and explicitly excludes full content. It distinguishes from sibling tools by framing it as a browsing/pre-check operation, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit use cases: browsing topics before searching, when keyword search returns empty, and before calling create_knowledge. This gives clear guidance on when to select this tool over alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_knowledgeA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the matching mechanism (case-insensitive text, tag boosting), the scope (including not-yet-triaged inbox captures), and the return format (sections with file paths). It does not mention potential side effects or limitations like rate limits, but the read-only nature is strongly implied by 'Search'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, consisting of two sentences that efficiently cover the tool's functionality, output, and usage context. There is no redundant or vague wording, and every clause contributes informational value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of annotations and output schema, the description single-handedly provides sufficient context: it defines the search scope, matching behavior, return details, and a clear use-case. It also clarifies that it returns sections rather than full documents, which is crucial for an agent deciding between this and a document retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool description adds meaning to the single 'query' parameter by explaining that it performs case-insensitive text matching and that tag matches boost results. This goes beyond the schema's example terms, giving the agent insight into how the query is interpreted.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: searching all engineering knowledge Markdown documents, including inbox captures. It specifies the resource, the verb 'Search', and the output (sections with file paths), making it unambiguous and distinct from other potential tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided: 'Use this before asking the user about internal conventions, infra, APIs, auth, or platform config.' This tells the agent exactly when to invoke this tool, and the implicit contrast with retrieving full documents (sections vs. full documents) hints at when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.