Server Quality Checklist

Tool Scores

• memory_readA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the return is a string prefixed with [TRUSTED] or [DATA] under security conditions. However, it omits details about error handling (e.g., missing file) or if the tool is read-only, leaving gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with three sentences: one for core purpose and two for path format and return value. Every sentence adds essential information with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no annotations, but has an output schema), the description covers the key aspects: what it reads, how to specify the path, and the return format with security prefix. It lacks error behavior but is fairly complete for a read tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It adds meaning by explaining that 'path is relative to the palace root' and provides a concrete example, giving context beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Read a file from the palace,' providing a specific verb and resource. While it distinguishes from sibling tools like memory_write and memory_search, it does not explicitly differentiate from memory_batch, which could also read files. Still, the purpose is well-understood.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a path format example but offers no guidance on when to use this tool versus alternatives like memory_list or memory_search. There is no mention of prerequisites, context, or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• memory_searchA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries full burden. It discloses implementation (ripgrep vs. Python re), result limit of 20 matches, output structure (path, line number, line, context), and use of path parameter for scope. Missing details like authorization or performance, but overall strong.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, front-loads the purpose, and every sentence contributes useful information without redundancy. It efficiently covers implementation, limits, parameters, and output format in a short paragraph.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (context signal) and the tool's complexity, the description covers core functionality, implementation, limits, and parameter effects. It omits explanation of the 'palace' metaphor and result ordering, but overall is sufficient for an AI agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate. It clarifies that 'query' is a search term (implied regex) and 'path' narrows to a subdirectory. This adds meaning but could be more specific about query syntax and behavior when omitted.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs full-text search across the palace (or sub-path), specifying the search engine used and result limits. This distinguishes it from sibling tools like memory_list or memory_read which are not search-oriented.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly advise when to use this tool versus alternatives. It only hints at narrowing search via the path parameter but lacks explicit 'when-to-use' or 'when-not-to-use' guidance, nor does it mention sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• memory_writeA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully bears the burden of disclosing behavior. It reveals the write is atomic (temp file then rename), rejects writes to certain directories and non-text extensions, and creates parent directories as needed. This goes beyond a simple 'write' statement, though it omits whether overwriting occurs or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (3 sentences) with front-loaded purpose and no redundant information. Every sentence adds value: purpose, path semantics, atomicity, restrictions, and directory creation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists (context confirms), the description does not need to explain return values. It covers key aspects: path behavior, atomicity, restrictions, and directory creation. However, it could mention overwrite behavior or error cases for completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning for 'path' (relative to palace root, restricted directories, non-text extensions) but for 'content' only mentions 'content' without format details. The description partially fills the gap but not fully.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Write content to a file within the palace.' with specific verb and resource. It differentiates from siblings like memory_batch (batch writes) and memory_read (read). The additional details about path, atomicity, and restrictions reinforce the purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  While the description provides important usage constraints (path relative to palace root, atomic write, rejected paths/extensions, creates parent dirs), it lacks explicit guidance on when to use this tool versus alternatives like memory_batch for batch operations. The context is implicit rather than directly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• memory_listA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of disclosing behavioral traits. It states the tool lists files (implied read-only) and differentiates between index retrieval and room listing. It does not mention side effects, permissions, or error handling, but the behavior is predictable and non-destructive. The presence of an output schema compensates for missing return value details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, each carrying distinct, necessary information. It is front-loaded with the core purpose, then provides specific usage details. No word is wasted, and the structure makes it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has one optional parameter, no required fields, and an output schema, the description covers the primary use cases. However, it does not specify what happens if an invalid path is given (e.g., non-existent room), and it mentions 'markdown files' specifically, which might imply only .md files are listed. These are minor gaps, but the description is mostly complete for a simple list tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no descriptions (0% coverage), so the description must add all parameter meaning. It fully explains the 'path' parameter's role: empty string retrieves the index, a room path lists files. This completely compensates for the schema's lack of documentation, making the parameter semantics crystal clear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists the palace index or files within a room, with specific behavior for empty vs non-empty path. It distinguishes the two main use cases and uses a specific verb ('list') and resource structure ('palace index', 'room files'). This is precise and leaves no ambiguity about the tool's purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear guidance on when to call without path (to retrieve INDEX.md) and when to pass a room path (to list files in that room). It does not explicitly mention when not to use this tool or suggest alternatives like memory_search or memory_read, but the usage context is well-defined and easy to follow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• memory_batchA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavior: returns joined files with headers, handles missing/invalid paths inline without exceptions, raises ValueError for invalid args, returns empty string for empty list. Transparent about internal constant _MAX_BATCH_PATHS.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise two-paragraph description, no redundancy. Front-loaded with purpose, then clear parameter descriptions. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple parameter (one array) and existence of output schema, the description covers input expectations, return format, error handling, and edge cases (empty list, missing files). Complete for this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but description adds meaning: 'paths' is a list of relative paths with a maximum count, and explains format. Provides enough context beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it reads multiple palace files in a single call, specifying the action (read multiple) and resource (palace files). It differentiates from siblings like memory_read (single file) and memory_write (write operation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides context on when to use (batch reading) and behavior details (max paths, partial results). Does not explicitly list alternatives but implies usage for multiple files versus single file reads.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.