Server Quality Checklist

Tool Scores

• create_documentA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=false (write operation) and openWorldHint=true (side effects). The description adds valuable behavioral context: pads are ephemeral (lost on server restart, after 24h inactivity), and anyone with the URL can read/edit. This goes beyond the annotations and informs the agent about data persistence and security implications.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences with no wasted words. The first sentence front-loads the main action and return value. The second sentence clarifies the id behavior. The third sentence explains ephemerality and access. Every sentence earns its place, and the structure is logical and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 3 optional parameters and no output schema, the description covers what it does, what it returns (URL), and important behavioral traits (ephemeral, public access). It does not mention error cases or prerequisites, but the annotations (openWorldHint=true) and sibling tools provide enough context for an agent to select and invoke this tool correctly. The description is reasonably complete for a creation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents parameters. The description adds meaning by stating parameters are optional, explaining the id generation behavior ('Without an id a random one is generated'), and noting the return of a shareable URL (not in schema). This provides additional semantic context beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates a pad, optionally with initial content and language, and returns a shareable URL. It distinguishes from siblings like get_document (read), set_document (update), and append_to_document (modify) by focusing on creation. The verb 'creates' and resource 'pad' are specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for creating new pads but does not explicitly state when to use this tool versus alternatives (e.g., set_document for updating). It provides no exclusion criteria or guidance on when not to use it. While the purpose is clear, the lack of explicit when-to-use/when-not-to-use reduces the score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_document_infoA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, indicating a safe, non-mutating operation with potentially side effects. The description adds critical behavioral context: pads are ephemeral (lost after server restart or 24h without connection). This goes beyond annotations and clarifies data durability expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two focused sentences: the first states what the tool does and what it returns, the second gives critical ephemerality context. No wasted words; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read operation with one parameter, no output schema, and full annotation coverage, the description is nearly complete. It could optionally mention that the metadata is real-time or that the socket connection affects latency, but this is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the description adds no additional parameter details beyond what the schema provides. The schema already explains the pattern and meaning of 'id'. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies the verb 'fetches' and the resource 'metadata about a pad', listing specific fields (content length, revision, editor language, open users). It clearly distinguishes from siblings like get_document (which presumably fetches content) and get_stats (which likely returns aggregate statistics).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for fetching live pad metadata via the collaboration socket, but provides no explicit guidance on when to prefer this over alternatives like get_document. It does not mention when not to use it or contrast with siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• replace_in_documentA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark this as mutable (readOnlyHint=false) and destructive (destructiveHint=true). The description adds unique behavioral context: 'Only the matched ranges are edited, so concurrent edits elsewhere in the pad survive,' clarifying the partial update strategy. It also confirms the search is exact (no regex). This goes beyond what annotations provide and is useful for agent reasoning.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences total, each adding distinct value: purpose, concurrency behavior, and parameter default. No filler words. The information is front-loaded with the primary action. Every sentence earns its place, making it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 4-parameter replacement tool with no output schema, the description covers the key behaviors: exact match, partial edit, uniqueness default, and replace_all alternative. It does not mention error cases (e.g., no match, multiple matches without replace_all) or return value, but given annotations handle safety signals, it is sufficiently complete. A minor gap is the absence of concurrency or lock warnings, but the concurrent edit statement partially addresses this.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so each parameter already has a description. The tool description reinforces the meaning of 'search' (exact string), 'replace' (may be empty), and 'replace_all' (default single-match). The additional context about default uniqueness is helpful but does not greatly extend beyond the schema. Baseline is 3, and description marginally meets it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with 'Replaces an exact string in a pad with another,' using a specific verb ('replaces') and resource ('string in a pad'). It clearly distinguishes from siblings like set_document (full overwrite) and append_to_document (append), leaving no ambiguity about its targeted edit function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states 'By default the search string must match exactly once; set replace_all to change every occurrence,' which guides when to use the boolean parameter. It implies this tool is for exact string replacement, not regex or whole-document replacement, but does not explicitly name sibling alternatives for contrast. Context is clear enough for an agent to decide.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• append_to_documentA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description explicitly states that the tool appends text verbatim without altering other content or concurrent edits. Annotations already indicate it is not read-only (readOnlyHint: false) and that it operates in an open-world setting (openWorldHint: true). The description adds essential context about nondestructive appending behavior, which goes beyond the annotation declarations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences, each earning its place. The first sentence clearly states the action and its safe side effects, while the second provides a crucial usage tip. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 required params, no output schema, no nested objects), the description is nearly complete. It covers the core behavior and a practical nuance. It does not mention failure modes (e.g., what happens if the pad does not exist), but the schema's required fields and pattern already constrain valid usage. With no output schema, the agent might wonder about the return value, but the tool's API likely mirrors the input confirmation (a minor gap).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description restates that 'text' is appended verbatim, which the schema already says ('Text to append verbatim'). It adds the practical tip about including a leading newline to start a new line, which is helpful but not a semantic improvement over the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Appends') and resource ('a pad'), with explicit scope ('to the end of a pad'). It clearly distinguishes from siblings like 'set_document' (which replaces entire content) and 'replace_in_document' (which modifies specific parts) by emphasizing it only appends.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear guidance on when to use this tool (to add text without affecting existing content or concurrent edits). It implies the alternative of using a newline to start a new line. However, it does not explicitly exclude cases where one should choose 'replace_in_document' for editing middle content or 'create_document' for new pads.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_documentA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool as readOnly and openWorld (true). The description adds critical behavioral detail not in annotations: pads are ephemeral (lost on server restart or after 24 hours of inactivity), and an empty result is ambiguous. This fully discloses behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place: first states the action, second explains ephemeral nature, third clarifies ambiguity. No wasted words, front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description implicitly indicates the return is the plain-text content (by stating what is read). It covers the ephemeral nature and ambiguity. It could be slightly more explicit about the output format, but is otherwise complete for a simple read tool with good annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage for the single parameter 'id', including a regex pattern and description. The description does not add additional meaning to the parameter beyond what the schema provides, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Reads' and resource 'plain-text content of a pad', clearly distinguishing this from sibling tools like 'get_document_info' (metadata) and mutation tools like 'create_document' or 'set_document'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides important context about when to use the tool by noting that empty results are ambiguous and pads are ephemeral. It implicitly advises against relying on a non-existent pad result, but does not explicitly state when to use alternatives like 'get_document_info'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_languageA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description clearly states 'Last writer wins; the change is visible to everyone who has the pad open,' which provides critical behavioral context beyond the annotations. Since annotations declare readOnlyHint: false (confirming mutation) and openWorldHint: true, the description adds value by explaining the real-time, collaborative conflict resolution behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with zero wasted words. The first sentence states the tool's core purpose with examples, and the second adds critical collaborative behavior. This is highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only 2 parameters with full schema coverage and no output schema, the description is largely complete. It covers purpose, parameter values, and concurrency behavior. It could potentially note that no return value confirmation is provided (useful for confirmation), but overall it is thorough for this scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents both parameters well. The description reinforces the language parameter with examples ('markdown', 'javascript', 'rust') and specifies it uses Monaco IDs, which adds practical guidance beyond the schema pattern. It does not add details about the id parameter beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool sets the syntax-highlighting language of a pad using Monaco language IDs, which is a specific verb+resource combination. It also includes examples of valid language IDs and a behavioral note about visibility, clearly distinguishing it from sibling tools like get_document or set_document.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool should be used when you want to change the syntax highlighting language for collaborative editing, and notes that the change is visible to all viewers. However, it does not explicitly state when not to use it (e.g., for read-only access or other document features), nor does it reference sibling tools for context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_documentA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare destructiveHint=true and readOnlyHint=false. The description adds crucial context beyond annotations: it explains the confirmation token mechanism for non-empty pads, including the two-call workflow. This fully discloses the tool's behavioral traits for mutation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences with no fluff. The first sentence states the core purpose. The second explains the critical workflow. The third provides usage guidance. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema, the description is complete given the tool's complexity and rich annotations. It covers the purpose, workflow, and alternatives. The required vs. optional parameters are clear from the schema, and the confirmation token mechanism is fully explained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters. The description does not add new meaning about individual parameters beyond stating the confirmation token workflow, which is already implied by the schema's description of the confirm_token field. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Replaces') and explicitly names the resource ('entire content of a pad'). It clearly distinguishes from the sibling 'replace_in_document' by highlighting the scope difference (whole content vs. targeted changes).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear guidance on when to use this tool (replacing entire content) vs. the alternative 'replace_in_document' (targeted changes, preserves concurrent edits). It also explains the confirmation workflow for non-empty pads, enabling correct agent reasoning.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_statsA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint: true and openWorldHint: true. The description goes beyond annotations by detailing exactly what fields are returned (start time, memory count, persistence count) and explicitly flags the persistence count behavior under zero-persistence mode. This adds complete and valuable behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the verb and resource, immediately states the scope, and lists all returned data points concisely. Every word earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and no output schema, the description fully covers what the tool does, what it returns, and a behavioral note (zero-persistence mode). No gaps remain for an agent to be uncertain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters and 100% description coverage, so schema is complete. The description adds no parameter info, but no further meaning is needed—there are no parameters to document.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a specific verb ('Reads') and resource ('Rustpad server statistics'), then lists the three exact data points provided: start time, in-memory documents, and persisted documents. It clearly distinguishes this read-only server-level tool from the document-level siblings (get_document, get_document_info, etc.).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly states what the tool does and implicitly sets context for when to use it (checking server health/state). However, it does not explicitly state when not to use it or contrast it with similar siblings (e.g., get_document_info for document-level stats). Given the zero-parameter, read-only nature, the lack of exclusions is acceptable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.