Server Quality Checklist

Tool Scores

• append_to_documentA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare the tool as non-read-only and non-destructive, and the description adds the concurrency-safety note ('including concurrent edits') and the verbatim handling of text. It does not disclose potential errors, permissions, or return behavior, but given the annotations cover the safety profile, the added context is modest.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary purpose, and contains no filler. Every phrase adds meaning: the verbatim behavior, the concurrency guarantee, and the newline tip.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with only two parameters, both documented, and an output schema is present. The description covers the core behavior, concurrency safety, and a usage nuance. It does not mention error handling or authentication, but these are not critical for an append operation given the existing annotations and schema coverage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides complete descriptions for both parameters (id and text), so the baseline is 3. The description adds a practical usage tip (include a leading newline to start a new line) that goes beyond the schema's text description, providing real value for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('appends text to the end of a pad') and identifies the specific resource (a pad). It explicitly contrasts with set_document and replace_in_document by noting that everything else, including concurrent edits, remains untouched, making the tool's unique purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (adding content to existing pads) and gives a concrete tip about leading newlines, but it does not explicitly name alternative tools or conditions for when to prefer this over set_document or replace_in_document. The guidance is helpful but not fully explicit on tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_documentA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only and idempotent behavior. The description adds valuable behavioral context about pad ephemerality and the ambiguity of empty results, which is not present in the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, front-loaded with the core purpose, and uses three short sentences to convey important caveats without unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple one-parameter input and presence of an output schema, the description adequately covers the tool's purpose, key edge cases (empty vs. nonexistent pad), and relevant behavior. No critical information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully describes the single 'id' parameter with a clear description. The tool description does not add additional parameter semantics, so a baseline of 3 is appropriate given high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's verb ('Reads') and resource ('current plain-text content of a pad'), and the title reinforces this. It is easily distinguished from siblings like get_document_info and set_document.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for reading pad content but does not explicitly state when to use this tool versus alternatives. Sibling names provide context, but the description itself offers no direct 'use this when' or exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_document_infoA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark read-only/idempotent; the description goes further by noting the collaboration-socket nature, the live user list, and the ephemeral storage caveat. It does not discuss error behavior, but the key side effects and data lifetime are disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no filler. The first states the operation and output fields; the second adds a caveat that belongs with the tool's context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only metadata lookup with one well-schema'd parameter and an output schema, the description covers what the tool returns and an important availability caveat. No critical context for invoking it appears to be missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter id is fully documented in the schema with format, length, and meaning. The tool-level description repeats no additional parameter constraints or dependencies, so schema carries the semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The purpose is explicit: 'Fetches metadata about a pad' rather than content. The title 'Inspect a pad' and listed fields clearly differentiate it from get_document.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes the read-only metadata use case clear, but it does not explicitly contrast with sibling tools or state when to prefer it over get_document/get_stats. The ephemerality caveat adds useful context, so there is some guidance beyond raw output.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_languageA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate this is a write operation (readOnlyHint=false) and idempotent (idempotentHint=true). The description adds valuable behavioral details beyond those: 'Last writer wins' and 'the change is visible to everyone who has the pad open'. This discloses concurrency and visibility effects that are not encoded in the annotations, enhancing the agent's understanding of side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero waste. The purpose is front-loaded, and the behavioral caveat is delivered concisely. Every word earns its place, and the structure makes it easy for an agent to quickly grasp both action and side effects.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool with a clear output schema (even though its details are not shown here), the description covers all essential aspects: what it does, its effect on other users, and concurrency semantics. The schema covers parameter specifics, so nothing an agent needs to invoke it correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides thorough descriptions for both parameters, including regex patterns, length constraints, and examples. The description adds no extra meaning about the parameters themselves—it only reiterates that language uses Monaco IDs. With 100% schema coverage, the description meets the baseline without needing to compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource ('Sets the syntax-highlighting language of a pad') and provides concrete examples of valid values (markdown, javascript, rust). It is clearly distinct from sibling tools like set_document, which likely handles content, making the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains what the tool does and mentions a concurrency behavior (last writer wins), which gives some context for when it applies. However, it does not explicitly contrast this tool with siblings such as set_document or append_to_document, nor state when not to use it. An agent must infer that this is exclusively for changing language, not content.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_statsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already establish readOnly, idempotent, and non-destructive behavior; the description adds concrete detail about what is returned, including the special case of 0 for persistence when not enabled. This goes beyond the annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, tight sentence with no redundant phrasing. Every clause adds useful information: the action, the subject, and the exact statistics returned.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides enough context for an agent to know the output contents and the special persistence case. It does not describe formatting or error conditions, but for a parameterless read-only stats tool this is sufficient and not a significant gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters and the schema coverage is trivially complete, so the description needs no parameter documentation. The baseline score of 4 applies because there is nothing missing.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description opens with the specific verb 'Reads' and names the exact resource, 'Rustpad server statistics'. It enumerates the returned metrics (start time, in-memory document count, persisted document count), leaving no ambiguity about the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes clear this is a read-only monitoring call with no parameters, so an agent can infer when to invoke it for server status. It does not explicitly contrast with sibling document tools, but none of them overlap with server-level statistics, so the use case is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_documentA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations, the description reveals crucial behaviors: pads are ephemeral (lost on restart and after 24h without connection), anyone with URL can read/edit, and creation is implicit via writing. These are not represented in the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is somewhat verbose (~90 words) but each sentence adds essential context (ephemeral nature, access control, no create operation). The main purpose is front-loaded in the first sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers output (returns URL), behavior (ephemeral, access), alternatives, and parameter semantics. The description is sufficient for an agent to decide when and how to invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the description adds no information about parameters beyond what the schema already provides (e.g., id optional, language format). Baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates a pad with optional initial content and language, and returns a shareable URL. It also clarifies that no dedicated create operation exists, distinguishing it from set_document and append_to_document.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly mentions alternatives: 'set_document or append_to_document on a fresh id makes one just the same.' Also implies when to use this tool (when a random id is needed, or when setting initial content/language together).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• replace_in_documentA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses destructive nature (replaces content) and concurrency safety (only matched ranges edited). Explains the confirmation flow for multiple matches, which goes beyond the destructiveHint annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise and well-structured: three sentences covering core function, concurrency, and confirmation behavior. No unnecessary fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, return values need no explanation. The description adequately covers the two-step confirmation protocol and edge cases (multiple matches, unique match requirement), making the tool's operation clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all parameters with descriptions; the tool description adds useful context for confirm_token (from previous call, omit on first) and replace_all behavior, enhancing schema semantics without redundancy.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool replaces an exact string in a pad, specifying the resource and action. Distinguishes from sibling tools like set_document or append_to_document by focusing on targeted replacement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides concrete conditions for use: default requires a unique match, replace_all for multiple occurrences, and confirm_token for confirmation. Implicitly contrasts with whole-document edits by noting concurrent edits survive, but does not explicitly name alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_documentA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description transparently mentions the confirmation requirement for non-empty pads and implies destructive behavior by stating it replaces entire content. The destructive annotation is not contradicted, but the description could have explicitly stated the idempotency aspect covered in annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences convey the core action, confirmation logic, and alternative recommendation with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with a confirmation mechanism, the description adequately covers the necessary workflow and points to the alternative for targeted edits. The presence of an output schema means return values need not be described.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully describes id, text, and confirm_token with individual descriptions. The description adds meaningful context for confirm_token by explaining its role in the confirmation flow, though it does not restate parameter details already in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's primary action ('Replaces the entire content of a pad') and distinguishes it from the alternative tool replace_in_document, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs when to use this tool versus replace_in_document and clearly explains the two-step confirmation process for non-empty pads ('call once to receive a token, then again with that token').

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.