Server Quality Checklist

Tool Scores

• unpublish_appB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It implies a state change but does not mention whether it is reversible, requires specific permissions, or has side effects on app accessibility. Given it is a mutation tool, this gap is significant.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no filler. It is appropriately sized for the tool's simplicity and front-loads the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one parameter) and has an output schema, so the description does not need to explain return values. However, it lacks context about prerequisites (e.g., app must exist and be published), reversibility, and side effects, making it only partially complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has a single 'name' parameter with 0% coverage, and the description does not add any information about it. While it is reasonably inferable that 'name' refers to the app name, the description fails to explicitly state this or provide format/validation context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Put an app back behind the login' clearly communicates that this tool reverts an app to a private, login-protected state. The verb 'put back' combined with the resource 'app' distinguishes it from siblings like publish_app, delete_app, and update_app.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit when-to-use or alternative guidance. The word 'back' implies the app was previously published, but there is no mention of prerequisites, exclusions, or when to choose this over publish_app or delete_app.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_appA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It discloses the core mutation (replace files) and the side effect of redeployment, which is good. However, it omits details such as whether existing files are permanently overwritten, any permission requirements, or what happens if the app does not exist. The description is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that conveys the essential action in under ten words. Every word earns its place, and there is no redundancy or extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema, so return values need not be explained, but the description is minimal for a mutation tool with no annotations and 0% schema coverage. It does not address prerequisites (e.g., app must exist), destructive implications beyond 'replace', or deployment process details. The tool has multiple parameters and side effects, yet the description covers only a fraction of the operational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must compensate by explaining the parameters. It only hints at the 'files' parameter by saying 'Replace files', but it does not explain 'name' (which app to update) or 'message' (commit message). The file structure is also left undefined. The description adds marginal value but fails to meaningfully clarify the parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly and specifically states the tool's function: replacing files in an existing app and redeploying it. The verb 'replace' and resource 'existing app' clearly distinguish this from siblings like create_app (new app), delete_app (remove), and publish/unpublish (deploy state changes).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys clear context for use: it targets an existing app ('in an existing app'), which implies it is for updates rather than creation or deletion. However, it does not explicitly exclude alternatives or provide explicit when/when-not guidance beyond that, so it gets a 4 rather than a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_appA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must bear the full burden. The verb 'Get' suggests a read-only operation, and the description lists return fields, but it does not disclose error behavior, authentication needs, or rate limits. This is acceptable for a simple read but lacks explicit safety confirmation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is front-loaded and free of fluff. It directly states the tool's purpose and output.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With a single parameter and an output schema present, the description provides sufficient context for an agent to invoke the tool. It lists the returned fields, and the simplicity of the operation means extensive explanation is unnecessary. However, it lacks error-case or alternative guidance, so not a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage for the single 'name' parameter. The description clarifies that the operation is for 'one app', implicitly linking the name to an app identifier, but does not explain the expected format or constraints. It adds some meaning beyond the schema but not comprehensive.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the verb 'Get' with a specific resource 'app' and lists the return data (status, URL, repository), distinguishing it from sibling tools like list_apps (list) and delete_app (delete).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies single-app lookup but does not explicitly state when to prefer this over list_apps or other alternatives. No when-not-to-use guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_appA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It clearly explains that the name is server-generated and cannot be chosen, that files must include index.html, and that the app is served as static content by nginx. It omits details like error conditions or side effects beyond publication, but the core behavior is well covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded, with the primary purpose in the first sentence. Each paragraph adds meaningful detail—file format, naming constraints, and deployment environment—without redundancy. It is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is quite complete for a creation tool with an output schema: it covers the required file structure, naming behavior, and deployment constraints. The main gap is the missing explanation of the requested_by parameter, and it doesn't mention potential error cases, but overall it informs the agent sufficiently for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It provides a precise format for the files parameter and explains the purpose of description, but requested_by is not mentioned. This leaves one parameter semantically undocumented, making the compensation incomplete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates and publishes a new static web app and returns its generated name and URL. It is specific about the resource (static web app) and action (create/publish), but it does not explicitly differentiate from the sibling tool publish_app, which could overlap in meaning.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool: for creating a new static site, requiring an index.html, and it explicitly states the limitation of no server-side runtime or database. It does not name alternative tools directly, but the requirements and exclusions give practical guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_appsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It accurately describes the main behavior (listing apps) and implies a read-only operation, but it does not mention authentication requirements, rate limits, pagination, or potential errors. For a simple list operation, this is adequate but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is front-loaded with the action and resource, containing no fluff or redundant information. It is maximally concise while conveying the essential purpose and output.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters), the presence of an output schema, and the straightforward purpose, the description is complete. It states what the tool does and the key fields returned, needing no further elaboration for an agent to select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema is empty, so the description need not explain parameter syntax. The baseline for zero parameters is 4, and the description does not need to add anything beyond what it includes.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists apps published by this server, with a specific verb ('list'), resource ('apps'), and scope ('published by this server'). It also mentions the output fields (status and URL), distinguishing it from siblings like get_app (single app) and create/update/delete mutations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need an overview of all apps on the server, but it does not explicitly state when to use this tool over alternatives like get_app for a single app. There are no direct exclusions or alternative tool mentions, leaving the usage context partially implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_appA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. It transparently reveals a key nuance: the source repository is kept unless 'delete_repository' is true. This is valuable behavioral context beyond the schema. It does not mention irreversibility or side effects, but the key conditional behavior is clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, front-loaded with the core action and immediately followed by the clarifying detail. Every word earns its place, with no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple delete operation, the description covers the primary action and the important conditional behavior. It does not detail return values or side effects, but an output schema is present and the operation is straightforward. It is sufficient for safe invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description directly explains the 'delete_repository' parameter, which is the non-obvious one, by stating that the repository is kept unless it is true. The 'name' parameter is not explicitly described, but it is self-evident from the tool name and context. With 0% schema coverage, this description compensates well for the critical parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Delete an app.' It uses a specific verb and resource, and the added nuance about the repository distinguishes it from sibling tools like unpublish_app or update_app.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for deleting apps, and the repository caveat gives some conditional guidance. However, it does not explicitly state when to choose this over alternatives (e.g., unpublish_app for temporary removal) or mention any prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_deploy_logsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the read-only nature via 'Return' and the 'most recent' scoping, which are important traits. However, it omits details about error behavior, auth requirements, or what happens when no logs exist, leaving some behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that leads with the verb and resource, then supplies the use case. Every word earns its place, with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one required parameter) and has an output schema, so the description need not detail return values. It covers the core use case, the 'most recent' limitation, and the trigger scenario. Minor edge-case omissions are not critical given the low complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description's phrase 'for an app' implies that the single 'name' parameter refers to the app whose deployment log is needed. This provides essential semantic context that the schema's bare 'name' string lacks, compensating well for the low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Return'), the resource ('deployment log'), the scope ('most recent'), and the target ('for an app'). It distinguishes itself from sibling app management tools by focusing on log retrieval, making it unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides a when-to-use trigger: 'use when a build fails.' It does not mention alternatives or exclusions, but no sibling tool serves the same log-retrieval purpose, so the guidance is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• publish_appA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully carries behavioral disclosure. It reveals the repository scan, refusal on sensitive data, public readability of the result, and the pattern-filter limitation. This is rich, safety-relevant context that goes beyond simple intent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences, each adding distinct value: purpose, automatic scan/refusal, limitation of the scan, and a practical caution. It is front-loaded with the intended action and avoids fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a security-sensitive publish operation with an output schema, the description covers the core action, safety checks, behavioral limitation, and user guidance. It does not describe return values, but the output schema exists, so no major contextual gap is evident.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one required string parameter, 'name', with no description. The tool description never mentions this parameter, and with 0% schema coverage it does not compensate; an agent must infer that 'name' refers to the app to publish.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource+outcome: 'Make an app reachable from outside without a login.' This clearly distinguishes publish_app from siblings like unpublish_app, list_apps, and delete_app.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context for when to use the tool: to expose an app publicly without login. It also warns to read the app's content before requesting publication, but does not explicitly name alternatives or provide a direct 'when not to use' statement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.