Server Quality Checklist

Tool Scores

• delete_fileC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description must carry the full burden of behavioral disclosure. It only states 'Delete' without explaining recursive behavior for directories, irreversibility, permission requirements, or side effects. This is a significant gap for a destructive operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence and is not verbose, but it is under-specified rather than concise. There is no structure or front-loading of critical information beyond the basic operation. It could be more informative without becoming lengthy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with two parameters and no output schema, the description is incomplete. It does not clarify whether deleting a directory is recursive, what happens if the path does not exist, or any error conditions. The agent would have to infer behavior from the name and parameters alone.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not mention either parameter (app, path). It offers no explanation of what 'app' or 'path' refer to, their formats, or how they interact. The agent must guess entirely from the schema names, which is insufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('Delete') and resource ('a file or directory inside an app'), which distinguishes it from sibling delete_app. However, it does not explicitly name the alternative or contrast with other tools, so it misses the top score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives like delete_app or list_files. No context is given about prerequisites, safety checks, or when deletion is appropriate. The description is purely a statement of function.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• preview_logsC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It only says 'Recent stdout/stderr' which implies a read operation, but it does not state whether the tool is read-only, whether it has side effects, what happens if the preview isn't running, or any error conditions. This is a significant gap for a debug tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no wasted words, making it concise. However, it lacks any structure (e.g., separate usage notes, parameter explanations) and packs only minimal information. It's appropriately short but could be organized better without adding length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool, the description is barely adequate. It does not mention prerequisites (e.g., that the preview must be running), what the logs look like, or how 'lines' behaves. With no output schema and no annotations, the agent is left without critical context for debugging scenarios, making the description incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the tool description does not explain either 'app' or 'lines'. The schema only provides types and constraints; the agent must infer that 'app' identifies the preview and 'lines' controls the number of log lines. The description adds no semantic meaning beyond the parameter names, failing to compensate for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states it returns 'Recent stdout/stderr from a running preview', which is a clear verb-noun combo indicating the tool retrieves logs for debugging. It distinguishes itself from sibling tools by focusing on preview log access, though it doesn't explicitly name alternatives. The purpose is unambiguous and not a tautology.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'use this to debug' gives a clear contextual hint for when to employ the tool. However, it does not mention conditions like requiring a running preview, nor does it state when not to use it or point to alternative tools (e.g., for deployed logs). The guidance is implied but not explicit about boundaries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states the action 'Read', which implies a non-destructive operation, but it does not reveal behavior on missing files, encoding, return format, or permission requirements. This is minimal and leaves significant uncertainty.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with zero fluff. The verb 'Read' appears immediately, and the entire statement is efficient. It earns full marks for conciseness, though this comes at the cost of completeness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given two parameters, no output schema, and no annotations, the description is far too sparse. An agent needs to know how to specify 'app' (e.g., from create_app/list_apps), the expected path format, and what the tool returns—none of which are covered. The description is incomplete for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 50%: the 'path' parameter has a schema description, but 'app' does not. The tool description adds no explanation for either parameter, failing to compensate for the missing 'app' documentation. An agent cannot tell what value to pass for 'app' or the expected path format from either source.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Read'), a resource ('a file'), and a container ('an app'), which clearly distinguishes it from siblings like write_file, list_files, and delete_file. It is concise and unambiguous, though it does not elaborate on what constitutes an 'app' or the file types supported.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, such as list_files (to enumerate files) or write_file (to modify). It does not mention any prerequisites, exclusions, or typical scenarios, leaving the agent to infer usage solely from the name and purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• stop_previewC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states that it stops a running preview, which is essentially the tool's name restated. It does not mention side effects, what happens if no preview is running, whether it is reversible, or any error conditions—information an agent might need.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no unnecessary words. It is well front-loaded, stating the action immediately. However, its brevity borders on under-specification, which is a trade-off, but it earns points for efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a required parameter, no annotations, and no output schema, this description is severely incomplete. It does not describe the parameter, any prerequisites, expected outcomes, or edge cases. An agent would have to guess how to call it correctly, making it inadequate for autonomous use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema includes a required 'app' string parameter, but the description provides zero explanation of what 'app' represents, how it should be formatted, or where to obtain it. With schema description coverage at 0%, the description fails to compensate, leaving the agent completely uninformed about the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (stop) and the resource (preview), and adds the condition 'running' which distinguishes it from merely starting or viewing a preview. It differentiates from siblings like start_preview by implying the complementary action, though it does not explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a preview is running, but it provides no explicit guidance on when to use this tool versus alternatives, when not to use it, or any prerequisites. The sibling list suggests start_preview is the obvious pair, but the description itself does not reference it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• install_dependenciesC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must convey behavior and safety. It states 'Run npm install' but does not disclose that this modifies package files, may access the network, or could have side effects. The agent is left unaware of the tool's impact.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—one short sentence—with no fluff. However, it is terse to the point of brevity, though that does keep it easy to parse. The structure is fine, but it sacrifices necessary detail elsewhere.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple one-parameter tool, the description still omits critical context: what 'app' refers to, what installation entails, and expected outcomes. Without output schema or richer description, the agent has insufficient information for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one required parameter 'app' with zero description coverage. The description does not explain what 'app' represents (e.g., an ID or path) or how to obtain it, so the agent cannot correctly construct the argument.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action 'Run npm install' and the target context 'in the app directory.', which is specific both in verb and resource. It is distinct from sibling tools like read_file or deploy, so an agent can immediately understand its function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no indication of when to use this tool versus alternatives such as create_app or set_env. It lacks any context about prerequisites, when dependency installation is needed, or that it should be called after app creation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_appB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing destructive behavior. It does say 'permanently delete' and 'and its data', conveying irreversibility and scope. However, the confirm requirement is already documented in the schema's confirm parameter description, so this adds no new information. No other behavioral traits (e.g., cascading effects, permission requirements) are disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that efficiently communicates the core action and the critical prerequisite. No filler or redundant phrasing; every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with no annotations and no output schema, the description covers the essential destructive nature and confirmation requirement but omits details like how the app is identified, whether deletion is reversible (though 'permanently' implies it), and any side effects on related resources. Given the low complexity (2 params, no nested objects), it is adequate but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 50% (confirm is described; app is not). The description mentions only the confirm requirement (redundant with schema) and provides no clarification on what the 'app' parameter should be (e.g., app ID, name, path). This leaves the agent without guidance for the 'app' parameter, which is a significant gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('delete') and resource ('an app and its data'), clearly distinguishing it from siblings like delete_file. The 'app' noun makes the target unambiguous, though it does not explicitly name the alternative tool for files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides a prerequisite ('requires confirm=true') but no explicit guidance on when to use this tool versus alternatives. It does not mention any context where delete_app is preferred or excluded, leaving the agent to infer based on the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_envB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must disclose behavioral traits. It clearly states that values are never returned, which is a key security/privacy behavior. However, it does not explicitly state that the operation is read-only (non-mutating) or describe any pagination, ordering, or error behavior. It adds some value but remains thin.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, succinct sentence that front-loads the key behavior (listing keys) and the security constraint (never values). Every word earns its place, with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool is a simple list operation with one parameter and no output schema, the description is minimally adequate. However, it lacks details about return format, ordering, or any side effects (e.g., whether it might trigger a refresh). For a read-only list, this is borderline acceptable, but the absence of annotations raises the bar.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has a single parameter 'app' with no description (schema coverage 0%). The description says 'for an app' but does not clarify what form the app parameter takes (e.g., name, ID, slug). With such low schema coverage and zero parameter explanation in the description, the agent may struggle to know how to populate this field correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb (List) and resource (env keys) scoped to an app, and explicitly notes it returns keys only, never values. This is clear enough to distinguish from set_env, though it doesn't name the sibling tool explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like set_env. There is no mention of prerequisites, filters, or scenarios where this tool is preferred, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_emailB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full behavioral burden. It only states the action and a prerequisite, omitting any details about success/failure handling, whether sending is synchronous, error conditions, or other side effects beyond the email dispatch itself.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence that front-loads the action and includes a key prerequisite without excess words. It is appropriately concise, though it sacrifices necessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 5 parameters, no schema descriptions, no output schema, and no annotations, the description leaves critical gaps. An agent cannot determine parameter semantics, expected behavior on failure, or return values. This is grossly insufficient for reliable invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, and the description provides no explanation of the five parameters (app, to, subject, html, text). It doesn't clarify what 'app' refers to, expected formats for 'to' or 'subject', or the optionality/relationship of html and text. The description adds zero value for parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb (send) and resource (transactional email) and specifies the transport (SMTP). This clearly distinguishes it from sibling tools like push_send, which handles push notifications, and file or deploy tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives a concrete prerequisite (SMTP env vars) and implies the tool is for transactional email delivery. However, it does not explicitly mention alternatives or state when not to use it. Since no other email tool exists among siblings, the context is clear enough, but exclusions are absent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• write_fileB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It reveals that the tool can overwrite (destructive) and that parent directories are auto-created. However, it does not disclose consequences of overwriting, permission requirements, or error behaviors, leaving gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, precisely worded sentence with no redundancy. It front-loads the core purpose ('Create or overwrite a file') and adds one key behavior (auto-creation of parent dirs). Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple write tool with three params, the description covers the primary action and a notable behavior. However, it omits any parameter details beyond the path, and with no output schema, an agent may wonder about return values or error handling. It is adequate but leaves room for improvement.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 33% (only 'path' has a description). The description does not explain the 'app' or 'content' parameters. It only mentions 'in an app' without detailing the app parameter, and 'content' is never described. With low schema coverage, the description should compensate, but it does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb and resource: 'Create or overwrite a file in an app.' This distinguishes it from sibling tools like list_files, read_file, and delete_file by implying a write operation. However, it does not explicitly name any sibling tools, so it does not fully capitalize on differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives. It does not mention any conditions, exclusions, or workflows. An agent must infer from the name and siblings that this is for writing files instead of reading or listing them.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• push_sendB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the subscription input format well, but says nothing about the required preconditions (VAPID keys configured, subscription validity), delivery semantics, failure behavior, or whether notifications are queued or fire-and-forget. For a network-side-effecting tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One efficient sentence that front-loads the core purpose before the parenthetical format detail. No redundant filler, though the parenthetical is slightly convoluted (three nested clauses). It earns its place by clarifying the crucial subscription input contract.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Web Push is a protocol-heavy operation with dependencies — VAPID key generation (push_generate_keys), env configuration (set_env), and app existence (create_app) — none of which are hinted at. There is no output schema and no annotations, so the description is the only guide, and it omits these prerequisites. Incomplete for a tool with this much hidden setup.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 25% (only 'subscription' has a schema description), so the description must compensate. It does add clear meaning for 'subscription' by specifying the exact source format (PushSubscription.toJSON()), which goes beyond the schema. However, 'app' is left undefined — an agent cannot tell whether it's a name, ID, or identifier from create_app — and 'title'/'body' rely on common sense. Partial compensation only.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Send') and a precise resource ('Web Push notification to a browser subscription'), which clearly differentiates it from sibling send_email. The explicit reference to Web Push and browser subscriptions leaves no ambiguity about what the tool does, and it stands apart from push_generate_keys by being the send-side counterpart.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives implied workflow context — that the frontend supplies a PushSubscription.toJSON() payload — but never explicitly states when to use this tool vs. alternatives. It notably fails to mention the prerequisite relationship with push_generate_keys (VAPID keys must exist) or that the app must be pre-created via create_app. No when-not-to-use guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_filesB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It adds meaningful context about node_modules and .git being skipped, which is valuable. However, it does not mention whether the listing is recursive, whether directories are included, or what the return format is, leaving significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence that front-loads the primary action and includes a useful scoping detail. There is no irrelevant content; every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter, no output schema, and no annotations, the description omits critical details such as whether it returns a recursive file list, includes directories, or the structure of the result. This incompleteness could lead an agent to make incorrect assumptions about the tool's behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It indicates that 'app' identifies the app whose files are listed, which provides essential context. Yet it does not specify whether 'app' is a name, ID, or path, nor any format constraints, leaving ambiguity for the single parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List all files') and the resource ('in an app'), and clarifies a specific behavior (skipping node_modules and .git). It distinguishes from siblings like read_file (which reads content) and list_apps (which lists apps).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives like read_file or list_apps. The context where this tool is appropriate (e.g., enumerating files for an app) is only implied, not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_previewB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. It only mentions starting a preview and returning the URL, but does not disclose that this may start a background process, require dependencies to be installed first, or that the preview persists until stopped. Side effects and prerequisites are not covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with zero fluff. The main purpose is front-loaded, and the instruction to give the URL to the user is an extra useful detail. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description covers the basic return value and user-facing instruction. However, it omits critical context like prerequisites (e.g., dependencies installed), whether the preview blocks or runs in the background, and that it can be stopped via stop_preview. Adequate but with clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides only a string type for 'app' with no description (coverage 0%). The description does not explain what 'app' refers to (e.g., app ID, name) or its expected format. The agent must infer that it is the app to preview, but no further guidance is given.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Start'), the resource ('a live dev preview of the app'), and the output ('return its URL'). It distinguishes itself from siblings like stop_preview and deploy by specifying 'live dev preview'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The purpose implies usage (for previewing during development), but the description does not explicitly state when to use this vs. deploy or how it differs from other preview-related tools. No exclusion conditions or alternative tools are mentioned, so guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• deployA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the behavioral disclosure. It does state that 'local' builds and serves persistently and that cloud providers publish to a public URL, and it notes free tiers and custom domain configuration. However, it omits other critical behaviors such as authentication requirements, whether it overwrites existing deployments, and what the return value (e.g., a URL) looks like. Given the absence of annotations, this is only partially transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no superfluous words. It leads with the action and then concisely details the main modes. Every sentence earns its place, and the structure is front-loaded for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a deployment tool with two required parameters, no output schema, and no annotations, the description covers the core behavior but remains incomplete. It lacks details on prerequisites (e.g., provider CLI installed), the result of deployment (e.g., a pa URL), and potential side effects. While adequate for a basic agent, it does not provide a full operational picture.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers only 50% of parameters with descriptions, and the provider description is just an enum repetition. The description text does not add any semantic value to the 'app' parameter (what it refers to) or elaborate on the provider choices beyond the schema. With low schema coverage, the description should compensate, but it does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Deploy the app') and immediately distinguishes between local deployment and public hosting via three named providers. It is unambiguous about what the tool does and separates it from sibling tools like start_preview. The verb and resource are specific, and the provider distinction adds valuable differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the difference between 'local' and the cloud providers, giving implicit guidance on when to choose each. However, it does not explicitly mention alternatives or when not to use this tool (e.g., for temporary previews, one might use start_preview). The context is present but not explicit, leaving the agent to infer alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• server_statusA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It does not confirm whether the tool is read-only, has side effects, or requires special permissions. 'Everything currently running' is ambiguous, leaving the agent without clear expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence conveys the tool's entire purpose without waste. Highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter status tool with no output schema, the description is mostly sufficient. However, 'everything currently running' could be more specific (e.g., apps, servers, or processes) to fully set expectations, but the simplicity of the tool makes this acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema declares zero parameters, so there is nothing for the description to clarify. The baseline of 4 applies, and the description appropriately explains the tool's purpose without parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Show') and identifies the resource (workspace location and running items). It clearly distinguishes from siblings like list_apps (specific apps) and list_files (files) by providing an overall status overview.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus specific siblings like list_apps or preview_logs. The description does not mention alternatives or conditions that would lead an agent to choose this tool over others.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_appsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full burden of behavioral disclosure. It indicates a read-only operation through the verb 'List' and mentions the output (URLs), but it does not explicitly state that it is non-destructive, requires no special permissions, or describe any side effects. The agent can reasonably infer a safe read, but the description adds only minimal behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that immediately states the action and scope, and includes the critical output detail. There is no redundancy or filler; every word serves a purpose. It is optimally concise and structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description outlines the return value (URLs) but does not specify the structure of the result, such as whether it returns app names, IDs, statuses, or a list of objects. This could leave an agent with questions about how to parse the output, though the basic purpose is clear. Adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema is fully covered (empty object). The description correctly avoids parameter details since there are none. The baseline for a zero-parameter tool is 4, and the description does not detract from that.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the resource 'all apps in the Slipway workspace', and adds the key output detail of running preview/deploy URLs. This distinguishes it from sibling tools like list_files, which focus on files, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No usage guidance is provided. The description does not explain when to use this tool versus alternatives, mention any prerequisites, or give conditions or exclusions. The agent must infer from the name that this is for listing apps, but the description itself offers no supporting guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• push_generate_keysA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It reveals idempotency ('or return existing') and a security property ('private key never leaves this machine'), which go beyond the surface operation. However, it doesn't mention error scenarios or dependencies (e.g., whether the app must already exist), so it falls short of comprehensive transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary action and includes only essential details. No wasted words, and the security note is placed at the end without bloating the entry.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's purpose, idempotency, output, and a security aspect, which is decent for a simple tool. However, the unexplained 'app' parameter is a significant omission, and there is no mention of errors or prerequisites. Overall, it's adequate but not fully self-sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description does not mention the 'app' parameter at all. An agent cannot infer what value to pass (app ID, name, etc.) from the description. Since the schema provides no guidance either, this is a major gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Generate') and resource ('VAPID keys for Web Push'), and clarifies that it returns an existing set if present. This clearly distinguishes it from siblings like 'push_send' (sending) and 'create_app' (app creation). The purpose is unambiguous and not a tautology.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: it returns the public key for pushManager.subscribe() in the frontend, implying it should be called when setting up push notifications. It doesn't explicitly mention alternatives or when not to use it, but the context is sufficient for an agent to select it appropriately among the listed siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_databaseA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior. It does reveal that SELECT returns JSON and DDL/DML returns change info, and it notes the file is shared with the server, implying side effects. However, it does not mention error handling, transactionality, potential destructive impact, or any rate limits, leaving some behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no waste. The primary purpose is front-loaded, the return behavior is stated succinctly, and the server-sharing note is useful context. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given two required parameters, no annotations, and no output schema, the description covers the core purpose and return types, and provides the file path. Missing context includes how errors are reported, whether SQL is executed in a transaction, and any read/write limitations. The server-sharing note is helpful but overall it leaves some operational details unspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must clarify parameters. It explains 'app' through the path (<app>/data.sqlite) and implies 'sql' is the query string. It doesn't explicitly define each parameter's format or constraints, but provides enough context to infer meaning. This is average compensation for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Run SQL') and a specific resource ('the app's built-in SQLite database'), and clarifies the file path. It distinguishes itself from siblings by being the only database-related tool, and the behavior of SELECT vs DDL/DML adds clear scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly indicates when to use it (for any SQL operation on the app's database) and contrasts its effect (SELECT vs DDL/DML). It doesn't mention alternatives, but since no direct alternative exists, the context is clear. The statement about the server using the same file hints at when to be cautious, though not explicitly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_envA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that values are secrets/config, that they are injected on preview and deploy, and that 'Values are never echoed back'—an important non-obvious behavior. This goes beyond the schema and offers useful transparency, though it does not cover overwrite semantics or auth requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, each earning its place: the first states the action and resource, the second explains timing and a key behavior. It is front-loaded with purpose and avoids extraneous detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations, no output schema, and 0% schema description coverage, the description is too sparse. It leaves open questions: what happens if the key already exists (overwrite?), is the app required to pre-exist, are there naming constraints, and are there any side effects. An agent might call it incorrectly without these details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, but it only says 'secret/config value in the app's .env' without explaining the individual parameters (app, key, value) beyond their names. It does not provide formats, constraints, or interaction details, leaving the agent to infer from context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Set' and the resource '.env' for a specific app, and it explicitly mentions 'secret/config value' which differentiates it from related tools like list_env (which lists) and write_file (which writes files, not environment variables).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It specifies when the value takes effect ('Injected into the app's process on preview and deploy'), which gives concrete context for use. However, it does not explicitly state alternatives or when not to use this tool, though the sibling tool names (e.g., list_env) imply a read counterpart.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_appA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full disclosure burden. It specifies the templates and mentions one behavioral prerequisite (react requires install_dependencies before previewing), which is helpful. Yet it does not describe side effects like files created, name collision behavior, or reversibility. This is a moderate disclosure for a scaffold tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence followed by a compact bullet-style list. It wastes no words and presents the template options in a scannable format. Every sentence adds information, making it appropriately sized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 2-parameter tool with no output schema, the description covers the essential purpose, template choices, and one operational prerequisite. It does not mention post-creation steps (e.g., deployment) but the sibling tools fill that gap context. The information is sufficient for an agent to safely invoke the tool without needing external clarification.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds significant value by expanding each template option with a short explanatory blurb (e.g., 'static (Plain HTML/CSS/JS site. Zero dependencies, instant preview.)'). This goes beyond the enum names in the schema. The name parameter is already well documented in the schema, but the template details elevate the tool's usability.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'scaffold' and the resource 'a new app from a template', with a specific list of template types. It distinguishes itself from sibling tools like list_apps and delete_app, which are unrelated operations. No ambiguity about what this tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (to create a new app) by describing its core function. It also provides a usage note for the react template ('Run install_dependencies before previewing'), giving actionable context. However, it does not explicitly state exclusions or alternatives, so it misses the highest bar.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.