Server Quality Checklist

Tool Scores

• install_dependenciesC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full burden of behavioral disclosure. It states it runs npm install, implying a mutation of the environment, but does not reveal potential side effects like overwriting node_modules, requiring network access, or idempotency, which are critical for safe invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a concise single sentence that conveys the core action without redundancy. It is front-loaded and efficient, though it may be overly terse at the expense of necessary details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one required parameter and no output schema, the description is incomplete. It does not clarify the parameter semantics, side effects, or expected result, leaving an agent unable to confidently invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The sole parameter 'app' has no description in the schema (0% coverage), and the description only vaguely references 'app directory' without explaining what value 'app' expects (e.g., an ID or path). This is a severe gap for a required parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Run npm install') and the context ('in the app directory'), making it unambiguous about what the tool does. However, it does not explicitly differentiate from sibling tools like deploy, which might also install dependencies, so it lacks a distinctive contrast.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites (e.g., app must exist) or exclusions (e.g., when dependencies are already installed), leaving the agent without direction on appropriate conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_previewC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must fully disclose behavioral traits. It only states that a preview is started and a URL returned, omitting details like whether this is asynchronous, has side effects on existing previews, requires authentication, or has rate limits. There is no contradiction, but the description adds minimal behavioral context beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences and front-loads the core action. The second sentence, 'Give the URL to the user,' is arguably redundant with 'return its URL' but adds a directive and is not harmful. Overall, it is concise and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations, no output schema, and an undocumented parameter, the description is insufficient. It fails to explain the input parameter, lacks usage context, and does not clarify behavioral expectations. An agent cannot reliably know what to pass or what to expect beyond a URL.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain the 'app' parameter. It does not. The agent is left to guess whether 'app' refers to an app ID, name, or path. This is a critical gap for a required single-parameter tool; the description provides no semantic value for the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Start'), resource ('live dev preview'), and expected output ('return its URL'). It clearly differentiates this tool from siblings like stop_preview and share_preview by implying it initiates a new preview, though it doesn't name alternatives. The core purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use this tool versus alternatives such as share_preview or stop_preview. There is no mention of prerequisites, typical workflow, or conditions that would favor this tool. The agent must infer that starting a preview is a first step, which is not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_fileC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility for behavioral disclosure. It only says 'Delete a file or directory', which implies mutation but does not disclose permanence, recursion behavior, error handling, or safety implications. This is insufficient for a destructive operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence stating the core action with no waste. It is appropriately concise, though it could include more detail without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with 2 required parameters and no annotations or output schema, the description is incomplete. It does not clarify how to reference the app, whether directories are deleted recursively, or what happens if the target does not exist. An agent would need to guess these critical details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does not elaborate on what 'app' (ID vs. name) or 'path' (absolute vs. relative, file vs. directory format) mean. The parameter names give minimal hints, but the description adds no additional meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb (delete) and resource (file or directory inside an app), clearly distinguishing it from siblings like delete_app (which deletes an entire app) and read/write tools. The scope 'inside an app' further clarifies the target.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, nor any prerequisites such as requiring the app to exist or the path to be valid. No exclusions or mentions of alternative tools are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must convey behavioral details. It only states the basic operation without disclosing whether the file content is returned, how errors are handled, or any permission requirements. For a read operation, this is minimal and leaves important aspects undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the verb and object. It contains no superfluous words and is appropriately minimal for a simple read operation. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although the tool is simple, the description is incomplete for an agent to call it correctly. It does not mention the return value (presumably file content), potential errors, or clarify the 'app' parameter. With no annotations or output schema, these missing details leave ambiguities that could lead to incorrect calls.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 50% coverage (path is described, app is not), and the tool description adds no parameter information at all. The description does not explain what 'app' refers to or clarify the path format beyond the schema. It completely fails to compensate for the undocumented app parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (read) and resource (a file from an app), which is specific and unambiguous. It implicitly distinguishes from sibling tools like write_file, delete_file, and list_files by naming the read operation, making it evident when to choose this tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention that list_files could be used to discover available files first, nor any conditions or exclusions. The intended usage is only implied by the verb 'read,' leaving the agent to infer context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• undeployB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does state the key effect (no longer restarts with the server), which is a meaningful behavioral trait. However, it does not mention whether the operation is reversible (i.e., re-deploying), whether it affects running processes or data, or any permission requirements. It covers the core effect but lacks broader context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence that immediately states the action and then clarifies the consequence. Every word adds value, and there is no wasted text. It is front-loaded with the primary verb and subject, making it easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one parameter, no output schema), but the description is incomplete for successful invocation. It does not explain the expected format or meaning of the 'app' parameter, nor does it mention any conditions (e.g., app must be deployed already). While the action is clear, the lack of parameter guidance and any prerequisites means an agent may call it with incorrect input.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has a single required parameter 'app' with no description, and the schema description coverage is 0%. The description fails to clarify what 'app' refers to (e.g., app name, ID, path). There is no explanation of accepted values or how to reference the local deploy. The description adds no parameter-level meaning, leaving the agent without sufficient information to populate the argument correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (take down) and the resource (a local deploy), and adds a clarifying behavioral outcome (it will no longer restart with the server). This distinguishes it from delete_app (which likely removes the app entirely) and deploy (the reverse operation), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like deploy or delete_app. It does not mention prerequisites (e.g., that the app must be currently deployed) or scenarios where it is appropriate. Users must infer from the name and the effect that it is used to stop a local deployment, but no explicit routing is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• server_statusB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are absent, so the description carries the full burden. 'Show' implies a read-only operation, but there is no disclosure of return format, authentication requirements, or potential side effects. Very sparse.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence with no fluff, front-loading the core purpose. Perfectly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is too ambiguous about what 'everything currently running' entails (processes, apps, services, etc.). Without an output schema, the agent must guess the return content, leaving a gap in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is complete. No parameter explanations are needed; the baseline score of 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Show') and resource ('workspace location and everything currently running'), but the scope is broad and does not explicitly distinguish from sibling status-like tools (e.g., list_apps, preview_logs). The intent is clear enough for an agent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool or how it differs from alternatives. The description only states the function without indicating appropriate contexts or contraindications.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_filesB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the burden of behavioral disclosure. It does disclose that node_modules and .git are skipped, which is useful context. However, it does not mention any other behavior such as whether the listing is recursive, sorted, or what the return format is. For a simple read operation, this partial disclosure earns a 3, but it could have easily added a note about being read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with the core action front-loaded. It states 'List all files' immediately and then adds the exclusion in parentheses. Every word earns its place, and it avoids unnecessary fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description should clarify what the tool returns (e.g., list of file paths, boolean, etc.) and any relevant details. It does not mention the return format or how the app parameter is used. The tool is simple, but the description leaves out important details that an agent needs to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning the parameter 'app' has no description in the schema. The description does not explain what 'app' refers to or how to specify it. Since there is only one parameter, the description should clarify its meaning, but it only says 'in an app' without linking it to the parameter. This leaves the agent to guess the value format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (List), the resource (files), and the scope (all files in an app), with an explicit exclusion of node_modules and .git. It distinguishes itself from siblings like read_file, write_file, and delete_file by specifying the listing action. No ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool versus alternatives. It does not mention any conditions, limitations, or alternatives. It simply states what it does without helping an agent decide if this is the right tool among the many siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• deployB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that local builds persistently and cloud providers publish via CLI, and mentions free tiers and domain configuration. However, it does not cover side effects, permission requirements, failure modes, idempotency, or what happens to prior deployments—significant gaps for an unannotated tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with zero waste: the first states the action, the second details provider semantics. All information is front-loaded and directly relevant, earning every sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  It covers the essential purpose and provider options, but misses guidance on when to use deploy versus sibling tools like start_preview or undeploy, and does not mention any post-deployment behavior or return values. Given no annotations and no output schema, the description could reasonably provide more operational context to fully equip an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds substantial meaning to the provider parameter by explaining the outcome of each enum value (local = persistent build, cloud providers = public URL via CLI) and additional context like free tiers and domain setup. This goes well beyond the schema's simple enum listing. The app parameter is not elaborated, but its meaning is self-evident from the tool's context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool deploys an app and distinguishes local vs. cloud providers, making its core purpose specific. However, it does not explicitly differentiate it from sibling tools like start_preview, share_preview, or undeploy, so it lacks a strong sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explains the provider options (local vs. vercel/netlify/cloudflare) and notes free tiers and custom domain configuration, but provides no guidance on when to use this tool versus alternatives like start_preview or undeploy. It does not state any exclusions or conditions that would route an agent to a different tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_emailB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must convey behavioral implications. It discloses the environmental dependencies and that it works with any SMTP provider, which is useful. However, it does not state what happens on success/failure, whether it is idempotent, or any rate limits. It neither contradicts annotations (none) nor provides deep behavioral context, so a 3 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that states the purpose and the critical prerequisite. It is efficient with no fluff, though it could have added parameter details without becoming verbose. It earns a 4 for being compact and well-structured, but loses a point for skipping crucial semantic info.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with five parameters, no output schema, and no annotations, the description is insufficiently complete. It fails to define the required parameters, the role of 'app', or the relationship between html and text (e.g., which takes precedence). An agent would need to guess parameter formats and meanings, making this tool difficult to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning the description must compensate for parameter meanings. The description does not explain any of the five parameters (to, app, subject, html, text). 'app' is especially ambiguous, and the distinction between html and text fields is unaddressed. With zero parameter guidance, the description fails to help the agent construct a valid invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('send'), resource ('transactional email'), and transport ('via SMTP'). It clearly distinguishes from sibling tools like push_send (push notifications) by specifying email, so an agent can identify the correct tool without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear prerequisite: the required environment variables (SMTP_HOST, SMTP_PORT, SMTP_USER, SMTP_PASS, MAIL_FROM) must be present in the app's env. It implies usage when sending transactional email, and notes any SMTP provider works. It does not explicitly mention when not to use it or name alternatives, but given the sibling list, the intent is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• stop_previewB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the burden, and it adds the useful detail that the public tunnel is also stopped if shared. However, it omits behavior such as what happens if no preview is running, whether the operation is idempotent, or any permissions required. It is partially transparent but incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that directly states the action and the key extra detail. There is no wasted verbiage, and the structure is optimal for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool without an output schema, the description covers the core action but omits important context such as what 'app' refers to, error behavior, or requirements (e.g., preview must exist). Given the lack of annotations, the description is not fully self-sufficient, but it is adequate for the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage for the only parameter 'app', and the tool description does not mention the parameter at all. The description adds no semantic meaning beyond the raw parameter name, leaving an agent to infer that 'app' refers to the app whose preview should be stopped. This is a significant gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'stop' and the resource 'preview', and adds a specific detail about stopping the public tunnel if shared. This distinguishes it from siblings like start_preview or share_preview without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used when a preview is running and needs to be stopped, but it does not explicitly state when to use it versus alternatives, nor does it provide any exclusions or edge-case guidance. The context is clear to a human but lacks explicit routing instructions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• push_generate_keysA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It adds useful behavioral context: the operation may return existing keys or generate new ones, and it explicitly states the private key never leaves the machine. However, it does not disclose potential side effects (e.g., whether it overwrites existing keys) or error conditions, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. It front-loads the purpose, states the output, and includes the security constraint. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is incomplete for a one-parameter tool. It lacks explanation of the 'app' parameter, any prerequisites (e.g., does the app need to exist?), and does not describe the return format beyond the public key. Given no output schema and no annotations, the description should provide more context but does not.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage and the description does not mention the 'app' parameter at all. An agent calling this tool would not know what value to supply for 'app' (e.g., an app identifier, name, or path). The description fails to compensate for the missing schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific verb (generate or return) and resource (VAPID keys for Web Push), and it distinguishes itself from the sibling push_send by focusing on key generation rather than sending notifications. It also clarifies the output (public key) and a key constraint (private key stays local).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the use case: when you need VAPID keys for Web Push. However, it does not explicitly mention alternatives or when-not-to-use. Since push_send is clearly about sending, the context is clear, but no exclusions or alternative routing are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• push_sendA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears the full burden of behavioral disclosure. It states the core action (sending a push notification) and the required subscription format, but it does not disclose potential side effects, prerequisites (e.g., prior key generation), error conditions, or success/failure semantics. It is truthful but minimal, leaving out operational details an agent might need for robust use.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is efficient and front-loaded with the primary action. The parenthetical clarifies the expected subscription format without added fluff. It is appropriately sized for a simple tool, though a brief structuring into separate clauses could improve readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description provides the essential context for identifying the tool and the subscription parameter format. However, it leaves 'app', 'title', and 'body' undefined, and does not mention return values or error behavior. For a 4-parameter tool with low schema coverage, this is incomplete but not critically misleading.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 25% (only the 'subscription' parameter has a description). The tool description only elaborates on the 'subscription' parameter (the JSON from PushSubscription.toJSON()), which aligns with the schema. It provides no explanation for 'app', 'title', or 'body', despite the low schema coverage. Since the description does not compensate for undocumented parameters, it adds little value beyond parameter names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb and resource: 'Send a Web Push notification to a browser subscription'. It also specifies the exact input format (JSON from PushSubscription.toJSON()), which uniquely identifies this tool among siblings like push_generate_keys (key generation) and send_email (email). No ambiguity about what this tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it (when the frontend provides subscription JSON to the backend) but does not mention alternatives or when not to use it. It provides context about the subscription source but does not explicitly contrast with send_email or push_generate_keys. The usage context is inferable but not explicitly stated, so guidance is implicit rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_envA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It usefully states that values are never echoed back and are injected on preview and deploy. However, it omits critical behaviors such as whether setting an existing key overwrites it, whether permissions are required, and what the tool returns on success or failure. These gaps limit transparency for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. The core action is front-loaded, and the additional behavioral notes (injection, no echo) are directly relevant. The structure is efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is incomplete. It does not clarify overwrite semantics, persistence across previews/deploys, or what the response contains (even though values are never echoed, the tool may return a success indicator). The simplicity of the tool (3 string params) makes the missing behavioral details more noticeable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It references 'app's .env' (clarifying the app parameter) and 'secret/config value' (shedding light on the value parameter), but it does not explain the 'key' parameter (the variable name) or its format. The description partially clarifies two of three parameters but leaves the most essential one ambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States 'Set a secret/config value in the app's .env' with a specific verb ('Set') and resource ('.env'). The description clearly indicates the action and implicitly distinguishes from sibling list_env (read vs write). The purpose is unambiguous and specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives or when not to use it. The mention of injection on preview and deploy provides some context, but it does not direct the agent to, for example, use list_env to verify existing values or explain scenarios where this tool is inappropriate. Usage is implied rather than explicitly specified.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• write_fileA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior. It does state the destructive overwrite behavior and automatic directory creation, which is helpful. However, it omits potential errors (e.g., app not existing), return values, and any prerequisites, leaving gaps for an agent to discover.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, front-loaded with the core action (create or overwrite) and then a key behavioral detail. There is no redundancy or unnecessary text, making it as concise as possible while still informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple file-write tool, the description covers the basic function and directory auto-creation. However, given no annotations or output schema, it lacks details on return behavior, error handling, and whether the app must already exist. These would be expected for a complete definition.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 33% (only path is described). The tool description adds no details about the app or content parameters, relying on inference from the tool name. Since the schema is sparse, the description should compensate but does not, leaving ambiguity for two of three parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates or overwrites a file in an app, using a specific verb and resource. It distinguishes itself from sibling tools like delete_file and read_file by focusing on the write operation, so an agent can easily tell what it does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (when you need to create or overwrite a file) and provides a useful hint that parent directories are auto-created, but it does not explicitly compare with alternatives like read_file or delete_file. The when-not-to-use guidance is left to inference, making it adequate but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• preview_logsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It states that the tool returns recent stdout/stderr, which conveys a read-only operation. However, it does not disclose potential side effects, error behavior (e.g., if preview is not running), or any limitations like log retention. Since the operation is simple and read-only, the basic behavior is covered, but additional details are missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler. It front-loads the core purpose ('Recent stdout/stderr from a running preview') and immediately states the use case ('use this to debug'). Every word earns its place, making it concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with two parameters and no output schema, the description should at least clarify parameter usage and error conditions. It does not explain what 'app' means or how 'lines' behaves, nor does it mention what happens if the preview is not running. The description covers the basic purpose and behavior but leaves significant gaps for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, and the tool description does not explain either parameter (app or lines). It does not clarify what 'app' refers to (e.g., app ID, name) or what 'lines' controls (number of log lines). The parameter names are self-evident to some degree, but the description adds no value beyond the schema, leaving the agent to guess exact semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: retrieving recent stdout/stderr from a running preview for debugging. It uses a specific verb ('preview') and resource ('logs'), and the context of sibling tools (start_preview, stop_preview) makes its role distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: it is for debugging a running preview, implying it should be used when logs are needed. It also implies a prerequisite (the preview must be running). However, it does not explicitly mention when not to use it or name alternatives, but given the sibling list, the guidance is sufficient for an agent to infer correct usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_databaseA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that SELECT returns rows as JSON and DDL/DML returns change info, and notes the server uses the same file, which implies side effects. However, it doesn't explicitly warn that DDL/DML is destructive or irreversible, nor mention error handling or permission requirements. For a tool that executes arbitrary SQL, this is a notable gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, no filler. The main action and key behaviors are front-loaded, and the context about the server is useful. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is complex (arbitrary SQL execution) with no output schema, no annotations, and minimal parameter descriptions. The description covers the purpose and return behavior but omits critical operational details: potential for destructive writes, error handling, validation, and parameter specifics. An agent lacks enough information to use it safely and correctly in all cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It implies that 'app' refers to the app identifier (via the path <app>/data.sqlite) and 'sql' is the SQL statement, but it doesn't provide format, constraints, or examples. The explanation is minimal and doesn't help an agent know how to construct valid values beyond the obvious.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb (Run SQL) and resource (app's built-in SQLite database at <app>/data.sqlite). It clearly distinguishes from siblings since no other tool in the list executes SQL. The description is unambiguous about the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives context that the database is the app's built-in one and that the server uses the same file, implying this is the correct way to interact with the DB. However, it doesn't explicitly mention when not to use it or name alternative tools for other data operations, though no sibling tool covers SQL.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• share_previewA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses the main side effect (starting the preview) and a requirement (cloudflared CLI). However, it does not clarify whether the tool returns the URL or opens it in a browser, nor the lifecycle of the tunnel (e.g., whether it persists after the command returns). This partial transparency is a gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, front-loaded with the core action, and states the prerequisite at the end. It uses a clean structure with no wasted words, making it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema, no annotations), the description covers the basics but misses critical details: what the tool returns (the URL?), whether it opens a browser, and what happens to the tunnel after the command. An agent cannot fully predict the tool's behavior without this information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. The description references 'the app's preview' but does not specify whether `app` should be a name, ID, or path. It does not clarify the expected format or link to list_apps for obtaining valid values. This leaves the agent to guess the parameter's format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (open a public HTTPS URL), the resource (the app's preview), and the method (Cloudflare quick-tunnel). It differentiates from siblings like start_preview by focusing on public sharing, and the note that it 'Starts the preview first if needed' clarifies it handles the startup, so an agent knows it doesn't need to call start_preview first.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context that this tool is for sharing previews publicly and that it will start the preview if needed, implying you don't need to call start_preview first. It also states a prerequisite (cloudflared CLI). However, it does not explicitly name alternatives or exclude cases where sharing is not desired, so it lacks explicit when-not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_appA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly states the action is permanent and reversible (implies destructive), and requires confirm=true for safety. It also indicates app data is deleted. This is sufficient transparency for a delete operation, though it doesn't elaborate on permissions or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the core action and the critical requirement. No unnecessary words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides the essential fact (permanent deletion) and a safety requirement, but leaves gaps: the 'app' parameter is unspecified, and there is no indication of return values or failure behavior. For a destructive tool with no annotations, more context would be helpful, though it is minimally viable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 50%: only 'confirm' has a description. The tool description adds no clarification for the 'app' parameter, leaving its expected format (e.g., ID or name) undefined. The requirement 'confirm=true' reinforces the schema but does not compensate for the missing app semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('Permanently delete') and resource ('an app and its data'), making the tool's purpose unambiguous. It also differentiates from the sibling delete_file by specifying that it targets apps, not files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (when you need to remove an app) but does not explicitly contrast with alternatives like delete_file or specify conditions for choosing this tool. It mentions the confirm requirement, but no when-to-use or when-not-to-use guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_appsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the tool's output scope (apps and their URLs) but does not explicitly confirm read-only behavior, mention any limitations like pagination, or describe response format. For a simple list, this is acceptable but not overly transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the core action and resource, then adds the valuable detail about URLs. There is no redundancy or filler; every word contributes to clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity—no parameters, no output schema—the description is largely complete. It states the scope and the included URL information. The only minor gap is the absence of any note about response structure or potential edge cases, but for a simple list tool this is not critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and schema coverage is trivially 100% since the schema is an empty object. According to the rubric, a tool with 0 parameters receives a baseline score of 4. The description does not need to explain parameters, and it does not introduce any confusion.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the resource 'all apps', scoped to 'the Slipway workspace'. It adds specificity by mentioning that running preview/deploy URLs are included, which distinguishes it from other app-related tools like create_app or delete_app. This is a precise and unambiguous purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is the tool for listing apps, and siblings like list_files clearly target different resources. However, it does not explicitly state when to use this vs. alternatives, nor does it provide exclusions or conditions. The guidance is inferred rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_envA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals a critical security behavior—never returning values—and scopes the operation to a specific app. However, it omits details like output format, error handling, or authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that immediately states the action ('List the env keys') and the key constraint ('keys only, never values'). There is no fluff, and the critical detail is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity—one parameter, no output schema, and no annotations—the description covers the core purpose and the important behavior of not exposing values. Minor gaps remain, such as what happens if the app does not exist or the response format, but these are not critical for a straightforward list operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema lists a single required parameter 'app' with no description, and schema coverage is 0%. The description adds minimal context by referring to 'for an app,' indicating that 'app' identifies the target application. It does not specify the expected format (e.g., app ID vs. name) or provide additional details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists env keys for a specified app, and explicitly notes that only keys are returned, never values. This distinguishes it from set_env and other sibling tools that modify or generate keys, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide explicit guidance on when to use this tool versus alternatives like set_env or push_generate_keys. The read-versus-write distinction is implied but not stated, and there is no mention of exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• export_appA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly mentions that it performs git init, commit, gh repo create, and push, and that secrets are automatically gitignored. It also notes the user-consent requirement for public repos. It does not mention edge cases like existing repos or failure handling, but the core side effects are clearly surfaced, making it fairly transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two well-structured sentences. The main action and steps are stated first, followed by key details about secret handling and user consent. Every sentence adds value, with no redundancy. It is concise and front-loaded, making it easy for an agent to quickly grasp the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with three parameters and no output schema, the description provides essential context: the operation performed, the automatic handling of secrets, and the prerequisite for gh CLI. It also covers the user-consent requirement for public visibility. Missing details like default behavior if a repo already exists or what happens on failure are not critical for initial selection and invocation. Overall, it is sufficiently complete for an agent to use the tool correctly in most cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes repo and visibility (67% coverage), leaving only the app parameter undocumented. The description does not add parameter-specific details beyond the schema, but it does mention that secrets are gitignored, which is relevant to the overall operation. Since schema coverage is above 80% (actually 67%—still above the 50% threshold), the baseline of 3 is appropriate; the description does not need to compensate for low coverage, and it adds minimal extra parameter insight.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's purpose: turning an app into the user's own GitHub repository through git init, commit, gh repo create, and push. This is specific and distinguishes it from sibling tools like deploy or share_preview, which serve different functions. The use of concrete actions and the inclusion of secret handling make the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states the prerequisite of the gh CLI being logged in and instructs to ask the user before making a repo public, providing clear usage context. It does not explicitly list alternatives or when not to use it, but the purpose is distinct enough that an agent can infer when it is appropriate. It covers key conditions without being exhaustive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_appA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It notes dependency and preview behavior for each template (zero deps vs react needing install_dependencies), but it omits side effects like directory creation, potential overwrites, or response format. This is incomplete for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Compact and front-loaded with the core purpose, followed by a structured, semicolon-separated list of templates. Every sentence carries necessary information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers all five templates with dependencies and preview notes, sufficient for an agent to choose correctly and invoke. Lacking details on return value or filesystem side effects, but given the tool's nature and absence of output schema, it is fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameters, yet the description adds substantial value by detailing each template's contents, dependencies, and preview expectations, going far beyond the enum. Name semantics are adequately covered in schema, and template semantics are richly explained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb 'scaffold' and resource 'app from a template', clearly distinguishing it from sibling tools that handle file ops, deployment, etc. The description unambiguously identifies the action and object.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implicitly indicates when to use (when creating a new app) and provides template-selection guidance, including an explicit prerequisite for react (install_dependencies before preview). However, it does not explicitly state when not to use or name alternatives since none exist for creation, leaving a slight gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.