Server Quality Checklist

Tool Scores

• delete_fileA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It adds useful context: the deletion is local, auto-commits, and remote deletion via theme_push has a caveat to mention. However, it does not disclose what the tool returns, whether deletion is permanent/recoverable, or errors such as nonexistent files.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, front-loaded with the core action, followed by a necessary warning. No fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive mutation tool with no annotations and no output schema, the description leaves important gaps: parameter meanings are not stated, return/confirmation behavior is absent, and failure/edge-case behavior is unspecified. The remote-deletion caveat is helpful but not sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers none of the parameters, and the description does not explicitly explain 'filename' or 'message'. The phrase 'auto-commit' loosely implies 'message' may be a commit message, and 'theme file' implies what 'filename' refers to, but this is weak compensation for 0% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Delete'), resource ('local theme file'), and consequence ('auto-commit'). The word 'local' also distinguishes it from remote/theme_push operations, and the description is not just a restatement of the title.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool: to delete local theme files with auto-commit. It also gives a relevant warning about theme_push's remote deletion behavior, which helps an agent avoid misusing the sibling. However, it does not explicitly state when not to use delete_file or name alternatives for other deletion workflows.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• git_historyA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description must carry behavior. It usefully discloses that edits auto-commit and that the history is local, but it does not describe the return shape, ordering, or whether the command is strictly read-only. The verb 'Show' implies no mutation, but the absence of a full behavioral contract is a gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One focused sentence with a useful parenthetical; no filler. The core action and context are front-loaded and every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity tool with no required parameters and no output schema, this is minimally adequate: an agent can call it without arguments and understand it returns recent commits. It is incomplete in explaining the optional limit parameter and the exact nature of the returned commit data.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has a single 'limit' parameter with min/max constraints but no description, and the tool description never mentions it. The parameter name is self-explanatory, but the description adds no semantic value for it and does not state a default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Show') and resource ('recent git commits for the local theme'), and the parenthetical clarifies the commit model. This is enough to distinguish it from file content tools and from git_revert_file, which is about reverting rather than viewing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use is implied by the purpose: call this when you need the recent commit history of the local theme. However, it does not explicitly state when not to use it or compare against alternatives such as read_file or git_revert_file.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• git_revert_fileA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It discloses two key behaviors: restoring a single file and automatically committing the revert. This is important context that an agent needs before calling. However, it does not disclose whether the commit is created automatically with a generated message, whether the working directory must be clean, or what happens to uncommitted changes in that file—these are significant gaps for a mutating git operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. Every word earns its place: the action, the input source, the follow-up action, and the intended use case. It is front-loaded and easily scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a mutating git operation with no annotations, no output schema, and two parameters. The description explains the core flow but misses several things an agent would need to know for safe invocation: whether the revert is committed immediately, how the commit message is generated, and any preconditions like a clean working tree. Given the complexity of git operations, the description is not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50%: the 'commit' parameter has a description, but 'filename' has none. The description does not add extra meaning for the parameters, but it does indicate that the commit should come from git_history and the filename is the target file. This is about baseline level—it neither compensates for the missing 'filename' description nor adds meaningful detail beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action: restore one file to a state at a given commit and commit the revert. It names the resource (file) and the source (git_history commit), which makes the tool's purpose unambiguous. However, it does not explicitly distinguish it from other file-related git tools beyond mentioning git_history, so it loses one point for lack of sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes the use case 'Use to undo a bad edit' and explicitly mentions pulling the commit from git_history, which provides clear context for when to use this tool. It does not explicitly state when not to use it or name an alternative tool, so it is a strong but not perfect usage guideline.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• publish_themeA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It explicitly warns 'DESTRUCTIVE for the current live theme's status,' disclosing the key side effect. It doesn't mention reversibility or the exact fate of the old theme, but the most critical behavioral trait is clearly disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three tight sentences with zero filler. The action is front-loaded, the destructive warning is placed immediately after, and the confirm requirement concludes it. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter tool with no output schema, the description covers the action, the destructive side effect, and the required confirmation. Minor gaps include no mention of return values or what exactly happens to the old live theme beyond its status changing, but these are not critical for successful invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already thoroughly documents confirm, and the description reinforces its precondition ('confirm must be true') without adding new meaning. The theme_id parameter has no schema description and the tool description doesn't explain it either, though its meaning is fairly inferable from the tool's purpose. At 50% schema coverage, the description only partially compensates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action: making an unpublished theme the live storefront theme. This distinguishes it from generic file operations and defines a clear state change. However, it does not explicitly reference sibling alternatives like list_themes or create_theme, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit precondition: 'only call after the user has previewed the theme and explicitly asked to publish.' This is clear when-to-use guidance and implies when-not-to-use (do not call before user confirmation). It doesn't explicitly name alternative tools, so it loses a point for missing alternative routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It discloses that the operation is read-only, includes line numbers, and supports slicing, but it does not mention behavior for missing files, default full-file reading, or any pagination/error details. Some useful transparency, but gaps remain.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences with no redundancy. The action is front-loaded, followed by conditional usage guidance and the recommended workflow. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description adequately covers the core read operation, line numbering, range slicing, and the relationship to search_files. It omits explicit handling of errors or nonexistent files, but for a simple read tool without an output schema, this is a reasonable level of completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 33% because start_line and end_line lack descriptions. The description adds meaning by explaining that these parameters are for reading a slice of a large file, but it does not clarify edge cases like whether ranges are inclusive or what happens when only one is provided. Partial compensation for low schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads a theme file's contents with line numbers, which distinguishes it from sibling tools like write_file, patch_file, delete_file, and search_files that either modify or search instead of reading. The action and resource are specific and immediately understandable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to use search_files first to locate the relevant region and to pass start_line/end_line for large files. It provides actionable guidance for when to use this tool and how to scope reads, though it does not explicitly state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_filesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of conveying behavior. 'List files in the local theme directory' clearly signals a read-only, non-mutating operation, and the filter examples illustrate accepted forms. It does not describe return format or recursion behavior, but for a simple listing tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The first sentence states the core operation and scope; the second provides practical filter examples. Every part earns its place, and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional parameter and no output schema or annotations, the description covers the key facts: what is listed, where, and how to filter. Minor gaps include the exact return format and the precise pattern syntax, but these are not critical given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes the filter parameter at 100% coverage, giving a baseline of 3. The description adds value by showing concrete examples ('sections/', '.liquid', 'countdown') and using 'simple pattern,' which suggests broader matching than the schema's 'substring' wording. This goes beyond the structured schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'List files in the local theme directory.' This clearly identifies the operation and distinguishes it from siblings like list_themes (lists themes) and search_files (searches file contents). The optional filter is described as a secondary behavior, not the core purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use is implied: to list files, optionally filtered. However, the description does not explicitly state when to prefer list_files over related tools such as search_files, read_file, or theme_pull, nor does it mention when not to use this tool. Guidance is left to the agent's inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_filesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral transparency burden. It clearly indicates a read-only search operation, scopes the search to text files, states the regex flavor, and discloses the return format: matching files with line numbers and context. It does not mention performance implications or default limits, but the read-only nature is clearly implied by the verb 'Search' and 'Returns.'

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three tight sentences with no filler. It front-loads the core action and scope, describes the return value, and then gives a practical usage rule. Every sentence contributes unique value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with five parameters and no output schema, the description covers the essential context: what is searched, what the query format is, and what the response contains. The schema covers parameter details. Minor gaps include no mention of default max_results behavior or what happens when no matches are found, but these are not critical for selecting and invoking the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 80%, so the input schema already documents most parameters well. The description adds a bit of meaning by specifying 'JavaScript regex' and by noting the return includes line numbers and context, but it does not add meaningful detail for max_results or file_filter beyond what the schema provides. This matches the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb and resource: 'Search all text files in the theme for a string or JavaScript regex.' It also describes the return value ('matching files with line numbers and context'), which makes the purpose specific. It distinguishes itself from read_file by explicitly saying this is for locating code rather than reading whole files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear usage direction: 'Always use this to locate code instead of reading whole files.' This tells the agent when to prefer this tool over traversing files directly. However, it does not explicitly name the alternative tool or state when not to use it beyond the 'instead of reading whole files' guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• theme_pullA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It surfaces the non-obvious side effect of snapshotting the result in git and clarifies that pulling refreshes local files. It does not explicitly warn that local changes may be overwritten, but the 'before editing' guidance and download semantics make the main behavior transparent enough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences with no filler. The core action is front-loaded, followed by parameter routing and a clear usage directive. Every sentence earns its place and the total length is appropriate for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core operation, git side effect, theme selection, and when to run it, which is sufficient for a tool with two optional parameters and no output schema. It could mention whether pulling overwrites local files or what the command returns, but the existing context makes correct invocation mostly unambiguous.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters. The description adds marginal value by pointing out that theme_id comes from list_themes and confirming the omission behavior, but it does not elaborate on the 'only' glob parameter. This matches the baseline expectation when schema already covers parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action with verb and resource: it downloads a theme's files from Shopify into the local theme directory and snapshots the result in git. It clearly distinguishes from sibling theme_push by describing the opposite direction of data flow, so an agent can tell them apart without opening schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance on when to run the tool: 'Run this before starting an editing session so local files are fresh.' It also instructs how to choose the theme via theme_id from list_themes and explains the omit behavior for the live theme. It does not explicitly list when not to use it or compare to alternatives, but the timing and context are clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• theme_pushA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the safety burden. It discloses that the default is non-destructive, that live pushes are blocked unless allow_live=true, and that a preview URL is returned. It does not spell out whether pushing to an existing theme overwrites or deletes files, but the safety-critical behavior is well covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is tight and efficiently structured: one purpose sentence followed by three focused bullets. Every sentence carries a distinct rule or fact, with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no annotations and no output schema, the description is strong: it covers the safe default, how to target an existing theme, the live-push guardrail, and the preview URL. It leaves some operational details implicit, such as sync/overwrite behavior and the exact response when theme_id is supplied, so it is not quite a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline applies. The description reinforces theme_id and allow_live behavior at a usage level, but it does not add parameter syntax or format details that the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a concrete action ('Upload the local theme directory to Shopify') and immediately distinguishes the safe default from targeted/live pushes. It is clearly differentiated from theme_pull and publish_theme, and the scope is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives clear conditional guidance: default creates an unpublished copy, theme_id targets an existing theme, and allow_live=true requires explicit user confirmation. It does not explicitly compare against sibling tools like theme_pull or publish_theme, but the invocation conditions are well specified.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_themesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description carries the full burden of behavioral disclosure. It clearly states a read-only enumeration behavior ('List all themes') and exposes that it returns roles including live, unpublished, and development. The verb 'list' strongly implies no mutation, though an explicit 'read-only' statement would have been slightly more transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: the first front-loads the action and output details, the second adds a practical usage hint. There is no filler, and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description gives the essential return value (IDs, roles) and a usage directive, which is sufficient for a zero-parameter list tool. Since no output schema exists, the description fills the gap with the key fields, though it could have been slightly more explicit about the shape of the returned list.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and an empty schema, so the 100% schema coverage makes parameter documentation moot. The description correctly avoids inventing parameters and focuses on the output, which is appropriate for a zero-parameter tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' and names the resource ('all themes in the store'), and adds key output details (IDs and roles with allowed values live, unpublished, development). This clearly distinguishes it from sibling operations like publish_theme or theme_pull, which act on themes rather than enumerate them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The sentence 'Use this first to find theme IDs' explicitly identifies the intended context for this tool. However, it does not name any sibling alternatives or state when not to use it, so it stops short of the full when/when-not/alternatives guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• write_fileA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden, and it discloses the two most important behavioral traits: full overwrite is destructive, and the tool auto-commits to git as a side effect. It does not detail return values, permissions, or rollback options, but the critical behaviors an agent must weigh before calling are covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero filler. The core behavior is front-loaded first, and the routing guidance follows immediately. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description covers the essential decisions: what it does, when to use it, and its notable side effects. It does not mention the return value or commit details, but an agent has enough to invoke it correctly and safely.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%: filename, content, and message all have descriptive text, so the baseline is 3. The description adds the auto-commit context that connects message to the git log, but all parameter meaning is already present in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Create a new theme file or fully overwrite an existing one', and adds the auto-commit behavior. It also explicitly distinguishes itself from patch_file, so an agent can tell them apart without inspecting either schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit routing guidance: 'For small edits to existing files prefer patch_file — it is safer and cheaper.' This clearly tells an agent when to choose this tool versus the alternative, leaving no condition to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• patch_fileA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden. It discloses a critical side effect ('Auto-commits to git'), the exact-match constraint, and the requirement that old_str appear exactly once. It does not go into failure modes or hidden-content behavior, but the most important behavioral traits are present.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two dense sentences deliver the action, constraint, side effect, and sibling-tool guidance with no filler. The most important operational details are front-loaded, and every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 5-parameter edit tool with no output schema, the description plus schema covers the core invocation: what to patch, how to make the match unique, the auto-commit effect, and when to prefer it over write_file. The allow_hidden parameter is documented in the schema, and return-behavior details are not essential for calling this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 80%, so the schema handles most parameter meaning. The description adds practical value beyond the schema by explaining that enough surrounding lines should be included to make old_str unique, which helps the agent construct a valid old_str value. This is a meaningful addition rather than mere repetition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Edit an existing theme file'), the exact mechanism ('replacing an exact string'), and the resource. It also distinguishes itself from write_file by explicitly saying it is preferred for edits, so an agent can differentiate it from siblings without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives clear, actionable guidance: use this for editing existing files by exact replacement, and it explicitly names write_file as the alternative for other cases ('Preferred over write_file for edits'). It also tells the agent how to make old_str unique by including surrounding lines, which is essential for correct invocation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.