Server Quality Checklist

Tool Scores

• get_boilerplateB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only states the action and resource, omitting any details about return format, recursion, read-only assurance, or potential side effects. The agent is left uncertain about what 'all files' entails.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, minimal sentence that conveys the core purpose without extraneous words. It is crisp and front-loaded, earning full marks for efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is too terse to be fully complete. It does not explain what 'all files' means, how they are returned, or any constraints, leaving important gaps for a tool with a single parameter but no structured output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully documents the single parameter 'name' with a description ('Boilerplate name'), and schema coverage is 100%. The tool description adds no additional parameter meaning beyond the schema, so the baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's verb ('Get') and resource ('all files for a specific boilerplate'). It distinguishes itself from siblings like list_boilerplates and search_boilerplates, which likely operate on boilerplate metadata rather than files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. It does not indicate prerequisites, expected context, or when not to use it, leaving the selection decision to the agent without explicit support.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scaffold_projectB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action but does not disclose whether files are overwritten, whether the target directory must be empty, or any additional side effects. 'Scaffold' implies file creation, but key behavioral traits are omitted.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. Every word adds meaning, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a mutation-like tool with no annotations and no output schema, so the description needs to explain more about the effects. It does not mention overwrite behavior, whether the target directory must exist or be empty, or how boilerplate names map to actual templates. The sibling tools help but are not referenced.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% because both parameters have descriptions ('Target directory path' and 'Boilerplate name'). The description adds no extra meaning beyond what the schema provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('scaffold') and resource ('boilerplate') with a clear destination ('target directory'). It clearly distinguishes this tool from siblings like list_boilerplates, get_boilerplate, and search_boilerplates, which are read-oriented actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for generating a project from a boilerplate, but provides no explicit guidance on when to use this versus alternatives, nor any exclusions or prerequisites. The presence of sibling tools suggests context, but the description itself does not state them.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_project_updatesA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the purpose ('check') but does not mention whether the operation is read-only, has side effects, performs network calls, or what the output format is. This is a significant gap for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that directly states the tool's purpose without any wasted words. It is front-loaded and immediately understandable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single parameter and no output schema, the description is minimally acceptable. However, it does not specify what the check returns (e.g., a boolean, a list of updates, or detailed information), leaving the agent to guess. This gap is partially mitigated by the high schema coverage but not fully addressed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully describes the 'target' parameter (100% coverage), including its role as a project root directory containing boilerforge.lock.json. The description adds only the 'scaffolded project' context, which is minimal and not essential given the schema's clarity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the verb 'check' with a specific resource ('scaffolded project') and purpose ('boilerplate updates available'), making its function clear. It distinguishes itself from sibling tools like list_boilerplates, get_boilerplate, search_boilerplates, and scaffold_project by identifying a distinct action and target.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when to use: for scaffolded projects that may have boilerplate updates. It doesn't explicitly mention alternatives or exclusions, but the context is specific enough that the agent can infer it is not for listing, getting, searching, or scaffolding boilerplates.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_boilerplatesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states 'Search' which implies a read-only operation, but it does not explicitly confirm non-destructive behavior, describe return format, or mention any limitations like pagination or exact-match vs. partial-match behavior. The description adds minimal behavioral context beyond the core purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of eight words, front-loaded with the verb and resource. Every word is purposeful, and there is no filler or redundant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple search tool with one parameter and no output schema, the description covers the core purpose. It could explicitly mention that it returns matching boilerplates, but given the simplicity and the clear 'Search' verb, the description is nearly complete. The absence of output schema slightly lowers the score from 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides 100% coverage for the single query parameter, including a description ('match against name, description, tags, and stack'). The tool description repeats a subset ('name, stack, or tags') but omits 'description,' and does not add new meaning beyond the schema. Baseline for high schema coverage is 3, and the description does not enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Search' and identifies the resource 'boilerplates' along with clear filtering criteria ('by name, stack, or tags'). This clearly distinguishes it from sibling tools like list_boilerplates (which lists all), get_boilerplate (fetches one), scaffold_project (creates a project), and check_project_updates (checks updates).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when the agent needs to find boilerplates matching specific criteria, but it does not explicitly state when to use this tool over list_boilerplates or other alternatives. No exclusions or when-not-to-use guidance is provided, so the usage context is only implied by the word 'Search.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_boilerplatesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. The verb 'List' makes the read-only nature evident and there is no hint of side effects. However, the description doesn't mention potential details like authentication requirements, pagination, or whether results are sorted. For a simple listing tool, this is reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that directly states the tool's function. Every word earns its place; no filler or redundant phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema, no annotations), the description is nearly complete for the agent's needs. It clearly states that all boilerplates in the registry will be listed. It could optionally specify the return format or the fields included for each boilerplate, but for a listing operation this omission is minor.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description doesn't need to explain parameter semantics, and the schema correctly shows an empty properties object. No additional parameter information is required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and clearly identifies the resource ('all available boilerplates in the registry'). It distinguishes itself from sibling tools like search_boilerplates by emphasizing 'all' rather than filtered results, and from get_boilerplate by referring to the full collection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a complete inventory of boilerplates is needed, but it does not explicitly state when to use alternatives like search_boilerplates for filtered queries or get_boilerplate for a single item. No exclusions or alternative guidance is provided, so the usage context is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.