Server Quality Checklist

Tool Scores

• fitter_run_fileA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description must fully disclose behavioral traits. It states the tool returns extracted data as JSON but does not mention whether modifications occur, required permissions, or error handling (e.g., file not found). The description is minimal and lacks transparency beyond the basic operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences, front-loading the primary purpose. Every sentence adds value: first defines the tool, second clarifies the difference from a sibling. No fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 2 parameters and no output schema, the description covers the basic purpose but omits important context like what happens if the file is invalid, permissions needed, or error scenarios. It is adequate for simple use but has gaps compared to a fully transparent description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds minimal additional meaning beyond the schema; it only reiterates that 'input' is optional and used with placeholders, which the schema already covers. No further value is added for the 'path' parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb (Run), the resource (Fitter config file), and distinguishes it from fitter_run by specifying 'reads the config from disk.' It also indicates the output format (JSON). This differentiates it from sibling tools like fitter_config_reference, fitter_run, and fitter_run_url.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly notes 'Same as fitter_run but reads the config from disk,' which helps users decide between this tool and fitter_run. However, it does not provide explicit when-not-to-use scenarios or mention other alternatives besides the direct sibling comparison.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fitter_run_urlA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It states the tool downloads config from a URL and returns JSON, but omits important details such as network error handling, timeout limits, authentication, size restrictions, or what happens with invalid configs. This lack of transparency could lead to unexpected failures.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with no unnecessary words. It front-loads the action and result, then adds the key distinction from 'fitter_run'. Every sentence provides useful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool involves remote fetching and parsing, but the description does not detail the return format beyond 'extracted data as JSON', nor does it explain error conditions or required permissions. With no output schema, more detail would be beneficial for an agent to anticipate the response structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already covers both parameters with descriptions (100% coverage). The description adds value by specifying the required top-level keys of the config ('item', 'limits', 'references'), which aids in understanding the expected structure beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs a Fitter config from an HTTP(S) URL and returns JSON data. It explicitly distinguishes itself from 'fitter_run' by noting the remote fetching behavior, making the purpose specific and differentiated from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates when to use this tool over 'fitter_run' (remote vs local config) and gives an example (raw GitHub link). However, it does not explicitly mention when not to use it or alternatives like 'fitter_run_file', though the context from the name and sibling list provides some guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fitter_config_referenceA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries burden. Describes output but does not explicitly state that tool is read-only or has no side effects, though context implies safe operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose, no fluff. Every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters or output schema, description sufficiently covers purpose and usage. Could mention response format but not critical for a reference tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters, schema coverage is 100% trivially. Baseline 4 applies, and description adds value by listing what the reference includes.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns a condensed reference of the Fitter config format with working examples, and distinguishes itself from sibling run tools by advising use before authoring a config for fitter_run.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly recommends using before authoring a config for fitter_run, providing clear context. However, it does not mention exclusions or alternatives, but siblings are run tools making differentiation obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fitter_runA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behaviors. It mentions fetching data via connectors and extracting data, implying network access. However, it omits potential side effects like rate limits, authentication needs, or error scenarios, which would strengthen transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief, with a clear structure: first sentence states the tool's purpose, second explains the underlying Fitter mechanism, third gives a usage tip. Every sentence contributes directly to understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with two parameters, no output schema, and no annotations, the description provides sufficient context: config format, supported selectors, and a reference to the config spec tool. It could be more complete by noting potential timeouts or result size limitations, but overall it covers the essential information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (both parameters described). The description adds context beyond the schema by explaining that config is JSON/YAML, highlighting top-level keys (item, limits, references), and stating that output is JSON. This adds meaningful value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action ('Run a Fitter scraping/parsing config passed inline') and the resource (inline config). It distinguishes from siblings by specifying 'inline', contrasting with file- and URL-based tools. The purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises calling fitter_config_reference first if unsure about the config format, providing clear guidance. However, it does not explicitly compare this tool to fitter_run_file or fitter_run_url, leaving the selection of the appropriate sibling somewhat implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fitter_validate_configA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully carries the burden, detailing what it checks (structural rules, condition compilation), that it is cheap and safe, and that it returns 'valid' or error. This comprehensively discloses behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place: purpose, checks, and usage advice. Front-loaded and succinct with no redundancies.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the single parameter and no output schema, the description fully covers purpose, behavior, usage context, and return type. It is complete for effective tool selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description reinforces the config parameter but adds no new parameter-level details beyond the schema description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Validate a Fitter config (JSON or YAML) without executing it,' clearly specifying the verb and resource. It distinguishes from sibling tools like fitter_run by advising use before calling fitter_run.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly advises using this tool while iterating on a config before calling fitter_run, providing clear when-to-use context. However, it does not explicitly state when not to use it or mention alternatives for different scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.