Server Quality Checklist

Tool Scores

• delete_projectB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description must fully disclose behavioral traits. It states the tool deletes a project and returns True, but omits key aspects like irreversibility, error handling for non-existent projects, or any side effects. This leaves ambiguity about the destructive nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with a front-loaded main verb phrase followed by a structured Args and Returns section. Every sentence is necessary and there is no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter tool, the description covers the main action, parameter, and return value. However, it lacks completeness on error conditions, success confirmation format, and any prerequisites. The output schema is described in text but not formally defined.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description should compensate. It restates the parameter name in the Args section but adds no new meaning (e.g., format, valid range, or source). The one parameter lacks any semantic enrichment beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Delete an OpenRefine project' with a clear verb and resource. It uniquely identifies the action from siblings like create_project, apply_operations, and export_csv, which are all distinct operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use (to delete a project) but provides no explicit guidance on when not to use or alternatives. No context about prerequisites or consequences is given. The sibling tools are different, but no comparative advice is offered.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• apply_operationsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries full burden. It states the return includes status and last modified time, indicating a mutating operation (not read-only). However, it does not disclose whether operations are reversible, idempotent, or what constitutes invalid input.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (two sentences for purpose, plus structured Args/Returns). No redundant or irrelevant information. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations, the description covers purpose, parameters (with some explanation), and return value. It could be more complete by noting operational nuances, but it meets basic needs for a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description partly compensates by explicitly explaining 'project_id' as the project ID and 'operations' as a JSON string. However, it does not elaborate on the expected structure or constraints of the operations JSON, leaving ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action: 'Apply operations to an OpenRefine project.' This distinguishes it from sibling tools like create_project, delete_project, and export_csv, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, nor does it mention prerequisites, limitations, or exceptions. It only describes the basic action without contextual usage advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• export_csvA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and the description only states the action and return type. It does not disclose whether the operation is read-only, destructive, or has any side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and well-structured: one sentence for purpose, then Args and Returns sections. Every part is necessary.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no annotations, the description covers purpose, parameter, and return value adequately. It lacks details like source of project_id or encoding, but is largely sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description explains project_id as 'ID of the project to export', adding basic meaning beyond the schema's type and title.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Export' and the resource 'CSV data from an OpenRefine project', distinguishing it from siblings like create_project or delete_project.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when CSV export is needed but provides no explicit when-to-use or when-not-to-use guidance nor alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_projectA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It only states basic creation without mentioning potential side effects, authentication, or error handling. It lacks details on what happens if the URL is invalid or timeouts.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with a clear docstring format, front-loading the purpose. The Args and Returns sections are structured and useful, though the return description could be more formal.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple tool with 2 parameters and an output schema, the description covers purpose and parameters adequately. However, it lacks information on error scenarios, prerequisites, or time expectations, making it slightly incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description includes an Args section that explains dataset_url and name, adding meaning beyond the bare-bones schema (which has 0% coverage). It provides clear purpose for each parameter despite lacking format constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it creates a new OpenRefine project from a dataset URL, using specific verb-resource combination. It distinguishes itself from sibling tools like delete_project, export_csv, and apply_operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (to create a project), but does not explicitly state when not to use it or mention alternatives. It is clear enough given the sibling tool names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.