Server Quality Checklist

Tool Scores

• get_project_infoA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It clearly indicates a read operation returning configuration and procedures. It does not mention side effects, authentication, or performance, but for a simple query tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very concise: a header, a sentence, args/returns sections, and an example. All information is front-loaded and no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, the description adequately covers the tool's purpose and return shape. Could mention it is read-only, but not necessary. Completeness is high for a simple query.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only says 'project: Project name' and gives an example, adding minimal meaning. No constraints or format details beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it gets information about a specific project and shows configuration and available procedures. This distinguishes it from sibling 'list_available_projects' which lists projects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for a specific project but lacks explicit guidance on when to use vs. alternatives like 'list_available_projects' or 'get_next_step'. No exclusions or prerequisites mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_workflow_sectionA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It indicates a read operation (retrieves section content) but does not mention side effects, error handling, or authentication needs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with a clear lead sentence followed by parameter docs, return description, and example. Each sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, parameters, return value, and example. With an output schema present, return details are sufficient. It lacks handling of missing sections but is complete for a simple retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description fully defines parameters: 'section' as title to retrieve and 'project' as optional with default. It also provides an example, adding value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves a specific section from the workflow, using the verb 'get' and resource 'section'. It distinguishes from siblings like 'get_next_step' by specifying it retrieves a named section.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions it's useful for jumping to parts of the workflow or getting reference info, which implies usage context. However, it does not explicitly contrast with alternatives or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_next_stepA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses basic behavior (returns formatted next step instructions, defaults to step 1) but does not mention error handling, side effects (likely read-only), or authorization requirements. More context on edge cases would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a brief intro, bullet-style args, returns, and examples. It is front-loaded and each section adds value, though could be slightly more concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given low complexity (2 params) and presence of output schema, the description covers purpose, parameter details, and behavior. It provides sufficient context for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It explains both parameters: 'project' with default behavior and 'current_step' with default of 0. This adds useful meaning beyond the schema's type and title.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get the next step in the workflow' and elaborates on retrieving the next step for a project with an optional current_step parameter. It distinguishes from siblings like 'list_workflow_steps' by focusing on a single next step rather than listing all steps.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool (to get the next step) and includes examples. It explains behavior for both providing and omitting current_step. However, it does not explicitly mention when not to use this tool or suggest alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_available_projectsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It states that projects with configuration files are shown, implying a read-only operation, but does not mention side effects, authentication requirements, or error handling (e.g., empty list). The description is acceptable for a simple listing tool but could be more transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise: three sentences for purpose, returns, and an example. It is front-loaded with the action and resource, and every sentence adds necessary information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and a simple output (list of project names), the description is complete. It explains what it returns and gives a usage example, which suffices for AI invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters, schema coverage is 100%, so the description adds value by specifying the return value ('List of available project names') and providing an example. This exceeds the baseline of 3 for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List all configured projects' and 'Shows all projects that have configuration files available,' which specifies the verb (list) and resource (configured projects). Among sibling tools that handle workflow steps or specific project details, this tool is distinct and its purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description only states what the tool does—list projects—without any guidance on when to use it versus alternatives like get_project_info or list_workflow_steps. No when-not or scoping constraints are mentioned, though the context of listing all projects is implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_workflow_stepsA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. States it lists steps and returns a list, but does not disclose read-only nature or other behavioral traits. Since it's a simple list operation, the lack of extra detail is acceptable but not exemplary.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is front-loaded with the main purpose, includes structured Args/Returns/Examples sections, and every sentence adds value. No waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool has one optional parameter and an output schema (not shown but indicated). The description explains the parameter and return type adequately given the presence of output schema. Complete for this simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, meaning description adds value. The description explains 'project: Project name (uses default if not specified)' which clarifies the parameter's purpose and default behavior beyond the schema's type and default values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'List all steps in the workflow' and 'Provides an overview of all numbered steps in the project's workflow.' Distinct from siblings like get_next_step (singular step) and get_workflow_section (section focus).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides parameter details and examples, showing usage with and without project argument. Does not explicitly contrast with sibling tools but the purpose is clear enough that an agent can infer when to use it based on need.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.