Server Quality Checklist

Tool Scores

• search_projectsB
  3/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits, but it only restates the tool's name. It does not mention that the search is read-only, whether it returns matches or a single result, pagination, or matching behavior beyond what the schema's 'partial match' note provides (which is in the schema, not the description).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single four-word sentence, leaving no wasted words. It is appropriately concise for the minimal information it conveys, though its brevity contributes to lack of context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple search tool with no output schema, the description is insufficiently complete. It does not indicate whether the result is a list of matching projects, their structure, or how the search behaves. It also lacks any usage context relative to sibling tools, making it difficult for an agent to fully understand the tool's role.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a clear parameter description ('partial match'). The tool description adds no additional semantic meaning beyond repeating the parameter name, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb (search) and resource (projects) with a scope (by name), distinguishing it from sibling tools like get_projects which likely fetches projects without searching. Though terse, it unambiguously conveys the tool's core function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool rather than get_projects or other siblings. It does not mention alternatives, prerequisites, or contexts where this search is preferred.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_articleB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only says 'Create a new article,' which implies a write operation, but fails to disclose side effects, authentication needs, rate limits, or what the response contains. For a mutation tool, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no waste. It is concise and to the point.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 6 parameters, no output schema, and no annotations. The description fails to explain return values, associations (e.g., project membership), or any caveats. While it covers the basic purpose, it is incomplete for an agent to fully understand the tool's behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and all parameters have meaningful descriptions with defaults. The description itself adds no parameter-level detail, but since the schema fully documents them, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Create a new article' uses a specific verb and resource, clearly indicating the action and object. It distinguishes itself from sibling tools (get_projects, get_project_articles, search_projects, get_article) as the only creation tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, no mention of prerequisites, context, or exclusions. It merely states the action without any situational framing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_project_articlesA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only states what the tool does but reveals no additional behavior such as auth requirements, pagination, error handling, or result format. For a read operation, some transparency about these aspects would be expected.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with no filler or redundant words. Every word contributes to the meaning, making it highly efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description is largely sufficient: it names the input and the expected result ('all articles'). However, the lack of behavioral and usage guidance slightly reduces completeness, preventing a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully describes the single parameter 'projectId' with a clear description. The tool description adds minimal extra meaning beyond restating the parameter's role. Since schema coverage is 100%, the baseline is 3, and the description does not enhance parameter understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get all articles'), the resource ('articles'), and the scope ('from a specific project by project ID'). It distinguishes itself from sibling tools like 'get_projects' (which lists projects) and 'get_article' (which likely retrieves a single article).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you have a project ID and need all its articles, but it does not explicitly contrast with sibling tools or state when not to use this tool. The context is sufficient for a simple case, but no alternatives or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_projectsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It implies a read-only operation via 'Get' and defines the scope as 'all projects from your article queue', but it does not disclose potential caveats like whether archived projects are included, pagination, or limits. This is adequate for a simple read tool but lacks rich behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that gets straight to the point with no wasted words. It is perfectly concise for a simple list-all tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless tool with no output schema, the description provides sufficient context: it states what is returned (all projects) and the source scope. While it does not describe the return format, the plural 'projects' implies a list, making it reasonably complete. However, adding a note about what a 'project' is might improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema coverage is 100%, so the description does not need to explain parameter meanings. Per the rubric, a baseline of 4 is appropriate when there are no params.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to retrieve all projects within a specific scope ('your article queue'). The verb 'Get' and plural 'projects' make it distinct from sibling tools like 'search_projects' (which implies filtering) and 'get_project_articles' (which targets articles, not projects).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search_projects'. It does not explicitly mention exclusions or prerequisites, leaving the agent to infer that this is the go-to for listing all projects.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_articleA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds that the tool returns 'full content', which is useful beyond the name, but it does not mention error handling (e.g., 404 if article not found), permission requirements, or response structure. For a read-only operation this is somewhat acceptable, but deeper context would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded, and contains no wasted words. It efficiently conveys the core action, target, and result in a compact form.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity tool with one parameter and no output schema, the description provides sufficient context to select and invoke it correctly. It clarifies that the response includes the full article content, which is the key missing piece an agent would need. It could add error behavior details, but given the simplicity and sibling context, it is mostly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes the only parameter (articleId) with 100% coverage. The description merely repeats 'by ID' and adds no additional meaning about the parameter's format, constraints, or behavior. Since schema coverage is high, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get'), the resource ('a specific article'), and the scope ('by ID with full content'). It distinguishes from sibling tools like get_project_articles (which likely retrieves articles belonging to a project) and search_projects (which searches projects), making its purpose precise.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the proper use case: retrieving a full article when you already have its ID. It does not explicitly mention alternatives or exclusions, but the sibling tool names provide context. The phrase 'by ID' signals that this is for targeted retrieval rather than listing or searching, offering clear context without explicit when-not-to-use directions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.