Server Quality Checklist

Tool Scores

• get_projectA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It mentions the return payload (summary, metadata, canonical URL), which is useful. However, it does not disclose potential error cases, authentication requirements, or any side effects, though the read-only nature is evident from the verb 'Get.' This is minimal but acceptable for a simple lookup tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is remarkably concise: one sentence for purpose and one sentence for parameters. Every word adds value, and the structure front-loads the main function before detailing arguments.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a straightforward get-by-identifier tool with no output schema, the description adequately covers the return value and required inputs. It does not explain usage context relative to sibling tools, but the core information needed to invoke the tool correctly is present. Given the simplicity, this is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description fully explains both parameters with concrete examples: 'owner (GitHub owner, e.g. 'NousResearch')' and 'repo (GitHub repo name, e.g. 'hermes-agent').' Since schema description coverage is 0%, these clarifications are essential and completely compensate for the missing schema-level descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get the full summary, metadata, and canonical URL for a specific Hermes ecosystem project.' The verb 'Get' and the resource 'specific Hermes ecosystem project' are unambiguous. It also distinguishes from sibling tools like search_projects and list_by_category by focusing on a single project identified by owner/repo.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied through the required parameters (owner and repo), suggesting the tool is for when you already know the exact project. However, there is no explicit mention of when not to use it or alternatives like search_projects for discovering projects, so it stops at implied usage rather than providing clear exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_by_categoryA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that the output is a 'ranked list' and enumerates valid slugs, but it does not explicitly state read-only behavior, error handling, or return format. For a simple read operation, the implied behavior is adequate but not richly detailed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that states the action and then lists valid inputs. It is front-loaded, contains no fluff, and every element earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema, read operation), the description is complete: it explains what the tool does, what the accepted inputs are, and what kind of output to expect ('ranked list of projects'). No essential information is missing for an agent to select and invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It adds meaning by defining the 'slug' parameter as a curated list identifier and enumerating all valid values. This provides sufficient semantic grounding despite the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Return') and identifies the resource ('ranked list of projects in a Hermes Atlas curated list'), clearly distinguishing itself from sibling tools like search_projects and get_project. The scope is well-defined by the enumerated slug values.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool by listing valid slug values, which implies usage when one of those curated categories is needed. It does not explicitly state alternatives or exclusions, but the context is clear enough for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_projectsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the scope (100+ community-built items), that queries can be natural language, and lists all category filter options. It does not describe the return format or ordering, but it provides substantial behavioral context that goes beyond the schema, such as what the category values are and the default limit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise but complete, front-loading the purpose then providing examples and parameter details. The category list is long but necessary, and every sentence adds value. It is well-structured and not verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the essential context: what the catalog contains, how queries work, and all filter options. It lacks mention of the return structure (what a result looks like) but given this is a search tool and there is no output schema, it is reasonably complete. It could improve by noting that results are matching projects, but it is sufficient for a qualified agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description fully compensates by explaining every parameter: query is described with natural language and examples, category is given an explicit enum list, and limit is defined with a range (1-50) and default 10. This adds meaning well beyond the bare schema fields.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: searching the Hermes Atlas catalog of 100+ community-built tools, skills, plugins, etc. It uses a specific verb ('Search') and identifies the resource ('Hermes Atlas catalog'), distinguishing it from siblings like get_project or list_by_category by emphasizing natural language querying across multiple project types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: it explains natural language queries, optional category filters, and a limit. It implies use for broad catalog searches but does not explicitly contrast with sibling tools like list_by_category or get_guide. It offers examples of queries and enumerates filter values, but no exclusionary guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_guideA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and discloses key behaviors: it returns full text, accepts only three specific slugs, and errors on invalid slugs. It does not mention non-mutability explicitly, but 'fetch' strongly implies a read-only operation, which is sufficient for this tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with the primary purpose stated first, followed by a compact parameter breakdown. Every word contributes value, and the structure is easy to parse for an agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter fetch tool, the description covers the purpose, accepted inputs, and error behavior completely. The absence of an output schema is mitigated by the phrase 'full text,' which sufficiently conveys the return value.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only defines slug as a string with no enum, so the description provides essential semantics by listing the exact allowed values and their corresponding URLs. It also clarifies the error condition for invalid slugs, greatly aiding correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it fetches the full text of a Hermes Handbook guide page, with a specific verb and resource. It enumerates the exact valid slug values, which distinguishes it from sibling tools like get_project and search_projects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage by listing all accepted slug values and noting that any other slug returns an error. It does not explicitly mention alternatives, but the allowed values and error behavior make the usage unambiguous for this simple fetch operation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_atlasA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses the tool's behavior: it delegates to the Hermes Atlas RAG endpoint, searches across specific sources (repos, summaries, guides, ECOSYSTEM.md), and returns a cited answer. This gives the agent a clear mental model of what occurs when invoked.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the core purpose, and each sentence adds value. The first states what it does, the second gives examples and the return format. No unnecessary filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no output schema, the description is complete. It explains the input, the process, the sources searched, and the output (cited answer), making it sufficient for an agent to decide when and how to use it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only defines 'question' as a string with no description. The description compensates by explaining the parameter's purpose ('free-form natural-language question') and supplying example questions that illustrate the expected format and content.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Ask a free-form natural-language question about the Hermes Agent ecosystem.' It further distinguishes itself by mentioning the RAG endpoint and cited answers, clearly separating it from sibling tools like search_projects or get_guide.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by stating it is for free-form natural-language questions and gives two concrete examples. However, it does not explicitly exclude sibling tools or state when to prefer alternatives, only implying usage through the question format.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.