Server Quality Checklist

Tool Scores

• hermes_checkC
  2.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description honestly states the tool is not implemented, providing full transparency about its behavioral trait of being a stub. With no annotations, this is sufficient disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely brief, stating only the essential fact (not implemented) and the reason (API parity). Every word serves a purpose with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a stub tool, the description adequately communicates its non-functional status. However, it lacks information about expected behavior if implemented, which limits completeness for a tool that appears in a tool list.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description adds no meaning to the 'job_id' parameter beyond what the schema already shows. The description does not explain its purpose or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states it is 'Not implemented' and kept for API parity, which clarifies it has no actual functionality. However, the name 'hermes_check' suggests a checking operation, creating a mismatch between expectation and reality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No usage guidance provided. The description does not advise when to use or avoid this tool, nor does it mention alternatives among siblings like hermes_ask or hermes_cancel.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hermes_cancelA
  4.1/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description fully discloses that the tool is non-functional and kept only for API parity. This is complete transparency about its behavior (no-operation or failure).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that conveys all necessary information about the tool's status without any extraneous content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a stub tool, the description adequately covers its non-functional state. However, it omits any information about the return behavior or the significance of the job_id parameter, which slightly reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides no explanation of the required 'job_id' parameter. With 0% schema description coverage, the agent receives no guidance on what this parameter represents, which is critical even for a stub tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Not implemented. Kept for API parity with hermes-mcp.' This clearly communicates that the tool is a placeholder and does not perform any functional operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description directly advises against functional use by stating it is not implemented. However, it does not suggest alternative tools among siblings (hermes_ask, hermes_check, hermes_reset) for any potential cancellation needs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hermes_askA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses that tasks persist after the chat and that session_id enables memory across calls. However, it does not mention potential side effects, permissions, rate limits, or whether the operation is synchronous (the return statement suggests it waits for final answer).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a title, usage paragraph, and argument list. It is front-loaded with the core purpose and includes necessary details without excessive verbosity. The inclusion of a method to discover profiles is useful but slightly extraneous.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has three parameters, one required, and an output schema. The description covers the return value ('Hermes's final answer text') and distinguishes from siblings. While it does not discuss error handling or timeouts, the overall clarity is adequate for an AI agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description thoroughly explains all three parameters. 'prompt' is described as a natural-language instruction, 'session_id' as optional for step memory, and 'profile' as optional with a method to discover available profiles. This adds substantial meaning beyond the schema's bare structure.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Delegate a task to Hermes Agent on this user's machine.' It lists specific examples of tasks (scheduling, web search, email, editing) that distinguish it from siblings (hermes_cancel, hermes_check, hermes_reset), which handle cancellation, status checking, and resetting respectively.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to use this tool 'for things the calling LLM cannot do directly' and provides a list of use cases. While it does not explicitly state when not to use it or name alternatives, the context of sibling tools and the emphasis on actions that persist after the chat ends gives clear guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hermes_resetA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility. It fully discloses that the tool has no behavior because it is not implemented.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no wasted words, achieving maximum conciseness while conveying essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a non-implemented tool with no parameters and trivial output, the description fully covers what the agent needs to know: it does nothing and exists for API parity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters; the description adds no parameter info, but none is needed. Baseline 4 applies per instructions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool is 'Not implemented', clearly communicating its status as a placeholder. It lacks description of intended functionality, but is honest about its current state.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description effectively advises against using the tool by stating it is not implemented. It does not provide alternative tools among siblings, but the negative guidance is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.