Server Quality Checklist

Tool Scores

• lookup_schemaB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description repeats the read-only nature already declared in annotations (readOnlyHint, idempotentHint) and provides no additional behavioral details such as search behavior, result limits, or authentication requirements. It does not contradict annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (12 words) and front-loaded with the core action. Every word is meaningful, and there is no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (4 optional parameters, no output schema) and the presence of annotations, the description is largely adequate. However, it lacks details on the return format (e.g., single schema vs. list) and search matching behavior, which would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All four parameters have clear descriptions in the schema (100% coverage). The description merely lists them without adding usage examples, constraints, or interactions, so it does not add significant value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Search' and lists the filtering dimensions (domain, task, category, verification state), making it clear what the tool does. It also notes it is read-only, distinguishing it from mutation siblings like submit_schema_request.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not specify when to use this tool versus its siblings (e.g., list_public_schemas) and does not mention any prerequisites or limitations. It only states what it does, offering no guidance on selection context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_public_schemasA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, so the description's 'Read-only' adds no new behavioral information. The description does not disclose additional traits such as pagination behavior, authentication requirements, or rate limits, but it does not contradict the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that immediately conveys the tool's purpose. It has zero wasted words and earns its place by being directly actionable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description does not clarify what information is returned (e.g., schema names, metadata). It adequately covers the tool's input behavior but leaves the output structure implicit, which could cause an agent to misunderstand the result format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for all five parameters, so the baseline is 3. The description only mentions 'optional filters' generically, adding no specific semantic detail beyond what the schema provides (e.g., it does not explain valid values for 'sort' like 'trending' or 'recently_verified').

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the action ('List'), the resource ('public schemas'), and the scope ('in the Hermai registry with optional filters'). It effectively distinguishes itself from sibling tools like lookup_schema (which likely targets individual schemas) and submit_schema_request (a write operation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like lookup_schema or submit_schema_request. It mentions 'optional filters' but does not explain the selection criteria or trade-offs between this and sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_schema_request_statusA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, covering the safety profile. The description adds 'Read-only', which is redundant. It does not provide additional behavioral context such as expected results or edge cases, so the value added is minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. Every word earns its place, making it highly efficient for an AI agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only check with one parameter and no output schema, the description is mostly complete. However, it could be improved by mentioning what kind of status information is returned, though the absence of an output schema makes this less critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with a clear description for the single parameter 'request_id' ('Schema request id returned by submit_schema_request'). The tool description does not add further semantic value beyond what the schema already provides, so a baseline score is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it checks the status of a previously submitted Hermai schema request. It uses a specific verb ('check') and resource ('schema request'), and distinguishes from siblings like 'submit_schema_request' (submission) and 'lookup_schema' (schema lookup).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage after submitting a request ('previously submitted'), providing clear context. However, it does not explicitly mention when not to use or suggest alternatives, which keeps it from a top score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• submit_schema_requestA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=false, idempotentHint=true, and destructiveHint=false, which partially cover behavioral traits. The description adds a security constraint (avoiding secrets) but does not disclose the outcome of submission (e.g., return value or next steps), which is a gap given the lack of an output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two concise, front-loaded sentences. Every word contributes meaning: the first defines the action and target, the second adds critical safety instruction. No unnecessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the description covers the core action and a key constraint, it lacks context about the submission process (e.g., expected output, how to track the request via sibling tool check_schema_request_status). Given the tool has 10 parameters and no output schema, more detail on post-submission behavior would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the baseline is 3. The description adds value beyond the schema by emphasizing the required six fields and providing a security constraint that applies to parameters like requester_contact. This guidance helps the agent avoid mistakes.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: submitting a six-field intake for missing or brittle browser workflows. It uses a specific verb ('submit') and resource ('schema request'), and it distinguishes itself from sibling tools like lookup_schema and check_schema_request_status which handle different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit context for when to use the tool ('missing or brittle browser workflow') and gives a security guideline ('Never include cookies, API keys, or private session data'). However, it does not explicitly mention when not to use it or compare with alternatives like lookup_schema for existing schemas.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• classify_browser_workflowA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Description adds 'Read-only and local' beyond annotations (readOnlyHint, idempotentHint), providing valuable context about mutability and scope. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One concise sentence that front-loads the action and includes all essential information with no superfluous words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple input (one string) and no output schema, the description adequately explains the classification categories and behavioral traits, though it does not specify the return format explicitly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'prose' is fully described in the schema (100% coverage). The description adds a list of example sources but does not significantly enhance meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'classify' and the resource 'prose workflow' and lists the four possible output categories, distinguishing it from sibling tools which are schema-related.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives, but the specific task (classifying workflows) and sibling tools (schema lookups) make usage context clear by implication.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.