Server Quality Checklist

Tool Scores

• meta_horizon_developer_read_operationC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It uses the word 'Read' to imply a safe, non-destructive operation, but it does not describe return formats, error handling, permissions, or side effects. This is minimal disclosure for a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no superfluous words. It efficiently communicates the verb and object, scoring high on conciseness and structural clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no annotations, no output schema, and three undocumented parameters, the one-sentence description is insufficient for an agent to confidently select and invoke it. It does not explain key terms like 'aggregate Query' or 'redacted snapshot', nor does it describe how parameters interact with the tool's behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, and the description does not explicitly explain the three required parameters (operation_name, app_id, snapshot_dir). It hints that 'aggregate Query' relates to the query name and 'latest redacted snapshot' relates to the snapshot source, but it fails to clarify formats or relationships, leaving the agent to infer from parameter names alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action ('Read') and the specific resource (a single allow-listed aggregate query from the latest redacted snapshot). It is concise and conveys the core function, though it does not explicitly distinguish this tool from sibling tools like status or collect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool instead of alternatives. It does not mention scenarios, prerequisites, or context where this tool is preferred, leaving the agent without decision support.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• meta_horizon_developer_statusC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states that it returns data, but does not mention whether it is read-only, what happens if the app_id or snapshot_dir is invalid, or any prerequisites. This is a significant gap for a tool that likely performs reads on a collection system.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, consisting of a single purpose sentence and two brief parameter annotations. No unnecessary words or repetition, though the parameter definitions could be integrated more elegantly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema, so the description should clarify what the returned 'collection health' looks like or how it is structured, but it does not. It also omits failure modes, preconditions, and how it relates to sibling operations like collect or read_operation, leaving the agent under-informed for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema lists parameters with no descriptions, so the description adds basic labels for both: 'app_id: target application ID' and 'snapshot_dir: directory holding that application's snapshots'. These definitions are helpful but shallow, providing no format, constraints, or relationships beyond what a schematic type already implies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Return') and resource ('latest collection health for one application'), making the purpose unambiguous. It distinguishes from sibling 'latest_snapshot' by focusing on 'health' rather than snapshot data, though the exact meaning of 'health' is not elaborated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus the sibling tools like 'meta_horizon_developer_collect' or 'meta_horizon_developer_latest_snapshot'. The description simply states what it does without any contextual advice or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• meta_horizon_developer_latest_snapshotB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden for behavioral disclosure. It adds useful context that the tool avoids returning a large payload, implying a performance/bandwidth advantage. However, it does not disclose whether the operation is read-only, has side effects, requires auth, or what happens if no snapshot exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the core purpose and key limitation (not the large payload). Every word earns its place, achieving maximum conciseness without empty filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (2 params, no output schema), the description should at least explain parameter semantics and when to use the tool. It fails on both counts, leaving critical gaps for correct invocation. The brief mention of metadata/health and payload size is insufficient for a tool with zero schema descriptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain the parameters app_id or snapshot_dir. Since both are required, the lack of any semantic guidance means an agent is left guessing their purpose. The description must compensate for the schema gap but does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Return' and the resource 'latest snapshot metadata and collection health,' and distinguishes itself from returning the large payload. This differentiates it from sibling tools like meta_horizon_developer_read_operation, which likely retrieves the payload itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when metadata or health is needed without the heavy payload, but it does not explicitly state when to use this tool versus alternatives like read_operation or collect. No exclusions or alternative tool names are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• meta_horizon_developer_collectA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full transparency burden. It discloses a prerequisite (authenticated browser via profile_dir) and a specific failure mode (ProfileBusyError when profile is locked), which adds meaningful behavioral context beyond the basic purpose. However, it does not mention other side effects or security requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences and front-loaded with the core purpose. The second sentence adds essential context (prerequisite and error behavior) without any redundancy or filler. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 4 parameters, no output schema, and no annotations. The description explains profile_dir and the error case, but leaves app_id, snapshot_dir, lookback_days, and the overall snapshot payload ambiguous. Given the lack of schema descriptions and annotations, the description does not provide enough context to fully understand the tool's behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for missing parameter meaning. It only explains profile_dir in the context of driving an authenticated browser. app_id, snapshot_dir, and lookback_days are not described in the schema or the description, leaving the agent to guess their roles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's primary action — 'Collect allow-listed aggregate GraphQL Queries into a new snapshot' — using a specific verb (collect) and resource (GraphQL queries into a snapshot). This distinguishes it from sibling tools like status and read_operation, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The purpose implies when to use the tool (creating a new snapshot), but there is no explicit comparison to alternatives or exclusions. The mention of needing profile_dir and failing with ProfileBusyError provides a prerequisite, but no direct 'use this when...' or 'use this instead of...' guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.