Server Quality Checklist

Tool Scores

• obs_event_logA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It explains the tool persists data (trace, duration, status, tokens, cost) for later analysis. However, it does not disclose side effects, authentication needs, rate limits, or whether it's safe to call multiple times. Adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, concise and front-loaded: first explains what it does, second explains the benefit. No wasted words. Length is appropriate for the complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool has 16 parameters but no output schema. Description does not explain what the tool returns (e.g., success indication, span ID, or error). Also does not clarify the 'response_format' parameter or how the nested 'attributes' object works. Given the parameter count and complexity, more detail is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (all 16 parameters have descriptions), so baseline is 3. The description adds context by listing operation types (model call, tool call, etc.) which align with the 'kind' enum. No additional detail beyond the schema, but no omission either.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool logs spans of AI agent execution (model calls, tool calls, retrieval, internal steps). It distinguishes itself from sibling tools like obs_trace_get, obs_events_search, etc., which are for querying/analyzing, not logging.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use (when recording events), but does not explicitly state when not to use or mention alternative sibling tools. Usage context is implicit, not directive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• obs_anomaly_scanA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries the burden. It discloses types of anomalies (error rate, latency, cost, retry loops) and severity classification, but does not mention if the tool is read-only, auth needs, rate limits, or side effects. Adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is a single sentence with a bullet list, front-loading the purpose and types of anomalies. No redundant information. Efficient for its length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 6 parameters and no output schema, the description covers the tool's purpose and anomaly types but lacks details on output structure or how to interpret results. The response_format parameter hints at formats but not explained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (all parameters have descriptions). The tool description adds context about anomaly detection logic but does not provide additional parameter-level semantics beyond the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool analyzes events to detect multiple types of anomalies (error rate, latency, cost, retry loops) and classifies them by severity. It distinguishes from siblings like obs_event_log (basic logging) and obs_metrics_summary (metrics) by specifying unique detection logic.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains what anomalies are detected but does not explicitly state when to use this tool instead of siblings (e.g., obs_events_search for specific events, obs_metrics_summary for summary stats). No when-not-to-use guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• obs_projects_listA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The disclosure is adequate for a read-only list operation, listing returned fields. No annotations exist, so the description should explicitly state side effects (none) or other behaviors, but it only describes output. It does not confirm idempotency or lack of side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no fluff, efficiently conveying the tool's purpose and output fields.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with one parameter and no output schema, the description provides sufficient detail on returned fields. It could mention the output structure (list of projects) but is otherwise complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single parameter (response_format), with clear enum choices. The tool description adds no extra meaning beyond the schema; baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Liste' (lists) and resource 'projets' (projects), clearly stating the data fields returned (volume, traces, errors, date, fill rate). It distinguishes from sibling tools like obs_trace_get or obs_event_log by focusing on aggregated project-level metrics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for listing project summaries but does not explicitly state when to use this tool versus alternatives. No exclusions, prerequisites, or when-not-to-use guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• obs_events_searchA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries the full burden. It mentions sorting by most recent first but does not disclose other behaviors like pagination, rate limits, or destructive potential. For a search tool, basic behavior is covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose and filters, no wasted words. Efficient and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a search tool with 11 parameters and no output schema, but could mention return format or pagination. The description provides enough context for basic use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters. The description adds no parameter-specific details beyond the schema, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches logged events with multiple filters and returns recent ones first, for investigating errors or latency spikes. It distinguishes from sibling tools by specifying a broad search capability.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides a clear use case (investigating errors/latency spikes) but does not explicitly mention when not to use it or list alternatives. However, the context and sibling names imply differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• obs_metrics_summaryA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It describes aggregation but does not explicitly state read-only nature, performance implications, or any constraints. The verb 'Agrege' suggests read-only, but it is not definitive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single efficiently constructed sentence that front-loads the primary action and enumerates the computed metrics. No redundant or extraneous content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 5 fully described parameters and no output schema, the description compensates by listing key output metrics. However, it lacks details on response_format behavior or constraints like window limits, which would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameters are well-documented. The description adds significant value by listing the output metrics (volume, error rate, latencies, etc.) that are not in the input schema, helping the agent understand what the tool computes.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it aggregates logged events over a time window and lists specific metrics (volume, traces, error rate, latencies, tokens, cost, top operations). It distinguishes itself from sibling tools like obs_trace_get and obs_events_search, which are for individual traces or event queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for aggregate metrics but does not explicitly state when to use this tool versus alternatives like obs_events_search or obs_anomaly_scan. No exclusion or prerequisite guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• obs_trace_getA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It correctly conveys the read-only nature and the type of data returned, but does not explicitly state that it is non-destructive or mention any permissions required.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no unnecessary words. First sentence defines the action and output, second sentence states the use case. Highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 3 parameters, no output schema, the description compensates well by describing the content of the response. All sibling tools are distinct, and the description is complete enough for an agent to understand when and how to use this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining what the output contains (spans, duration, statuses, tokens, cost), giving deeper meaning to the trace_id parameter, though it could elaborate on response_format and project filtering.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool reconstructs the full timeline of an agent trace, listing specific components (spans, duration, statuses, tokens, cost). It distinguishes itself from sibling tools like obs_events_search or obs_metrics_summary which focus on different aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly mentions it is useful for understanding why an execution failed or was slow, providing clear context. However, it does not provide explicit when-not-to-use or direct comparisons to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.