Server Quality Checklist

Tool Scores

• correlationB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits but only states 'calculate correlation', implying a read-only computation. It does not mention performance implications, error conditions (e.g., missing tags), correlation method (e.g., Pearson), or any side effects, leaving significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long and front-loaded with the core purpose. The second sentence is slightly redundant but not wasteful. Overall, it is concise and contains no extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is reasonably complete for a tool with a provided output schema (which can describe return values). However, it lacks specificity on the correlation method used and whether correlations are computed pairwise between all tags. This ambiguity prevents a higher score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the parameters' meaning is already defined in the schema. The description adds minimal extra value, only clarifying that tags are correlated 'with other tags'. This meets the baseline of 3 without adding significant new semantics beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool calculates correlation between scalar tags in a TensorBoard event file, specifying the action (calculate) and resource (scalar tags). This verb+resource combination distinguishes it from siblings like query or tag_stats, which likely fetch raw data rather than compute relationships.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide explicit guidance on when to use this tool versus alternatives. It describes what the tool does but lacks any when-to-use, when-not-to-use, or alternative recommendations, leaving the agent to infer usage from purpose alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tagsA
  4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must cover behavioral traits. It only mentions functionality and filtering, but it omits read-only status, error conditions (e.g., invalid file), performance implications, or what happens when no tags exist. This leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences, front-loading the core purpose. Every sentence is necessary and no redundant information is present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has a simple interface (2 parameters, no enums, no nested objects) and an output schema exists, the description covers the main use case and filtering. However, it could mention that the tool is read-only or safe to call, but since the output schema is present, the lack of return format details is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the description adds clarification on how the filters parameter works, specifying that tags containing filter strings are returned. This goes beyond the schema description by explaining the matching behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists scalar tags from a TensorBoard event file with optional filtering. The verb 'get' and resource 'scalar tags' are specific, and sibling tools like tag_stats or tag_steps are distinct in function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the filters parameter to narrow results, providing clear usage guidance. However, it does not explicitly state when not to use the tool or mention alternatives like the 'query' tool for complex searches, so it's slightly incomplete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_eventsA
  4.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. Description only says 'find all... in directory and subdirectories' without disclosing error handling, performance, or return format. More behavioral context is needed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no redundancy. First sentence states purpose, second gives usage guidance. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With one parameter and an output schema, the description is mostly complete. However, missing behavioral transparency (e.g., directory validity) reduces contextual completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with parameter description. Description adds value by specifying 'and its subdirectories', which is not in the schema, clarifying recursive search.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'find' and the resource 'TensorBoard event files' in directories recursively. It distinguishes from sibling tools that focus on tags, queries, and stats.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'when you don't know the available events or user has not given you any event file name to analyze.' This provides clear context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tag_statsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It clearly states the tool computes min, max, mean, std, which implies a read-only operation. However, it does not explicitly confirm no side effects or mention data handling assumptions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, 14 words, front-loaded with the key action and output. No wasted content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool is straightforward, has an output schema (so return value explanation is not needed), and has 100% schema coverage, the description is complete enough for an AI agent to understand and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 100% description coverage, so the baseline is 3. The description does not add additional semantic detail beyond the schema's parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the action ('Get'), the resource ('statistical measures'), and the scope ('for each specified tag's values'). It distinguishes from siblings like 'correlation' or 'query' by focusing on statistical measures.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives such as 'query' or 'tag_steps'. Usage is implied from the description, but no when-not or alternative tool names are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tag_stepsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description notes that output can be large without tags, but does not disclose other behavioral traits like performance or side effects. With no annotations, it carries a moderate burden but still lacks detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the main purpose, no extraneous words. Every sentence contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, the description is sufficient for understanding purpose and usage. Could mention prerequisites (e.g., event file existence) but overall complete for a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema already describes both parameters with 100% coverage. The description adds value by explaining why the 'tags' parameter is important (avoid large output), going beyond schema details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get the steps') and resource ('each specified tag from a TensorBoard event file'). It distinguishes from sibling tools like list_tags (which lists tags) and tag_stats (which gives statistics).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides guidance on using the 'tags' parameter to avoid large output, implying when it's beneficial to specify tags. Does not explicitly discuss alternatives but clearly differentiates from sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• queryA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description fully carries the burden. It clearly states that the result can be very large and consume context limit if not filtered, and that it returns all data if no filters are applied.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with bullet points and clear warnings. Slightly long but every sentence adds value, and the format aids readability for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Comprehensive for a query tool with an output schema. Warns about large data, explains filtering, and references sibling tools. Adequately prepares the agent for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All parameters are documented in the schema (100% coverage). The description adds value by explaining the impact of not filtering (tags, steps) and how to use start_step/end_step for ranges, but the schema already covers basics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Explicitly states 'Query scalar data from a TensorBoard event file', clearly identifying the action and resource. It differentiates from siblings like list_tags and tag_steps which deal with metadata.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to use this tool (to get scalar data) and when to avoid (large outputs). Recommends using tags and step filters, and references sibling tools (find_events, list_tags, tag_steps) for prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.