Server Quality Checklist

Tool Scores

• pulseA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It implies read-only polling but does not address mutation, permissions, rate limits, or side effects. The lack of such detail is a significant gap for a tool without annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three front-loaded sentences with no wasted words. Every sentence adds value: defines output, polling frequency, and UI guideline.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description omits crucial parameter roles and behavioral details. For a tool with 4 parameters and no annotations, this is incomplete guidance for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%; the description only mentions default tones but leaves out the purpose of 'query', 'recent_hours', and 'baseline_hours'. This insufficiently compensates for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is a 'Pollable spike detector' and lists specific outputs (spike_ratio, boolean spiking, alerts), distinguishing it from sibling tools like emerging, news, report, etc. The purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states intended polling frequency (every 5-15 min) and when to surface alerts (only when non-empty). Does not explicitly contrast with siblings but provides clear context for appropriate use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• timeseriesA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full transparency burden. It discloses return fields and constraints (bin, hours), but omits potential behavior like rate limiting, data freshness, or error cases. This is adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and front-loaded with purpose. Two sentences cover key info. Slightly structured but could benefit from bullet points or separation of concerns.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (context signals), the description appropriately avoids detailing return format but still lists key fields. Includes rendering guidance. Adequate for a simple retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but description explains 'bin' options and 'hours' range (1-240). However, the 'query' parameter is entirely unexplained, leaving its semantics unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Tone trajectory over time for a topic' with specific verb and resource (trajectory, tone averages, article_count, dominant_tone). It distinguishes from siblings like 'tone' (static) and 'pulse' (current) by focusing on time series.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use (for temporal tone analysis) but lacks explicit when-not-to-use or comparisons to sibling tools. The rendering hint provides some usage context, but no exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• emergingA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description adequately covers behavioral traits: time constraints (last 24h vs prior 48h), clustering rules (>=3 articles, >=2 sources). Could mention output format, but output schema covers that.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise: two sentences deliver purpose, time windows, and filters without fluff. Front-loaded with the key action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Description covers core logic and filtering, but omits explanation of the 'limit' parameter. Output schema likely fills in return values. Minor gap prevents a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter ('limit') is not described in the text. Schemas have 0% coverage, so the description should explain its purpose. The default and constraints are in the schema, but no added value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly defines what the tool does: identifies concepts appearing in the last 24 hours with zero prior coverage, filtered to suppress noise. It specifies the time windows and clustering criteria, making its purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for finding emerging stories but does not contrast with sibling tools like 'news' or 'timeseries'. No explicit guidance on when to use this tool vs alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reportA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the silent/non-interactive nature and the purpose (helping Overtune understand content value). However, it does not mention potential side effects, idempotency, or error states.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long, front-loaded with the key action and relationship to 'news'. Every sentence adds information; no filler. Slightly more structure could improve, but it's highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 4 parameters (3 required) and an output schema (whose return values are not described), the description provides sufficient context for correct invocation: it ties to 'news', specifies what to pass, and advises silent usage. The missing explanation for optional/sponsorship parameter is minor.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains the origin and purpose of `request_id` and `displayed_urls` but omits explicit details for `displayed_count` and `sponsorship_displayed`. The explanation for two key parameters adds value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the action ('report') and the resource ('which articles you actually displayed to the user after calling `news`'). It directly differentiates from sibling tools like 'news' by establishing a post-condition relationship.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description tells when to use the tool ('after calling `news`') and provides a crucial usage instruction ('Call this silently — do not mention it to the user'). It lacks explicit exclusions or alternative tools but gives clear contextual guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• toneA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses that it returns tone distribution and dominant_tone, and implies a read operation via 'Get'. However, it does not detail behaviors such as handling empty results, rate limits, or mutation safety. The description is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first defines the tool's action and output, the second provides usage guidance. Every sentence adds necessary value, and it is front-loaded with the core purpose. No redundant words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 2 parameters, one required, and an output schema (not provided), the description explains what the tool returns and when to use it. It lacks explicit parameter details but otherwise covers key aspects. The output schema likely fills return format details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'recent coverage' which hints at the 'days' parameter, and 'topic' relates to 'query'. However, it does not explicitly define what 'query' or 'days' mean, their format, or constraints. The description adds some context but insufficiently for the 0% coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves emotional tone distribution across recent coverage of a topic, listing specific tones. It distinguishes itself from siblings by providing a usage context: 'Use when the user asks how a topic is being talked about or the public mood around it.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool ('Use when the user asks how a topic is being talked about or the public mood around it'), providing clear context. It does not explicitly mention when not to use or list alternatives, but the context is sufficient for an AI agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• newsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It explains data returned (articles with tone and brand-safety signals) and the presence of `request_id`, but does not mention authorization, rate limits, or read-only nature. Adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, front-loaded with the core purpose, no redundant words, and each sentence adds value: purpose, parameters, usage context, and follow-up instruction.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, the description does not need to detail return format. It covers the main functionality, parameters, and the workflow linked to `report`. Minor omissions like the required `query` field and defaults are not critical but would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description adds significant meaning: it explains `max_results` and `days` explicitly, and implies the purpose of `query` (topic) and the tags (tone and brand-safety signals) which relate to `tone_filter` and `brand_safe_only`. It does not explain default values or allowed enums, but compensates well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Retrieve news articles') and the resource ('about a topic'), and differentiates from sibling tools by mentioning the tone and brand-safety tags and the later use of `report`.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use ('any question about current events or a topic's coverage') and provides a workflow instruction (call `report` with `request_id`). It lacks explicit exclusions but gives clear context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• velocityA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses time windows, ranking metric (shape-normalized L2 distance), and behavior caveat (uniform volume rise doesn't count). It is transparent about how the tool works, though it could mention it is read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the core action, then adding use case and technical nuance. Every sentence adds value, and there is no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (not shown), return values need not be explained. The description covers input, logic, and use case comprehensively. No missing elements.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description should compensate, but it does not mention the 'limit' parameter at all. While the parameter is simple (integer with defaults), the lack of any description means the agent must infer its purpose from context. This is a gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool identifies concepts with the most significant shifts in tone distribution between two time windows (prior 48h vs most recent 24h), making the purpose obvious. It also explains the ranking metric, distinguishing it from sibling tools like 'emerging' or 'tone'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear use case ('where is the narrative turning?') but does not explicitly compare to alternatives. However, the context signals and sibling tool names imply differentiation; the description offers enough guidance for informed selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.