Server Quality Checklist

Tool Scores

• list_events_by_source_and_dateC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must cover behavioral traits. It only says 'Return... including scraper status,' implying a read operation, but lacks details on permissions, side effects, or error handling. Minimal disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, which is concise, but it omits necessary details. While not verbose, it is under-specified for the information an agent needs.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given two required parameters, no annotations, and an output schema that exists but is not described, the description is too minimal. It fails to address common concerns like date format, valid source values, or default behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description adds no meaning beyond the parameter names. It does not clarify the expected format for 'date' (e.g., YYYY-MM-DD) or constraints on 'source'. This leaves the agent without crucial usage details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns events for a specific source on a specific day, plus scraper status. While not explicitly contrasting with sibling tools like get_events_for_time_period, the per-day and per-source scope is implied.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs alternatives, no prerequisites, and no exclusions provided. The description only states what it does, not when it is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_events_for_time_periodC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It only states the return type but omits read-only nature, authentication needs, error behavior, or any constraints beyond the time frame.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no wasted words. It is appropriately concise for a simple tool, though it could benefit from a bit more detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity and presence of an output schema, the description is incomplete. It lacks parameter details and usage context, especially with sibling tools that overlap in functionality.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description should clarify parameters. It only implicitly mentions 'days' via 'within the next N days', but does not explain 'source' or 'as_of', leaving their meanings ambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Return'), resource ('semantic events'), and scope ('for a source within the next N days'). It distinguishes from sibling tools by specifying 'semantic events', though 'semantic' could be clarified further.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'list_events_by_source_and_date'. The description lacks context for usage decisions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scraper_is_validC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must fully convey behavior. It only says 'still returning semantic data', but doesn't explain what semantic data means, what response to expect (likely boolean), or any implications like error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, directly conveys purpose. Could be improved by structuring with additional context, but remains concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple boolean check tool, the description is somewhat complete, but lacks details on output schema (though exists externally) and context on when scraper becomes invalid. Given sibling complexity, more completeness would help agents.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% and description offers no additional meaning for the 'source' parameter beyond its name. The parameter is left completely undocumented in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Check' and the resource 'source-specific scraper', and distinguishes this tool from siblings like list_sources or get_events_for_time_period by focusing on validity of scraper output.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool instead of siblings like list_sources or get_events_for_time_period. No prerequisites or when-not-to-use information provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_sourcesB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full behavioral disclosure burden. It only states the output content (sources with status/health) but does not mention whether the operation is read-only, requires authentication, or if there are any rate limits, pagination, or caching behaviors. For a tool with no parameters and an output schema, more context on behavior would be helpful.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is entirely front-loaded with the key information: 'list sources with functional status and scraper health.' No filler or redundant text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (zero parameters, no annotations, but an output schema exists), the description adequately covers the high-level purpose. It could be improved by noting that it returns all sources without filtering or ordering, but it is largely complete for a straightforward list operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the schema coverage is 100% by default. The description does not add information about parameters, but this is acceptable as there are none. A baseline of 4 is appropriate, though a brief statement that it lists all sources without filtering would improve clarity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists sources and includes functional status and scraper health. This is a specific verb+resource combination that distinguishes it from sibling tools focused on events or individual scraper validation. However, it does not explicitly define what 'functional status' or 'scraper health' entail.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. The sibling tool names suggest different purposes (events, validation), but the description does not state when to prefer this tool or what scenarios are inappropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.