Server Quality Checklist

Tool Scores

• search_foundersC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does not state whether this is a read-only search, what scope it searches (episodes? full archive?), how results are ordered, or what side effects exist. An output schema covers return shape, but operation semantics remain undocumented.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero filler, and the primary purpose is front-loaded before the workflow note. Every sentence earns its place; there is no redundancy with the schema or siblings.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists, which relieves the description of explaining return values, and sibling names provide context. However, with 0% parameter documentation, no annotations, and a domain-specific parameter like episode_number left unexplained, the description is not complete enough for an agent to call this tool correctly in all intended cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, yet it says nothing about any of the three parameters. Required 'query' is not qualified (format, reframing strategy), and 'episode_number' — which clearly scopes the search to a single episode — is never mentioned, leaving agents to guess its semantics from the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description gives a clear verb ('Find') and resource ('cited evidence') tied to a strategic-question context, so an agent understands the core purpose. However, it does not differentiate from the sibling 'get_founders_passage', which plausibly also returns passages/evidence, leaving the boundary between the two tools implicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Run reframed and conflicting-view searches before synthesizing' gives workflow-ordering guidance (use it before synthesis) but says nothing about when to prefer this tool over its siblings, nor when not to use it. No alternatives are named and no conditions select one tool over another.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_founders_episodeA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the burden of behavioral disclosure. It does add the key matching rule (exact number or partial title), which is a meaningful behavioral trait. However, it does not mention pagination, result type, or any side effects. The existence of an output schema reduces some burden, but safety or read-only behavior is not disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-formed sentence that states the action and the method without any wasted words. It is front-loaded with the action and efficiently communicates the core behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple lookup tool with an output schema, the description covers the main input semantics and the matching rule. However, it lacks usage guidance and does not mention alternatives, which slightly reduces completeness. Given the tool's simplicity and the presence of an output schema, this is adequate but not exhaustive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It clarifies that `query` accepts an exact episode number or a partial title, giving concrete meaning to the otherwise bare string parameter. The `limit` parameter is not explicitly explained, but its purpose (limiting results) is self-evident. Overall, the description adds significant meaning to the main parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Find' with a clear resource ('Founders episodes') and states the method ('by exact episode number or partial title'). This distinguishes it from a generic search, but it does not explicitly name or contrast with sibling tools like `search_founders`, so it stops short of full differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when the user has an exact episode number or a partial title, but it provides no explicit guidance on when *not* to use this tool or which alternative (e.g., `search_founders`) to choose. The usage context is inferred, not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_founders_passageA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It does state the core behavior ('expand a cited passage with up to three neighboring passages'), giving the upper bound. However, it doesn't disclose whether the operation is read-only, what happens when no neighbors exist, or how the 'context' parameter modifies the behavior. The description is not misleading, but it lacks depth that would be valuable given zero annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the primary function ('expand a cited passage') and immediately states the limit ('up to three neighboring passages'). There is zero waste, and every word contributes to understanding the tool's core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description is insufficient for a simple tool with 0% schema description coverage. It fails to explain the 'context' parameter or clarify what 'neighboring passages' means (e.g., previous/next, one each side). An agent might not know how to control the number of passages returned or whether the default behavior meets the stated 'up to three' limit. This leaves critical usage details ambiguous.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain the parameters. It never mentions 'context' at all, leaving its meaning (likely number of neighbors) opaque. 'passage_id' is only implied by 'cited passage' but not explicitly tied to the parameter. The description adds almost no value beyond the bare schema names, failing to compensate for the missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('expand') and a clear resource ('cited passage'), and adds precise scope ('up to three neighboring passages from the same episode'). This clearly distinguishes it from siblings like search_founders (searching) and find_founders_episode (finding an episode).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a clear use case: when you have a cited passage and want to see its neighbors for context. It doesn't explicitly name alternatives or state when not to use it, but the purpose is distinct enough that an agent would know when to invoke it. This qualifies as clear context without explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• founders_archive_statusA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It indicates a read-only status check ('check whether'), but does not describe what the response contains (e.g., boolean, detail object, error conditions) or any side effects (likely none). Since an output schema exists, some return details are covered structurally, but the description adds little beyond the action itself.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, tightly worded sentence that front-loads the core action and object. No filler, no repetition, and every word contributes to the meaning. Ideal for a status-check tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the zero-parameter surface, the presence of an output schema, and the clear distinction from sibling tools, the description fully specifies what the tool does. There is no missing information an agent needs to decide to call it or interpret its result beyond what the schema already provides.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes zero parameters, so there is nothing to document. Per the baseline guidance, a 4 is appropriate when there are no parameters to describe; the description correctly avoids fabricating parameter details and focuses on the tool's purpose.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('check whether') and names the exact resources checked ('private transcript corpus and hybrid research index'). This clearly distinguishes it from the sibling search/fetch tools, so an agent can tell at a glance this is a readiness probe rather than a data-access operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a preflight role (checking readiness before using the search tools) but does not explicitly say 'use before searching' or mention when not to use it. It gives the purpose but no direct guidance on when to call it versus alternatives, leaving the agent to infer the gate-keeping scenario.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.