Server Quality Checklist

Tool Scores

• ask_projectB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description bears full responsibility for disclosing behavior. It mentions natural language filtering but omits critical details such as whether the tool is read-only, any required permissions, error handling, or response format. This lack of behavioral context makes it difficult for an agent to anticipate side effects or constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two concise, front-loaded sentences that efficiently convey the core purpose and a key prerequisite. Every word serves a purpose, and there is no superfluous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of an output schema, the description should compensate by explaining what the tool returns (e.g., answers to questions, respondent counts). It also fails to mention the 15-question limit from the schema or how to handle invalid filters. The tool is a query endpoint, yet the description provides no information about response structure or error conditions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with clear descriptions for each parameter. The tool description adds modest value by explaining that filter uses natural language and that list_projects provides project IDs, but these points are largely redundant with the schema's parameter descriptions. The baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Ask questions to a filtered subset of a project's survey panel', specifying a concrete verb-resource pair. It distinguishes from list_projects by mentioning a prerequisite, but does not explicitly differentiate from sibling tools ask_audience and ask_twins, which may have overlapping purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear usage hint: 'Use list_projects first to see available projects', which helps with context. However, it does not specify when to use this tool versus alternatives like ask_audience or ask_twins, nor does it mention any exclusions or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_twinsA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It mentions using 'Original Voices AI twins' but does not explain the result format, response time, or any side effects. For a tool that returns results (no output schema), more behavioral detail is needed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences, front-loaded with the primary purpose, and contains no redundant information. Every word contributes to understanding the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with two simple parameters and no output schema, the description explains what the tool does but omits details about what the response contains or how to interpret results. It is minimally adequate but lacks completeness for an agent to fully understand usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes both parameters (audience and questions) with clear examples. The description adds a mention of 'free-text audience description' but does not significantly enhance understanding beyond the schema. With 100% schema coverage, a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Ask questions to a specific audience using Original Voices AI twins') and specifies it is for 'ad-hoc audience queries with a free-text audience description,' distinguishing it from sibling tools like ask_audience and ask_project which likely target different use cases.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this for ad-hoc audience queries with a free-text audience description,' providing clear context for when to use this tool. However, it does not explicitly mention when not to use it or compare to siblings, which would earn a higher score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_audienceA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description carries full burden. Mentions 'ask questions to Digital Twins' implying a read operation but lacks details on side effects, error behavior, or response format. Adequate but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: one for purpose, one for prerequisite. Every sentence is informative; no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given simple tool (2 required params, no output schema), description covers main aspects: what it does and prerequisite. Could mention output nature, but still reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage with descriptions for both parameters. Description adds minor value by suggesting 'Open-ended questions work best' but largely restates schema. Baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the action (ask questions) and resource (previously saved audience by ID). Distinguishes from siblings by specifying to use list_projects first to discover audiences, implying this tool is for querying existing audiences.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to use list_projects first to discover projects and their audiences, providing a clear prerequisite. Does not explicitly exclude alternatives but implies context for using this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_projectsA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It clearly states it is a read operation ('List'), but does not disclose any potential side effects, limitations, or rate limits. For a 0-parameter list tool, the transparency is adequate but lacking extra behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no fluff. The core purpose is front-loaded, and the second sentence provides context and usage guidance efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and no output schema, the description fully covers what the tool does, what its results contain, and how it relates to sibling tools. No missing elements.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameters, so schema coverage is 100%. The baseline is 4, and the description adds context about what each project contains (survey questions, audiences), which is helpful beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' and resource 'all research projects', clearly stating the tool's output scope. It also distinguishes from sibling tools by mentioning it is for discovery before using ask_project or ask_audience.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to use it to discover projects before using ask_project or ask_audience, providing clear context on when to use this tool. It could be slightly improved by mentioning when not to use it, but the guidance is strong.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.