Server Quality Checklist

Tool Scores

• synthesizeA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses key behavioral traits: the session stays open after synthesis, contradictions should be noted, and gaps suggest next steps. This goes beyond a generic 'synthesizes' statement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the purpose. It is a bit lengthy with multiple paragraphs, but each sentence provides useful guidance with little fluff, so it remains appropriately concise for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description thoroughly covers the tool's purpose, use timing, and behavior, and the input schema is complete. The only minor gap is that it does not explicitly describe the tool's return value or output, which matters because there is no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, giving a baseline of 3. The description adds meaning by explaining what should go into `synthesis`, `contradictions`, `gaps`, and `recommendations` (e.g., agreement, contradiction, gaps), enriching the schema's short descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Synthesize findings across all applied lenses into a unified view.' This clearly distinguishes it from sibling tools like start_analysis and apply_lens, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says to call after applying 2+ lenses, and it advises what to do if gaps remain (apply more lenses and synthesize again). While it doesn't explicitly mention alternatives like get_strategy, the usage condition is clear and well contextualized.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• expand_selectionA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It thoroughly lists the return payload (full details, graph neighbors, counterbalances, uncovered categories), making clear it's a read/expansion operation. It doesn't explicitly state that no state is modified, but the language strongly implies it's informational.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-organized. The opening sentence states the core purpose, followed by a bulleted list of return types. Every sentence serves a purpose, and the structure aids scanning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with two straightforward parameters, no annotations, and no output schema, the description is complete. It explains what the tool returns (via bullets) and where it fits in the workflow, providing sufficient context for the agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the two parameters, and both are described adequately in the schema. The description adds only contextual reinforcement ('after start_analysis' for sessionId, 'models you've chosen' for modelIds), not new parameter-level semantics, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get full details for selected models plus their graph neighbors and counterbalances.' It uses a specific verb ('get') and resource ('selected models plus graph neighbors'), and distinguishes itself from siblings by positioning it between start_analysis and apply_lens.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit workflow guidance is given: 'Call after start_analysis with the model IDs you've chosen' and 'Use this to review your selection before committing to apply_lens calls.' This unambiguously tells when to use the tool relative to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• start_analysisA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden for behavioral disclosure. It states the tool 'Returns model clusters organized by category,' which indicates a read-like output. It also describes the iterative workflow and emphasizes composing multiple perspectives. However, it does not explicitly address whether the tool has side effects (e.g., creating persistent session state) or require any specific permissions, leaving a small transparency gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded: it opens with the core purpose, then provides a decision point about get_strategy, followed by a clear numbered workflow. Every sentence contributes value, and the length is appropriate for the tool's complexity. It avoids fluff and repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's role as an entry point to a multi-step analysis process, the description provides complete context: what it returns, how it fits into the larger workflow, and a best-practice recommendation (apply at least 2-3 lenses). No output schema exists, but the description gives a sufficient high-level view of the result. The workflow context makes it easy for an agent to use this tool appropriately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for all three parameters (scope, context, problem), each with a clear textual description. The tool description does not add additional semantics beyond the schema, but it also does not need to. Per the rubric, baseline 3 applies when schema coverage is high and description provides no extra parameter detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Begin a systems thinking analysis session' and specifies the immediate outcome: 'Returns model clusters organized by category.' It differentiates from siblings by positioning it as the first step in a defined workflow, distinct from expand_selection, apply_lens, synthesize, and get_strategy.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided: 'BEFORE calling this, consider whether a predefined strategy fits your use case. Call get_strategy...' It also outlines the complete expected workflow with numbered steps, making it clear when to use this tool and when to use alternatives. This is strong, non-ambiguous usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_strategyA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the two invocation modes and their outputs (list vs. concern map), notes that strategies are model-agnostic, and even mentions parallel execution. It does not explicitly state the read-only nature, but 'Get' and 'predefined' strongly imply no side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than average but well-structured: front-loaded purpose, a necessary list of strategies (since no enum exists), and a clearly labeled USAGE section. Every sentence delivers useful information, though a slightly tighter presentation would be possible.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given one optional parameter, no annotations, and no output schema, the description is remarkably complete. It explains both call modes, what the result contains (list or concern map), how to interpret concerns, and how to proceed with sibling tools, providing sufficient context for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides a description for strategyId, but the tool description adds the complete list of valid strategy IDs and clarifies the behavior when omitted (list all). This goes far beyond the schema baseline, giving the agent all acceptable values directly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Get a predefined analysis strategy.' It clearly defines what strategies are and distinguishes the tool from siblings by positioning it as the entry point for retrieving strategy definitions rather than executing analysis steps.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly explains when to call with vs. without strategyId, and provides a detailed usage workflow referencing sibling tools (expand_selection, apply_lens) as subsequent steps. It also clarifies that strategies apply to any system and explains the meaning of 'required' vs. 'conditional' concerns.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• apply_lensA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full transparency burden. It discloses critical behavioral traits: the 'analysis' parameter is narrative, 'findings' must be nested as a JSON object, and the response includes prior findings from earlier lenses (stateful behavior). It also notes ordering constraints ('Allowed after synthesize') and that it can be called multiple times, all beyond what the schema conveys.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-structured, with important usage notes and examples front-loaded. Every section adds value: purpose, critical parameter instructions, example, model reference, and usage lifecycle guidance. The exhaustive model list is arguably necessary for selection, but it does add bulk, preventing a perfect score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and complex nesting requirements, the description covers all essential aspects: how to structure calls, what parameters mean, which models are available, and how the tool fits into the session flow. The only gap is an explicit description of the response format beyond the note that it includes prior findings, but overall completeness is high.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 100%, the description adds significant meaning beyond the structured definitions. It clarifies the distinction between 'analysis' and 'findings', explains that findings keys must match the model's required field names, and provides a full example call. The extensive model list in the description also expands on the generic 'Model to apply' schema hint, making parameter selection much easier.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb+resource: 'Apply a systems thinking model to the current problem.' It clearly distinguishes from siblings by noting it applies one lens, while 'synthesize' is for integrating across lenses. The tool's role in the session lifecycle ('Can be called multiple times per session. Allowed after synthesize.') further clarifies its placement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided on when to use the tool ('Apply 2-4 lenses per session') and when to use an alternative ('When you've built enough perspective, call synthesize to integrate across lenses'). The description also tells users not to stop after one lens and warns against passing findings as top-level parameters, offering a concrete example call structure.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.