Server Quality Checklist

Tool Scores

• render_site_planA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden. It mentions 'Returns one SVG' which indicates the output format. It also references 'server instructions for conventions and the coherence contract', implying there are rules the agent must follow, but it does not disclose what those conventions are. It doesn't mention side effects, permissions, or failure modes. The behavior is fairly predictable as a rendering operation, but the coherence contract reference is vague and could hide important constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, two sentences, and includes the core purpose. It mentions the output format and references the server instructions, which is a useful pointer but keeps the description short. It is somewhat front-loaded with the action and input, but the reference to 'coherence contract' is a bit vague, which is a minor deduction.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complex nested schema, the description is quite brief and relies heavily on the schema and external 'server instructions'. It doesn't explain how to handle coordinates, units, or style defaults. The output is described as one SVG, but no details on scale, viewbox, or error behavior. The description is sufficient for a basic understanding but leaves a lot to the agent to figure out from the schema and external instructions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with the spec parameter having a description and properties, but the description adds little beyond the schema. The tool description mentions the SiteSpec and its element types, but doesn't clarify the required fields (unit, scale, theme) or the optional outputPath behavior. The spec description in the schema is generic ('See this schema's properties for every field'), and the tool description doesn't help the agent construct valid SiteSpec. The outputPath parameter is briefly described in the schema, but the tool description doesn't reinforce defaults or paths.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool renders exterior site plans from a SiteSpec, enumerating the element types (buildings, parcels, roads, etc.) and notes it returns one SVG. This distinguishes it from sibling render_floor_plan, which presumably handles interior plans. The verb 'renders' plus the resource 'site plan' and input 'SiteSpec' is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states when to use it (to render exterior site plans) but does not explicitly discuss when not to use it or explicitly route to siblings. The noted sibling render_floor_plan is implied to be for floor plans, but no direct comparison or exclusion is given. Some guidance is provided via the element list, but not explicit alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• render_floor_planA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It discloses what is rendered, that it returns one SVG per floor, and points to server instructions for conventions and the coherence contract. It does not mention side effects of the optional outputPath, but the schema covers that parameter.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core action and output. Every clause earns its place, and the pointer to server instructions is compact rather than duplicating conventions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is complex with a deep nested spec and no output schema. The description states 'Returns one SVG per floor' but not the container format (e.g., array, object keyed by floor id) or how outputPath affects the result. It leans on external 'server instructions' for important contract details, leaving a moderate gap for an agent invoking it cold.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description adds no new parameter-level meaning beyond the phrase 'from a FloorPlanSpec', which is a slight reinforcement but not necessary.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Renders'), a resource ('interior floor plan'), and a source ('from a FloorPlanSpec'), plus the return form ('one SVG per floor'). The word 'interior' distinguishes it from the sibling render_site_plan without needing to open that tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: use when you need an interior floor plan from a FloorPlanSpec. However, it never explicitly says when not to use it or contrasts it with render_site_plan. No exclusions or alternatives are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pingA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description is the only behavioral disclosure. Responds with pong is a complete observable behavior for a zero-parameter tool, and the absence of side effects is implied by the trivial nature of the operation. An explicit safety statement would have made it fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short sentence with no wasted words. It is immediately readable and every word contributes to the meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of parameters, lack of side-effecting inputs, and limited response, the description fully communicates what the agent needs to invoke and interpret the tool correctly. There are no missing requirements.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, so the input schema already covers everything. The description does not need to add parameter information, and it does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and outcome: respond with pong. It makes the tool recognizable as a liveness check and distinguishable from the rendering siblings, though it does not explicitly name those siblings or state what it is not.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit usage guidance is provided. When to use this tool versus the render tools is inferable from the name and the convention of ping, but the description itself does not state when it should be chosen.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.