Server Quality Checklist

Tool Scores

• search_planning_applicationsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, placing the full burden on the description. It mentions the returned fields but fails to disclose pagination behavior, rate limits, authentication needs, or error handling. This is insufficient for a tool with 9 parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise: two sentences that front-load the purpose and include examples. No wasted words or redundant information. Ideal length for a search tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (9 parameters, no output schema, no annotations), the description is adequate but not fully complete. It lists return fields and gives examples but omits details like pagination, optionality of all parameters, and free vs paid tier limit (which is in schema but not description). Could be more comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the parameters are already well-documented in the schema. The description groups them into categories (keyword, council, etc.) but adds no additional semantics beyond the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it searches UK planning applications by various criteria, listing the fields returned. However, it does not explicitly distinguish itself from sibling tools like nearby_planning_applications or get_planning_application, which have distinct purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides concrete example queries ('recent planning applications in Camden', 'refused extensions in OX1') that illustrate appropriate use cases. Does not include when-not-to-use or alternatives, but the examples give clear context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• nearby_planning_applicationsA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It only says 'find applications within a radius' but does not mention if results are sorted, limited to active applications, paginated, or any rate limits. There is no contradiction but significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two sentences, front-loading the core action. There is no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters and no output schema or annotations, the description is incomplete. It does not mention output format, pagination, or default values, leaving an agent with insufficient information for robust invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds no additional parameter semantics beyond what is in the schema (e.g., default values for limit or radius_km are not clarified). It minimally meets expectations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds planning applications within a radius of a given point, using 'what's been applied for near this location' as a usage example. This distinguishes it from siblings like 'get_planning_application' (single app) and 'search_planning_applications' (likely text-based).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a usage context ('what's been applied for near this location') and implicitly recommends this tool for proximity queries. However, it does not explicitly state when not to use it or compare with siblings like 'search_planning_applications' for text searches.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_planning_applicationA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It only states the basic action without disclosing any behavioral traits such as error handling (e.g., 404 if id not found), authentication requirements, rate limits, or idempotency. The description is too minimal for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 10 words, containing exactly the needed information with no redundancy. It is front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple fetch-by-id tool with a single parameter, the description is complete. It specifies what the tool does and the required identifier. The absence of an output schema is acceptable as the return value (a planning application) is implied by the action. Sibling tools cover different use cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (one parameter with description in schema). The description adds no additional meaning beyond the schema; it merely restates the parameter's purpose. Baseline of 3 is appropriate since the schema already provides sufficient definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('fetch'), the resource ('a single planning application'), and the identifier ('by its PlanWire id'). This distinguishes it from sibling tools that list, search, or find nearby applications.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a specific PlanWire id is known, but it provides no explicit guidance on when to use this tool versus alternatives like search_planning_applications or list_councils. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_councilsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the basic function (listing councils with IDs) but omits details like data freshness, sorting, pagination, or whether it's read-only. For a simple list tool, this is adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is extremely concise and front-loaded, containing no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description covers the essential purpose and use case. It could be slightly improved by specifying the return format (e.g., list of objects), but overall it is complete enough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, and the schema coverage is 100% (empty object). The description adds no parameter details, but with 0 parameters the baseline is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('list'), the subject ('UK councils (local planning authorities)'), the scope ('PlanWire covers'), and the purpose ('with their IDs for use in searches'). It effectively distinguishes from sibling tools like 'get_planning_application' which focus on applications.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'for use in searches' implies the IDs are meant for tools like 'search_planning_applications', providing usage context. However, it does not explicitly state when not to use this tool or mention alternative approaches.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.