Server Quality Checklist

Tool Scores

• get_endpointA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It thoroughly discloses the output format (markdown) and enumerates all included content: base URLs, authentication, parameters, request body schema, response schema, expanded $refs, and examples. It does not mention potential errors or rate limits, but for a read-only retrieval tool, the behavioral transparency is strong.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, consisting of two sentences. The first sentence states what the tool does and lists the contents; the second directs how to obtain the id. Every sentence adds value without redundancy, and the structure front-loads the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter) and the absence of an output schema, the description fully covers the return value by listing exactly what the markdown includes. It also provides the prerequisite context about the id source. There are no significant gaps in understanding how to invoke the tool or what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes the id parameter with an example and format, and schema coverage is 100%. The description adds marginal value by specifying the id comes from search_docs/list_endpoints, but this is more usage guidance than parameter semantics. With high schema coverage, a baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with a specific verb and resource: 'Get the full contract of a single Pancake REST API endpoint as markdown.' This clearly distinguishes it from siblings like search_docs and list_endpoints, which return listings, and get_webhook/get_guide, which target different resource types. The tool's unique function is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs 'Pass an id from search_docs or list_endpoints,' connecting the tool to its discovery workflow. While it doesn't explicitly state when not to use it, the guidance implies it should be used after obtaining an endpoint id, and sibling tool names provide context for alternatives. This is clear but lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_guideA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool returns the full narrative text and mentions the type of content included. It doesn't discuss side effects (none expected for a read operation) or potential errors, but the read-only nature is clear from 'Get'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the primary purpose, followed by specific return content and id guidance. No redundant wording or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter read tool, the description covers purpose, return type, and id source. The lack of an output schema is mitigated by describing the narrative text. It's complete enough for an agent to call correctly, though it could optionally mention the response format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a detailed id description and examples. The description adds extra semantic guidance by instructing to pass an id from list_guides or search_docs, reinforcing provenance and usage beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'full text of a single Pancake API guide section'. It distinguishes from siblings like list_guides (listing) and get_endpoint/get_webhook (specific API/webhook details) by focusing on narrative documentation content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context on when to use: 'Pass an id from list_guides or search_docs' implies a preceding lookup step. It doesn't explicitly name alternatives or exclusions, but the description of return content (authentication, rate limits, integration flow) signals the appropriate use case compared to endpoint/webhook tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_guidesA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of behavioral disclosure. It does mention that the guides cover authentication, rate limits, usage flow, webhooks, etc., but does not explicitly state the return format, whether it is read-only, or if there are any rate limits. This is adequate but not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: the first sentence states the purpose, the second expands on content and provides a pointer to get_guide. Every sentence earns its place with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple listing tool with no parameters and no output schema, the description is quite complete. It tells what the list contains, lists example topics, and directs to get_guide for full text. It could mention the return format explicitly, but this is not essential given the low complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description adds useful context about the list content (concepts and topics) without needing to explain parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'List the Pancake API guide sections (narrative documentation).' This specifies the verb (list), the resource (guide sections), and clarifies these are narrative docs rather than endpoints, distinguishing it from sibling tools like list_endpoints.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that these guides cover concepts rather than a single endpoint, and explicitly points to get_guide for the full text. This provides clear context for when to use this tool versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_webhookA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It transparently discloses the return content: event description, payload structure with expanded $ref schemas, and a sample JSON body. This gives a clear picture of the tool's behavior and output, though it doesn't cover error handling or edge cases, which are less critical for a read-only get operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each earning its place: first states the primary action, second details the return value, third gives parameter sourcing guidance. It is front-loaded, concise, and free of redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter read operation, the description is complete. It covers what the tool does, what it returns, and how to find valid parameters. There is no output schema, and the description sufficiently explains the output structure without needing to enumerate all fields.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the 'event' parameter. The description adds further meaning by stating the event name should come from list_webhooks or search_docs, providing sourcing guidance beyond the schema's example list. This elevates the baseline from 3 to 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get the full payload schema and an example for a single Pancake webhook event.' It uses a specific verb ('get'), identifies the resource (Pancake webhook event), and distinguishes it from siblings like get_endpoint and list_webhooks by focusing on webhook payload schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on usage: it tells the user to pass an event name from list_webhooks or search_docs, implying when this tool is appropriate (when you need details for a single event). It doesn't explicitly state when not to use it, but the mention of 'single' and references to other tools provide effective guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_endpointsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosure. It discloses that results are grouped by tag, and each endpoint includes id, method, path, and summary. It also notes the optional filters. This is solid transparency, though it omits details like pagination or auth, which are not critical for a listing tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with purpose, then explains return format and links to the next step. Every sentence adds value, with no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (optional filters, no output schema), and the description fully covers its behavior, return fields, grouping, and relationship to get_endpoint. Nothing important is missing for a list tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents both parameters. The description reinforces that both filters are optional and what they filter on, but adds no new semantic detail beyond what the schema provides, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with a specific verb+resource: 'List all Pancake REST API endpoints, grouped by tag.' It clearly distinguishes from sibling tools by noting the returned id can be passed to get_endpoint for the full contract, making its role as a high-level list endpoint explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly names get_endpoint as the alternative for full contract details, effectively telling the agent when to use this tool vs another. It also explains optional filtering by tag or keyword, which guides usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_docsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden and does well: it discloses the search scope, ranked results, and result contents (type, id, snippet). It doesn't explicitly state read-only behavior, but the description makes it clear this is a search operation returning matches, which is sufficient for a safe tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each serving a distinct purpose: core action, workflow, and scope/result detail. No fluff, tightly written, and front-loaded with the key action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description fully equips the agent to use the tool: it explains the result format (type, id, snippet) even without an output schema, and ties into sibling tools for the next step. Coupled with fully described parameters, it is complete for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value beyond the schema by explaining what the 'area' parameter maps to (REST endpoints, webhooks, guide sections, schemas), giving the agent a richer understanding of enumeration values. This elevates it above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches the entire Pancake API documentation and returns ranked matches, with explicit scope (REST endpoints, webhooks, guides, schemas). It distinguishes itself from sibling get_* and list_* tools by positioning itself as the entry point for discovery.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit workflow: 'search first, then fetch full detail with get_endpoint, get_webhook, or get_guide using the returned id.' This gives clear contextual guidance but does not explicitly state exclusions or when not to use the tool (e.g., 'use list_endpoints to browse all endpoints instead'), so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_webhooksA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of explaining behavior. It states the output (each event's name and summary) and clarifies the nature of webhooks ('HTTP POST notifications... in real time'). While it doesn't mention potential limitations like pagination or auth, the tool's simple list behavior is sufficiently disclosed. No contradictions with existing annotations (none provided).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact: a lead sentence stating the core function, a sentence explaining the return value and a pointer to get_webhook, and a brief definition of webhooks. Every sentence earns its place, with no fluff. The structure is front-loaded with the primary action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (no params, no output schema), and the description covers the essential return information (name and summary) and the next-step guidance (get_webhook). It lacks details on potential edge cases like empty lists or ordering, but given the tool's simplicity, it is reasonably complete. The mention of webhook semantics adds useful context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema is empty. Per the rubric, a baseline of 4 is appropriate when no parameters need explanation. The description adds no parameter-specific details because there are none, but this does not reduce the score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a specific verb+resource: 'List all Pancake webhook events.' It clearly identifies the tool's function as a listing operation and distinguishes itself from siblings like get_webhook (which fetches a single webhook's details) by focusing on the enumeration of all events.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly directs users to 'Pass an event name to get_webhook for the full payload schema and an example,' providing a concrete alternative for deeper investigation. This clearly states when to use this tool (to see available events) versus another tool (to get detailed payload info).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.