Server Quality Checklist

Tool Scores

• list_proposalsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states 'Read-only' which is helpful, but it does not disclose pagination, filtering behavior, or that it returns a list of proposals. The description is too minimal for a tool with 9 parameters and no output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. It is concise but sacrifices completeness; however, for a list/search tool a brief description can suffice. The structure is front-loaded with purpose and read-only nature.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite 9 parameters and no output schema, the description provides no guidance on how to use the parameters effectively, no explanation of search vs list behavior, and no context on pagination or output format. The sibling tools are listed but not compared. The description is inadequate for the complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 67% (6 of 9 parameters have descriptions). The description adds no additional parameter meaning beyond the schema. The word 'search' implies filtering, but no parameter details are given. Baseline 3 is appropriate given moderate coverage and no added value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool lists or searches Portal.io proposals. The verb and resource are specific. However, it does not differentiate from the sibling 'get_proposal' which likely retrieves a single proposal, so the distinction between list and search is implied but not explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions 'Read-only', indicating it should not be used for mutations. However, it provides no guidance on when to use this tool versus alternatives like 'get_proposal' or how to choose between list and search behaviors. No explicit when-to-use or when-not-to-use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_catalogB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries burden. States 'Read-only' (behavioral) and prerequisite, but does not describe pagination, rate limits, or return format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words: purpose and a key prerequisite+behavior. Highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 6 optional parameters, no output schema, and no annotations, the description is too sparse. Missing guidance on search behavior, parameter combining, and result details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50%; description adds no extra parameter meaning beyond what schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb+resource: 'Search the Portal.io catalog.' Distinguishes from sibling tools like search_contacts by resource, but does not explicitly differentiate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Mentions prerequisite (Catalog API access must be enabled) and read-only nature, but lacks explicit when/when-not guidance or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_contactsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It only states 'Read-only', which is a behavioral trait, but lacks details on pagination behavior, rate limits, authentication needs, or how the search logic works. Minimal transparency beyond the read-only attribute.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short—one sentence plus a fragment—with no wasted words. It is concise, but could slightly expand on critical usage points without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description is insufficient. It does not explain return values, pagination behavior, or how filtering works across contact types. For a search tool with 6 parameters, more contextual information is needed for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50% (3 of 6 parameters have descriptions). The tool description adds no additional meaning to the parameters; it does not explain how 'search_text' interacts with 'contact_types' or the effect of 'sort_by' and 'sort_direction'. The description fails to compensate for the undocumented parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('search or list') and resource ('Portal.io contacts'), and adds a behavioral note ('Read-only'). It distinguishes from siblings like 'list_proposals' and 'search_catalog' by specifying contacts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide explicit when-to-use or when-not-to-use guidance, nor does it mention alternative tools. However, the context of sibling tools makes it clear that this is for contacts, so usage is reasonably implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_proposalA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  States 'Read-only', which is a key behavioral trait, but no annotations exist; description does not cover error handling, auth requirements, or rate limits. Adequate but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise: 8 words plus 'Read-only.' Front-loaded with the action and resource, no wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (1 param, no output schema), the description sufficiently covers what the tool does. Could mention what 'complete details' includes, but not essential.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the description merely echoes the schema's parameter description ('Portal proposal ID.') without adding meaning or constraints (e.g., type, range). Baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb 'Get' and resource 'complete details for one Portal.io proposal', distinguishing it from siblings like list_proposals (which lists multiple) or search_catalog (which searches different entities).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage when needing single proposal details, but provides no explicit when-not-to-use or alternatives (e.g., when to use list_proposals instead).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• test_connectionA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It mentions read-only and return of proposal metadata, but does not disclose error behavior, latency, or potential side effects. Adequate for a simple test call but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. The key information (purpose, read-only, return type) is front-loaded and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with no output schema, the description covers basic purpose and return type. However, it lacks details on error handling or what happens on failure, which would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, so schema description coverage is 100%. The description provides context that the tool tests credentials and returns metadata, adding value beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is a credential test for Portal.io API, using the verb 'Confirm' and specifying 'read-only' and 'returns only proposal metadata'. This distinguishes it from sibling tools like get_proposal or list_proposals which retrieve data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for testing credentials but does not explicitly state when to use it versus alternatives. No guidance on prerequisites or exclusions is provided, though the purpose is straightforward.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_usersA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description explicitly states 'Read-only' and 'List active users,' disclosing key behavioral traits. With no annotations provided, the description carries the full burden and adequately conveys safety and scope.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action and scope, followed by usage guidance. Every word earns its place, with no redundancies.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple, parameterless list tool, the description covers the key points: what it returns (active users), purpose (find IDs), and side effects (read-only). It could mention return fields, but the lack of output schema makes this acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the schema coverage is 100%. The description does not need to add parameter details. Baseline 4 applies due to no parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists active users in the Portal.io account, with a specific purpose of finding salesperson IDs. It distinguishes itself from sibling tools like list_proposals and search_contacts, making the tool's unique role obvious.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this to find salesperson IDs,' which provides clear context for when to use the tool. However, it does not mention when not to use it or compare directly to alternatives, so it lacks full exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.