Server Quality Checklist

Tool Scores

• verify_partiesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full behavioral burden. It only states the action and does not disclose side effects, failure behavior, required permissions, or the nature of the verification process. This is a significant gap for a tool that could have side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It fits the tool's simplicity and is front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has three required parameters and no output schema, the description is too thin. It does not explain what verification entails, how the parameters are used, or what response to expect. This would leave an agent underinformed for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description mentions none of the three parameters (landlord_nif, tenant_nif, oauth_token). It adds no meaning beyond the raw schema, so the agent has no guidance on what these values represent or how they are used.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs ('Authenticate and verify') and clearly names the resource ('the landlord and tenant'). It distinguishes this tool from siblings like generate_lease or get_rent_status by focusing on identity verification, which is a unique action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives. The description implies it is for identity verification, but there are no explicit when/when-not instructions or references to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• propose_unpaid_insuranceA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description bears the full burden of behavioral disclosure. It mentions 'consented data' but does not explain the tool's side effects, whether it is read-only, how the OAuth token is used, or what happens with the data. The description is too vague about the operational behavior and limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that immediately states the action and the result. It is well-front-loaded with 'Assess tenant default risk' and contains no unnecessary words or filler, making it efficiently concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and no annotations, so the description must convey enough context for correct invocation. It explains the high-level function but does not specify the return format or any edge cases (e.g., invalid consent, failure modes). For a moderately complex tool with three required parameters, this is only minimally adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameter descriptions, and the description does not elaborate on the parameters. While names like 'monthly_rent' are somewhat self-explanatory, 'tenant_consent_token' and 'oauth_token' are not fully explained. The description mentions 'consented data' but does not clarify the role of each parameter, leaving the agent to infer too much.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Assess tenant default risk from consented data and return unpaid-rent insurance quotes.' It uses specific verbs ('assess', 'return') and identifies the resource (tenant default risk, insurance quotes). This distinguishes it from sibling tools like verify_parties, generate_lease, notify_tax_authority, and get_rent_status, which serve different functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: when you need to assess a tenant's default risk and obtain insurance quotes. It implies the need for consented data, which suggests a prerequisite, but does not explicitly state when not to use it or name alternatives. The context is clear, though no exclusions are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• notify_tax_authorityA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses a key behavioral requirement (landlord consent) but lacks other behavioral details such as side effects, irreversibility, or response format. Since there are no annotations, the description carries the full burden and only partially fulfills it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, consisting of two short sentences that directly convey the purpose and a key prerequisite. No redundant information is present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 required parameters, no output schema, and no annotations, the description is minimal. It does not explain the auth tokens, the meaning of 'time-bounded consent', or what happens on submission, leaving the agent with insufficient information to use the tool confidently.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description should explain each parameter. It only implicitly touches on consent_token via the consent requirement, leaving oauth_token, at_credentials_token, and signed_contract_hash unexplained. This does not compensate for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Submit a signed lease') and the target resource ('Portuguese Tax and Customs Authority (AT)'), distinguishing it from sibling tools like verify_parties or generate_lease.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context by noting the prerequisite of explicit, time-bounded landlord consent, implying when the tool should be used. However, it does not explicitly mention alternatives or when not to use it, preventing a higher score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_leaseA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosure. It explicitly states the tool never auto-signs and always returns a draft, which is a key behavioral trait. It does not mention side effects or storage, but the main safety-relevant behavior is transparently disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary action and a clear safety qualifier. Every word adds value, with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 required parameters, no output schema, and no annotations, the description provides a solid overview of the tool's function and its human-in-the-loop aspect. It lacks details on auth token usage or error behavior, but the core purpose and immediate outcome are sufficiently clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% parameter description coverage, so the description must compensate. It hints at 'verified party' and 'property metadata' corresponding to parties_verified_id and property_id, but it does not explain oauth_token, duration format, or rent_value semantics beyond the schema type. Partial compensation only.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool drafts a standardized rental contract using verified party and property metadata. The verb 'draft' and resource 'rental contract' are specific, and it is clearly distinct from sibling tools like verify_parties and get_rent_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates the tool consumes verified party and property metadata, implying it should be used after verification. It also clarifies that it never auto-signs, leaving room for human review, which is a workflow guideline. However, it does not explicitly name alternative tools or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_rent_statusA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It explicitly discloses 'Read-only' and 'scoped to the authenticated user's properties,' which are key behavioral and authorization traits. It does not describe return format or edge cases, but for a read operation this is reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action and resource, and contains no filler. Every clause adds value: 'Read rent-payment status for a property over a given period' states purpose, and 'Read-only, scoped to the authenticated user's properties' adds constraints.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool and the absence of an output schema, the description covers the essential context: what it does, scope, and read-only nature. It does not elaborate on the content of the status response, but the core selection and invocation needs are addressed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description adds meaning to 'property_id' and 'period' by mentioning 'property' and 'a given period.' 'oauth_token' is self-explanatory. This partially compensates for the lack of schema descriptions, but period format and property identification details remain unspecified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Read') and resource ('rent-payment status') and clearly scopes it to a property and period. It distinguishes itself from sibling tools like generate_lease and notify_tax_authority, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states 'Read-only, scoped to the authenticated user's properties,' which provides clear context on when to use it (for read-only queries) and its authorization scope. It does not explicitly name alternatives but the distinction from siblings is obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.