Server Quality Checklist

Tool Scores

• analyze_dealB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must cover behavioral aspects. It describes outputs but omits important traits like read-only nature, error handling, rate limits, or permission requirements. The agent cannot infer safety or side effects from the description alone.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences that efficiently convey purpose, outputs, and usage scenarios. No superfluous text; front-loaded with key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 7 parameters, no output schema, and no annotations, the description provides a reasonable overview of outputs and usage but lacks details on output structure or assumptions behavior. Adequate but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with detailed field descriptions (e.g., default for monthly_expenses). The description adds no additional parameter meaning beyond the schema, meeting the baseline for good schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool analyzes a multifamily real estate deal and lists specific financial returns (cap rate, cash-on-cash, DSCR, etc.). It provides usage context but does not explicitly differentiate from sibling tools (market_data, reverse_calc, score_deal), though the use cases are implied.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives clear usage scenarios ('when someone asks about analyzing a property, evaluating a deal, or running the numbers on a multifamily investment') but does not indicate when NOT to use it or mention alternative tools. Lacks comparative guidance for siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• score_dealA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full responsibility for behavioral disclosure. It states the tool 'returns the DFP Score and key metrics' but does not clarify if the operation is read-only, whether it modifies data, or any prerequisites like authentication. The description lacks essential behavioral context beyond its input/output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: two sentences front-loading the core purpose and key differentiator. Every word adds value, with no redundancy or filler. It efficiently communicates the tool's primary use case.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters fully described in the schema, no output schema, and sibling tools providing context, the description covers the main purpose and distinction. However, it omits the structure of the returned 'key metrics' and does not address potential edge cases or error states, leaving slight gaps for a comprehensive understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, meaning each parameter is already described in the input schema. The description does not add further meaning to parameters (e.g., expected ranges, units clarification) beyond what the schema provides. Baseline 3 is appropriate as the schema does the heavy lifting; the description adds no additional parameter insight.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('quick-score'), the resource ('multifamily deal'), and the scale ('DFP 0-100'). It distinguishes from sibling tools by noting it's 'faster than full analysis' and intended for 'quick screening', setting it apart from more detailed analysis tools like analyze_deal.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly notes when to use the tool ('for quick screening') and implies an alternative ('full analysis'), but does not name the alternative tool (analyze_deal) or provide explicit criteria for when not to use it. The guidance is clear but not exhaustive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reverse_calcA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavior. It explains the calculation process but does not mention side effects, required permissions, or whether the tool is read-only. It safely implies no destructive actions but lacks deeper transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no extraneous text. The first sentence states the core function, the second provides usage context. Every sentence earns its place, and key information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has multiple parameters and no output schema. The description covers the core purpose and usage context well. Missing details about the output format (e.g., what the returned price looks like) are minor given the clarity, but could be improved for full completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents each parameter. The description adds value by explaining how parameters relate (e.g., 'Back-solves from your desired cap rate...') and clarifying the purpose of the 'assumptions' nested object, enhancing semantic understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs 'Calculate' and 'Back-solves' to describe exactly what the tool does: compute maximum offer price for multifamily deals. It clearly distinguishes from siblings like analyze_deal and market_data by focusing on offer price determination.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage guidance is provided with 'Use this when someone asks...', giving clear context for when to invoke the tool. However, it does not explicitly state when not to use it or mention alternative tools for similar but different tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• market_dataA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries the full burden. It discloses the returned data (flood zone, income relative to state median, job growth rate), which implies a read-only operation. No side effects or prerequisites are mentioned.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no fluff, front-loaded with the main purpose. Every sentence adds information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (2 parameters, no nested objects, no output schema), the description covers the essential behavior. It could mention the return format but is sufficient for an agent to understand the tool's capability.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, but the description adds value by providing an example address and explaining that zip is optional and may be extracted. This goes beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'look up' and the resource 'market intelligence' for a property address, listing specific return fields (flood zone, income, job growth). It is distinct from sibling tools like analyze_deal or score_deal, which likely involve calculations or analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this when someone asks about a market, neighborhood, or location.' It does not mention when not to use or alternatives, but the context is clear enough for an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.