Server Quality Checklist

Tool Scores

• get_market_summaryB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the data returned but doesn't mention critical traits like whether this is a read-only operation (implied by 'Get'), potential rate limits, authentication requirements, data freshness, or error handling (e.g., for invalid state codes). The description adds minimal context beyond the basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Get a market summary for a state') and immediately lists the key data points. There is no wasted verbiage, repetition, or unnecessary elaboration—every word contributes directly to understanding the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (aggregating multiple market metrics) and lack of annotations or output schema, the description is adequate but has clear gaps. It specifies what data is returned but not the format (e.g., JSON structure), handling of missing data, or time range for the summary. For a tool with no structured output documentation, more detail on return values would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting the single required parameter 'state' as a '2-letter state code'. The description adds no parameter-specific details beyond what the schema provides, such as examples of valid state codes or formatting requirements. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get') and resource ('market summary for a state'), listing the exact data points returned (total active listings, score distribution, average cap rate, deal count by property type, and top submarkets). It distinguishes itself from siblings like 'get_deal' (individual deals) or 'search_deals' (filtered searches) by focusing on aggregated market-level statistics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., state must be valid), exclusions, or comparisons to siblings like 'get_market_benchmarks' (which might offer different metrics) or 'search_comps' (for property-level comparisons). Usage is implied only by the tool's name and description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_1031_candidatesA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While it mentions filtering criteria and hints at optimization ('for best results'), it lacks critical details such as whether this is a read-only operation, potential rate limits, authentication requirements, or what the output format looks like. For a search tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences, with the first stating the purpose and key filters, and the second providing usage guidance. Every phrase adds value without redundancy, making it easy to parse and front-loaded with essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a 1031 exchange tool with 6 parameters and no output schema, the description is adequate but incomplete. It covers the purpose and basic usage but lacks details on behavioral traits (due to no annotations) and output expectations. The high schema coverage helps, but more context on results format or limitations would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 6 parameters thoroughly. The description adds minimal value beyond the schema by implying that 'max_price' and 'min_price' relate to exchange value and timeline, but it doesn't provide additional syntax, format details, or clarify relationships between parameters like 'min_cap_rate' and the 1031 context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Find deals suitable for a 1031 exchange') and resource ('income-producing properties'), with explicit criteria like clean structures, price bands, and lease profiles. It distinguishes from siblings by focusing on 1031 exchange suitability rather than general deal retrieval or analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('for a 1031 exchange') and includes best practice guidance ('Pass the exchanger's target price range and timeline for best results'). However, it does not explicitly state when not to use it or name specific alternatives among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_market_benchmarksA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the data source ('derived from closed comps') but lacks details on behavioral traits like rate limits, error handling, or response format. This leaves gaps for a tool that likely involves data retrieval.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the purpose and followed by usage context. Every sentence adds value without redundancy, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is adequate for a simple lookup tool but lacks completeness. It explains the purpose and usage but omits details on output format, data freshness, or potential limitations, which could be important for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents the parameters (state and property_type). The description adds context by linking parameters to benchmarks, but does not provide additional syntax or format details beyond what the schema offers, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get'), the resource ('cap rate and price/SF benchmarks'), and the source ('derived from closed comps'), distinguishing it from siblings like get_market_summary or search_comps by focusing on benchmarks rather than summaries or raw comps.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides a clear context for use ('to determine if a deal is priced above or below market'), which helps guide when to apply this tool. However, it does not explicitly mention when not to use it or name alternatives among siblings, such as get_market_summary for broader data.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_dealA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It describes the return content (AI scoring, flags, etc.), which adds value beyond the input schema. However, it doesn't disclose behavioral traits like error handling, permissions required, or rate limits, leaving gaps for a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, dense sentence with zero waste—every word contributes to clarifying purpose and output details. It's front-loaded with the core action and efficiently lists the specific data returned.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description partially compensates by detailing return content. However, for a tool that retrieves complex data (AI scoring, flags, etc.), it lacks information on response structure, error cases, or prerequisites, leaving the agent with incomplete context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the 'deal_id' parameter. The description adds context by noting the ID comes from 'search_deals results', which is helpful but doesn't provide additional syntax or format details beyond the schema. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('full details for a specific deal by ID'), distinguishing it from siblings like 'search_deals' (which lists multiple deals) or 'score_deal' (which focuses on scoring). It specifies the exact scope of details returned, including AI scoring breakdown, green/red flags, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need comprehensive details for a known deal ID, contrasting with 'search_deals' for finding deals or 'score_deal' for scoring. However, it lacks explicit when-not-to-use guidance or named alternatives, such as not using it for bulk retrieval or when only basic info is needed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_compsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the data source size ('100,000+ closed commercial transactions') and a tier requirement ('agent_starter tier'), adding useful context. However, it lacks details on rate limits, pagination, error handling, or the format of results, which are important for a search tool with no output schema. The description does not contradict any annotations, as none are given.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, consisting of two sentences that efficiently convey the tool's purpose and usage context. Every sentence earns its place: the first defines the search scope and purpose, and the second adds critical usage guidelines and prerequisites. There is no redundant or verbose language.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a search tool with 9 parameters, no annotations, and no output schema, the description is moderately complete. It covers the purpose and basic usage but lacks details on behavioral aspects like result format, limitations, or error handling. The high schema coverage helps, but without annotations or output schema, the description should do more to compensate, such as explaining what the search returns or how results are structured.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, meaning all parameters are documented in the input schema. The description does not add specific parameter semantics beyond what the schema provides, such as explaining how 'state' or 'property_type' affect the search. It implies filtering for benchmarking but does not detail parameter interactions. With high schema coverage, the baseline score of 3 is appropriate, as the description adds minimal value over the structured data.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: searching a large database of closed commercial transactions for comparable data. It specifies the exact resource ('100,000+ closed commercial transactions') and verb ('Search'), and distinguishes itself from siblings by focusing on comp data for benchmarking price/SF or cap rates against actual recent sales, unlike tools like 'search_deals' or 'get_market_benchmarks' which may have different scopes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: to benchmark a deal's price/SF or cap rate against actual recent sales. It mentions a prerequisite ('Requires agent_starter tier'), which helps guide usage. However, it does not explicitly state when not to use it or name specific alternatives among the sibling tools, such as when to choose 'search_deals' instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_dealsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the output format ('AI-scored deals with cap rates, pricing, green/red flags, and sell signals') and geographic scope ('across the Northeast'), but lacks details on permissions, rate limits, pagination, or error handling. It adds value beyond the schema but does not fully compensate for the absence of annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences: the first states the purpose and output, and the second provides usage guidelines. Every sentence earns its place with no wasted words, and key information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (9 parameters, no output schema, no annotations), the description is reasonably complete. It covers purpose, output format, and usage scenarios, but could benefit from more behavioral details (e.g., response format, limitations) to fully compensate for the lack of structured metadata. It is adequate but has minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all 9 parameters thoroughly. The description does not add any parameter-specific information beyond what the schema provides, such as explaining interactions between parameters or default behaviors. The baseline score of 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Search active commercial real estate listings'), resource ('across the Northeast'), and output ('Returns AI-scored deals with cap rates, pricing, green/red flags, and sell signals'). It distinguishes this tool from siblings like 'find_1031_candidates' by explicitly mentioning 1031 exchanges as one use case among others.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('to find deals matching an investor's criteria, scout a submarket, or identify opportunities for a 1031 exchange'), which helps differentiate it from siblings like 'get_market_summary' or 'search_comps'. However, it does not explicitly state when NOT to use it or name specific alternatives for overlapping functions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_sell_signalA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's function, output meaning ('High sell signal = motivated seller, potential to negotiate'), and a key constraint ('Requires agent_starter tier'), covering essential behavioral traits without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with two sentences that efficiently convey purpose, signal components, interpretation, and tier requirement. Every sentence adds value without redundancy, making it easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (single input, no output schema, no annotations), the description is largely complete: it explains what the tool does, the signal's basis and meaning, and a tier requirement. However, it lacks details on output format or error handling, which could be useful for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the single parameter 'listing_id' with its type and description. The description doesn't add any parameter-specific information beyond what the schema provides, such as format examples or source details, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verb ('Get') and resource ('sell probability signal for a listed property'), including what the signal represents ('likelihood it transacts in the next 6 months based on days-on-market, ownership age, distress tier, and score'). It distinguishes from siblings by focusing on sell probability rather than market data, deal scoring, or searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: for assessing sell probability of a property, with a prerequisite ('Requires agent_starter tier'). However, it doesn't explicitly state when not to use it or name alternatives among siblings, such as when to use score_deal instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• score_dealA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the return format (score, flags, benchmarks, thesis) and a tier requirement, but lacks details on rate limits, error handling, or data persistence. It adequately covers the core behavior but misses advanced operational traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by output details and usage guidelines in two efficient sentences. Every sentence adds value without redundancy, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (AI scoring with 8 parameters) and no annotations/output schema, the description is mostly complete—it covers purpose, output, and usage context. However, it lacks details on behavioral aspects like rate limits or error cases, leaving minor gaps for a tool with significant functionality.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 8 parameters thoroughly. The description adds no additional parameter semantics beyond implying the tool uses these inputs for scoring. Baseline 3 is appropriate as the schema handles the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Submit a deal for AI scoring') and the resource ('deal'), distinguishing it from siblings like 'get_deal' (retrieval) or 'search_deals' (search). It explicitly mentions the comprehensive output (score, flags, benchmarks, thesis), making the purpose distinct and well-defined.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use ('when evaluating a deal not yet in the NDI database') and includes a prerequisite ('Requires agent_pro tier'), which helps differentiate it from alternatives like 'get_deal' (for existing deals) or 'search_comps' (for comparisons). No exclusions are stated, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.