Server Quality Checklist

Tool Scores

• search_filingsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so the description adds limited behavioral context beyond that (e.g., paginated list, agencies). It does not disclose rate limits, authentication, or response structure, but the main traits are adequately covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the action and key filters. Every sentence adds value with no redundancy. Efficiently communicates the tool's purpose and scope.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 10 parameters and no output schema, the description is adequate but incomplete. It mentions pagination and agencies but doesn't describe the response fields, count_only behavior, or default ordering. More context would help agents understand the returned data structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the description adds little beyond summarizing the main filters (keyword, agency, type, status, date range). It omits mentioning page, per_page, count_only, docket, which are in the schema, but overall it provides a concise overview without new semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search space regulatory filings by keyword, agency, type, status, and date range', specifying the action, resource, and key filters. It distinguishes from siblings like get_filing_detail (single filing) and search_entities (entities).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. For example, it doesn't mention that get_filing_detail should be used for individual filing details. The description only states what it does, not when to choose it over siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_entity_profileA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description correctly implies a read-only operation ('Get'), consistent with readOnlyHint=true. However, it adds no further behavioral context beyond what annotations already provide (e.g., no mention of rate limits, authentication, or return structure).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with purpose, no wasted words. Efficient and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema, the description provides a reasonable overview of the return content (filing history, related entities, etc.). However, it could be improved by mentioning the output is a structured object, but this is not critical for a read-only tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with clear descriptions for all parameters. The description adds no additional meaning beyond the schema (e.g., entity UUID, date filters). Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('get') and resource ('entity profile'), and lists key components (filing history by agency, related entities, linked satellites, cross-references). This clearly differentiates it from sibling tools like 'get_entity_dossier' or 'get_bond_portfolio'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus the 20+ sibling tools. There is no mention of alternative use cases or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_sec_filingsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so the description adds limited behavioral context beyond specifying the scope (space companies, form types). No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no fluff. Front-loaded with the core action and scope. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 7 parameters, full schema coverage, and no output schema, the description is mostly complete. It could mention sorting or default limits, but the limit parameter covers pagination. Adequate for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 7 parameters adequately. The description lists some filter options but adds no new meaning beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches SEC filings for tracked space companies and lists specific form types (8-K, 10-Q, 10-K). It distinguishes from siblings like search_filings by mentioning space companies, but does not explicitly differentiate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description suggests usage for financial signals with an example, but does not provide explicit when-not-to-use guidance or name alternatives. Usage context is implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_launch_historyA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and description adds source database (FAA/GCAT) and return fields, but no additional behavioral details like pagination or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences are concise and front-loaded, but could be slightly more structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple query tool with no output schema, description covers main return fields and requirement. Adequately complete given context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. Description does not add significant meaning beyond what schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb 'Get' and resource 'launch history for a space entity', and distinguishes from sibling tools like get_entity_dossier or get_entity_profile.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Specifies requirement for entity_id but lacks guidance on when to use this tool versus alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_spectrumA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description is consistent with the readOnlyHint annotation, but it does not disclose additional behavioral traits such as pagination, rate limits, or data freshness. It adds minimal context beyond what annotations already provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: two sentences with no wasted words. The first sentence immediately states the purpose, and the example reinforces usage efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and 7 parameters, the description adequately explains inputs (range) and outputs (allocations joined to filing and applicant). It could mention default limit but is otherwise complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 7 parameters have descriptions in the schema (100% coverage), so the description adds little new semantic value beyond contextualizing freq_low_mhz and freq_high_mhz as a range. The schema already handles parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool finds entities holding a frequency band and returns spectrum allocations. It provides a clear example and distinguishes from sibling tools which cover different domains (e.g., satellites, filings).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage via the example and listing of filter options, but it does not explicitly state when to use this tool over alternatives or provide exclusions. Usage is clear but not comprehensive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_filing_distributionA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description aligns with the readOnlyHint annotation, indicating safe read operation. However, it doesn't disclose additional behavioral traits such as pagination, rate limits, or whether the distribution includes all filing types or only those with data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with front-loaded action and clear purpose. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Without an output schema, the description fails to specify the structure of the returned distribution (e.g., list of filing type counts). It mentions 'filing count distribution' but lacks detail, which is a gap for an aggregation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All four parameters have descriptions in the input schema (100% coverage). The description adds minimal value beyond reminding about agency and date-range filters, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get filing count distribution by filing type,' specifying the verb and resource. It distinguishes from siblings like get_filing_detail or get_filing_trends by focusing on distribution across types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Use for questions about what types of filings are most common.' While it doesn't list alternatives, the context of sibling tools makes when-to-use clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_filing_trendsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true, so the description correctly implies a read operation. The description adds that it returns trends and top movers but does not detail behavior like aggregation methods, pagination, or rate limits. This is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the primary function, and contains no redundant or extraneous information. Each sentence serves a clear purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 6 parameters and no output schema. The description explains what the tool returns (trends and optional top movers) but lacks details on the output structure (e.g., list of periods, counts). This is a gap for an agent to fully understand the response format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents each parameter. The description adds context about optional top movers but does not explain parameter specifics beyond the schema. Therefore, it adds minimal value, matching the baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool retrieves filing volume trends over time periods, with optional parameters for top movers. This distinguishes it from siblings like get_filing_detail (specific filing) and get_filing_distribution (filing distribution), making its purpose specific and distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises using the tool for questions about growth, trends, emerging players, or year-over-year comparisons. However, it does not provide exclusions or mention alternative tools for different queries, such as get_filing_distribution for distribution queries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_system_statusA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, indicating a safe read operation. The description adds context about specific health components but does not disclose any additional behavioral traits (e.g., rate limits, data recency). It does not contradict annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently conveys the tool's purpose and key components with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Without an output schema, the description lists what the tool checks but does not describe the return format or how status is reported. This gap leaves the agent uncertain about the response structure. Adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, and schema coverage is 100% trivially. The description does not need to add param info, and it appropriately focuses on tool purpose. Baseline for zero parameters is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'system health status', listing specific components like database connectivity, pipeline queue depth, and per-source crawl health. This distinguishes it from sibling tools which focus on entities, filings, or other data queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for checking system status, but provides no explicit guidance on when to use this tool versus alternatives, nor any when-not scenarios. Sibling tools are diverse, but no direct alternative for system health is present; guidance is minimal.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• researchA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotation already indicates readOnlyHint=true (safe read operation). Description adds that it searches in parallel but does not disclose performance characteristics, rate limits, or empty result behavior. Adequate but not enriched.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences, front-loaded with purpose. No redundancy or filler. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a broad search tool with 3 well-documented parameters and no output schema, the description covers usage intent and priority. Lacks return value description but is sufficient for selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameters are already well-documented. Description adds usage guidance for the agency parameter but duplicates the focus parameter's default. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it is the primary research tool that searches filings, entities, and semantic index in parallel. Distinguishes itself from more specific sibling tools like search_filings, search_entities, and search_semantic.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this FIRST for any question' and advises passing the agency parameter for agency-specific questions. Provides clear context but does not explicitly mention when not to use it or name alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_entitiesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true, so no contradiction. The description adds 'Supports fuzzy name matching', which is useful but does not disclose pagination behavior, rate limits, or default ordering. With annotations covering read-only, the description adds moderate value.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no fluff, front-loaded with purpose and key features.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with 5 optional parameters and no output schema, the description covers the major dimensions: entity types, filters, and fuzzy search. It lacks details on default behavior when no parameters are given or result sorting, but is otherwise adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description mentions filtering by name, type, or country, but this repeats schema info without adding new semantic details like accepted data formats or enum values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb (Search) and resource (regulatory entities), lists entity types and filter criteria, and clearly distinguishes from sibling tools like search_filings or search_satellites.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states the tool searches regulatory entities by name, type, or country with fuzzy matching, providing clear context. However, it does not explicitly specify when not to use this tool or compare with alternatives like search_screening or search_semantic.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_federal_awardsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true; description adds that results are ordered by award amount and joined to resolved entity, but does not cover pagination, rate limits, or error states.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise: two sentences with no unnecessary words. Action, parameters, ordering, and use cases are all presented efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, description provides enough context (join, ordering, examples) for typical award search queries. Could mention return fields, but still functional.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions already cover all parameters (100% coverage). Description merely lists example filter fields without adding new semantic meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches U.S. federal awards from USAspending, specifying the resource (contracts+IDVs) and join to resolved recipient entity. It distinguishes from sibling search tools like search_entities which search for entities, not awards.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides two explicit use cases ('NASA contracts to Boeing over $1B' and 'operator's federal funding footprint'), indicating when to use. However, does not explicitly contrast with alternatives or state when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_filing_detailA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark readOnlyHint=true. The description adds value by detailing returned content (spectrum data, attachments, etc.), but does not disclose other behavioral traits like error handling or rate limits. It does not contradict annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently conveys core functionality and data categories, with no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one required param) and lack of output schema, the description thoroughly enumerates the types of data returned (spectrum, orbital parameters, etc.), making it complete for understanding the tool's scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single parameter 'id', with description 'Filing UUID'. The tool description adds no further meaning to this parameter, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get full details of a specific regulatory filing' with a specific verb and resource, and lists distinct data categories (spectrum, orbital parameters, etc.), differentiating it from sibling tools like get_filing_distribution or get_filing_trends.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for obtaining comprehensive filing details but does not explicitly state when to use this tool versus alternatives like search_filings or get_filing_distribution. No when-not or exclusion criteria are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• milestone_adherenceA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=true, which matches the read-only nature of the tool. The description reinforces that it returns data on milestone adherence, with optional aggregate counts. No contradictions; no additional behavioral concerns need disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence with a semicolon separating purpose from filter instructions. It is front-loaded and contains no extraneous words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, filters, and summary mode. Absence of output schema is noted but acceptable for a simple query tool. It could mention default behavior or data format, but overall adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the description adds minimal new meaning beyond listing the parameters. It provides a quick reference for classification values (met|pending|etc.) which adds marginal value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: reporting FCC deployment milestone adherence for satellite systems, including regulatory citation. It distinguishes itself from sibling tools like get_bond_portfolio or search_satellites.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description lists available filters (call_sign, classification, is_ngso, summary) and implies usage context for regulatory compliance queries. It does not explicitly state when not to use or suggest alternatives, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_positionsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true, so the description doesn't need to restate safety. It adds value by explaining the data origin (LLM-extracted) and return granularity (one row per filing, argument), which are behavioral traits beyond the annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences: first sets function and scope, second lists filter options, third gives concrete examples and return format. No redundancy, front-loaded with key purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, but the description explains the return structure (one row per filing, argument). It covers filter categories and provides examples. Missing details on pagination or ordering, but limit parameter covers result count.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description lists filter categories but does not add detail beyond what the schema already provides for each parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches LLM-extracted policy arguments across COMMENT/REPLY/PETITION filings with specific filters. It distinguishes from sibling tools like search_filings (filings-level) and search_semantic (semantic search) by focusing on structured arguments and providing concrete example questions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes example questions ('who opposed X?', 'which filings support modular satellite licensing?'), which imply appropriate use cases. However, it does not explicitly state when not to use this tool or mention alternatives like search_filings for raw filings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_satellitesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses data sources, the join to resolved operator entity, and a nuance about operator field using country codes. Adds value beyond the readOnlyHint annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence with examples, no fluff. Front-loaded with action and efficiently conveys key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers search purpose and data sources well. Lacks explicit mention of pagination or default/max limits, but limit is described in schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. Description lists parameters but adds no new information beyond the schema except noting the operator quirk.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches satellite catalogs (UCS + Space-Track SATCAT) by multiple identifiers and provides specific example queries, making the purpose unambiguous and distinct from sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides concrete example use cases ('what does SpaceX have in LEO?', 'find NORAD 44713'). Does not explicitly state when not to use, but context makes it clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_screeningA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description aligns with the readOnlyHint annotation, as 'Check entities' implies a read operation. It adds value by specifying the consolidated lists and filtering options, going beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with purpose followed by usage tip. No wasted words, highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 5 optional parameters and no output schema. The description explains the purpose and lists checked but omits response format. However, it is adequate for an agent to infer the tool's role.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameters are well-documented in the schema. The description recaps the filters (entity, name, list, similarity) but does not add significant new semantic detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool checks entities against sanctions/export-control screening lists, listing specific lists like OFAC SDN, BIS Entity List, ITAR Debarred. This distinguishes it from sibling search tools like search_entities, which is a general entity search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use for compliance / due-diligence questions,' providing a clear usage context. It doesn't specify when not to use or mention alternatives, but the guidance is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_bond_portfolioA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true, and the description aligns by stating 'Get' and describing the data returned. It adds valuable behavioral context by listing specific data categories (bond amounts, releases, summary statistics, FAA data), going beyond the schema annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, efficiently covering purpose, content, and usage guidance. No wasted words; information is front-loaded with the core action and regulation first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the lack of an output schema, the description sufficiently explains the return data: bond amounts, releases, operator details, summary statistics, and FAA financial responsibility. This is complete for a read-only data retrieval tool with no nested objects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with all 5 parameters described. The description does not add significant semantic detail beyond the schema (e.g., it mentions 'operator' and 'orbit_type' indirectly but not explicitly). Baseline 3 is appropriate as the schema already does the work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves FCC surety bond portfolio data for satellite operators, referencing the specific regulation 47 CFR 25.165. It details what is shown (bond amounts, releases, operator details, summary stats, FAA data) and differentiates it from siblings, all of which are other search/retrieval tools not focused on bonds.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states 'Use for questions about satellite operator financial obligations, bond compliance, or TPL coverage,' providing clear when-to-use guidance. It does not explicitly mention when not to use or cite alternatives, but the context is sufficient given the diverse sibling set.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_top_filersA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations include `readOnlyHint: true`, confirming no destructive behavior. The description adds that results are ranked by filing count and filters by agency/date range, which goes beyond the annotation. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words. The first sentence states the core purpose and ranking; the second provides filtering context and usage examples. Highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a retrieval tool with 5 optional parameters and no output schema, the description sufficiently covers the purpose, filtering, and typical use cases. It is complete given the complexity level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 5 parameters are described in the input schema with 100% coverage. The description mentions filtering by 'agency and date range', which aligns with the schema but adds no additional meaning beyond what is already present.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Get' and resource 'top filing entities' ranked by number of filings. It distinguishes itself from sibling tools like `get_filing_trends` and `get_filing_distribution` by focusing on rankings of entities rather than trends or distributions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use: 'Use for questions about who files the most, biggest players, or filing rankings.' This provides clear context. It does not mention when not to use or list alternatives, but the sibling tools cover distinct purposes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_entity_dossierA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true, consistent with a read operation. The description adds useful behavioral context: it returns 'counts plus recent samples' and explains how parameters affect the rollup. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact (3 sentences), front-loaded with the core purpose, and every sentence adds value. No fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (dossier from multiple sources) and no output schema, the description informs that results include 'counts plus recent samples'. It covers key parameters well. A brief note on the general output structure would improve completeness but not critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (all parameters described). The description adds meaning beyond the schema: it explains that include_family 'rolls the totals up across the entity's corporate family', clarifies family_confidence strictness levels with examples, and describes include_subsidiaries as rolling up direct subsidiaries from SEC data.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb phrase 'Get a cross-source dossier for an entity (by UUID)' and lists concrete data types (regulatory filings, SEC signals, sanctions screening, asset footprint). It clearly distinguishes from siblings which are mostly search tools or single-source tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the key parameters (include_family, include_subsidiaries) and states the tool provides 'the most complete single view'. It implies use for a comprehensive overview but does not explicitly exclude alternatives or state when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_semanticA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds value beyond annotations by specifying the use of nomic-embed-text-v1.5 embeddings and noting limited FCC filing coverage. This complements the readOnlyHint annotation with operational constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, each serving a clear purpose: the first defines the tool's function and method, the second provides a critical caveat and alternative. No filler or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 6 parameters and no output schema, the description sufficiently covers its purpose, embedding technique, limitation, and alternative tool, providing a complete mental model for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema coverage, the description adds context about the embedding model and why parameters like limit might matter, but does not repeat schema details. This provides meaningful additional guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs semantic vector search using natural language, distinguishing it from keyword search by explicitly mentioning 'meaning' vs 'keywords'. It also names the sibling tool search_filings for FCC searches.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises to use search_filings for FCC keyword search due to limited embedding coverage, providing clear when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_ground_stationsA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true, and the description adds transparency by stating results are ordered by great-circle distance and labeled with source. No contradictions; it provides useful behavioral context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured: a summary sentence, followed by source details and an example. Every sentence adds value without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 8 parameters and no output schema, the description comprehensively covers search functionality, source selection logic, defaults, ordering, and examples. It leaves no ambiguity for agent invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema description coverage is 100%, the description adds significant meaning: it clarifies the interplay between source, near, band, operator, and default radius. For example, it states band search requires source=extracted, which enriches the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches ground/earth stations by name, frequency band, operator, or geographic proximity, with specific examples. It distinguishes from sibling search tools by detailing its unique capabilities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use each source (fcc vs extracted) and provides explicit examples like 'earth stations within 200km of 38.9,-77.0'. It also notes default behaviors for proximity vs band/entity searches, giving clear guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.