Server Quality Checklist

Tool Scores

• get_reporting_seasonA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint, openWorldHint, idempotentHint, and non-destructive hint. The description adds valuable context: charging model (per aggregate row), authentication requirements (APIFY_TOKEN, Apify credits), and data source restrictions (CC BY-SA exclusion, blank check handling). This goes beyond annotations with concrete behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is comprehensive at ~150 words without being verbose. It front-loads the main purpose and return structure, then explains parameters, then pricing, then auth. Some redundancy exists (e.g., 'Read only' at end repeats annotations), but overall well-organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 22 parameters, 100% schema coverage, and no output schema, the description adequately explains how to use the tool for its main purpose (bucketed reporting season analysis). It covers filter constraints, pricing, authentication, and data limitations. However, it does not describe return format or pagination (though 'Truncation is always reported' is hinted in limit description).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds value for key parameters (season_group_by, season_split_by) by explaining their role in bucketing and splitting, and notes default windows. However, many parameters have clear schema descriptions already (e.g., regions, sectors), and the tool's narrative does not significantly augment them.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to show reporting load distribution over time, distinguishing it from siblings like get_reporting_timing which likely focuses on individual company timing. It specifies returns aggregate rows per bucket, not per company, which aligns with readOnlyHint and openWorldHint.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description notes that at least one filter is required and company identifiers are not accepted, providing basic usage guidance. However, it lacks explicit when-to-use vs alternatives: for example, when to use get_reporting_season vs build_company_universe for bulk data needs. No exclusion criteria or prerequisite context beyond the required filter.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• resolve_companyA
  4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds valuable behavioral context beyond annotations: charging per row (including no_match), Apify token/credit consumption, match_method and matched_on fields, and limitations (former names not available). It also explicitly states 'Read only' aligning with readOnlyHint. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is about six sentences, front-loading purpose and return fields. It includes important details (charging, no_match rows) without being overly verbose. Minor redundancy: 'Read only' is stated despite annotation, but it's acceptable. Efficient overall.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (23 parameters, no output schema), the description covers essential aspects: what identifiers are accepted, return format (44 fields including match_method), behavior on no-match, charging model, and prerequisites (APIFY_TOKEN). It compensates well for the lack of output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents each parameter. The description does not add new parameter-level information beyond what the schema provides, except for high-level context about what the tool resolves. Hence baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool resolves identifiers (domain, ticker, ISIN, LEI, CIK, company name) to a listed company identity, listing many returned fields. It is specific about the verb and resource, and the name 'resolve_company' aligns well. While it doesn't explicitly differentiate from siblings like 'qualify_company', the distinct purpose is obvious.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not mention when to use this tool versus its siblings (qualify_company, get_reporting_timing, etc.). It provides context on charging and behavior (e.g., always returns one row) but lacks guidance on selecting alternatives or excluding certain use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_reporting_timingA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description richly supplements the readOnlyHint/idempotentHint annotations by explaining that dates are predicted, confidence thresholds, non-US resolution behavior, per-row charging, APIFY_TOKEN requirement, and consumption of credits. It adds substantial behavioral context without contradicting any annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but organized around the purpose, output fields, limitations, and cost. Almost every sentence adds information, though the final 'Read only' is redundant with the readOnlyHint annotation, and the all-caps emphasis is unnecessary.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 30-parameter tool with no output schema, the description covers the essential output fields, predicted-date caveats, US-only scope, non-US fallback, and pricing. It does not enumerate all 75 output fields, but it names the most important ones and gives enough context for an agent to invoke and interpret the result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, so the baseline is 3. The description adds value beyond the schema by giving defaults for window_lead_days/window_lag_days, enumerating window_status values, and explaining confidence thresholds, which helps select key parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds when a public company next reports and an outreach window around that event, with a specific verb and resource. However, it does not explicitly differentiate from the sibling get_reporting_season, so the distinction is implicit rather than stated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context: US-only coverage, non-US behavior and charging, and points at window_status as the field to filter on. It does not explicitly say when to prefer this over resolve_company, qualify_company, or get_reporting_season, but the described use case is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• qualify_companyA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses key behavioral traits beyond the annotations: the default null vs false for unmatched companies, the opt-in behavior with assume_unmatched_is_private, the cost model ('Charged per company row returned'), and authentication requirements ('Requires an APIFY_TOKEN'). The annotations already provide readOnlyHint, openWorldHint, idempotentHint, and destructiveHint, and the description adds valuable context without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and uses five sentences to cover purpose, identifiers, mode parameters, unmatched behavior, and cost/auth. Every sentence is informative and earns its place. Given the tool's complexity (26 parameters), this is efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the tool having 26 parameters, the description only explains the three mode parameters and the unmatched behavior. The role of the other 23 parameters (sectors, regions, exchange_codes, etc.) is not mentioned. The description does not clarify how these filter parameters relate to the 'qualify' action, leaving a significant gap in context for an agent. No output schema is provided, and the description only vaguely mentions 'returns a listed status verdict per input' without specifying the return shape.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds meaning by explaining the three 'qualify mode' parameters (listed_only, suppress_listed, assume_unmatched_is_private) in the context of the tool's objective, and clarifies the default behavior for unmatched inputs. This provides extra guidance beyond the individual parameter descriptions in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with 'Answer whether a company is publicly listed.' which is a specific verb+resource. It distinguishes from the sibling 'resolve_company' by stating it returns a 'listed status verdict' and uses the same identifiers. This clearly conveys the unique purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use the listed_only and suppress_listed modes ('when selling only into private companies'), and explains the default behavior of unmatched companies. It references the sibling resolve_company for identifiers, giving context for usage. However, it does not explicitly state when not to use this tool or compare to other siblings, so a 4 is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• build_company_universeA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes well beyond the annotations (which already declare readOnlyHint, openWorldHint, idempotentHint, destructiveHint). It discloses: minimum one filter required, default limit of 1000 rows, per-row charging, truncation reporting behavior (never silent), APIFY_TOKEN requirement, and Apify credit consumption. This fully characterizes operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the key purpose, followed by essential usage constraints and behavioral details. Every sentence adds distinct value—no fluff or repetition. It efficiently covers purpose, alternatives, cost implications, and special notes in under 150 words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the high parameter count (17) and no output schema, the description is remarkably complete. It notes the output structure (44 fields, same shape as resolve_company), explains truncation reporting, and covers all key behavioral constraints. The annotations further support completeness by indicating idempotent and read-only behavior. No obvious gaps remain for an agent to fail.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Even though schema description coverage is 100%, the description adds substantial context beyond the schema. For example, it states that regions widen rather than replace filters, public_float_bands are based on public float because market cap is unavailable, fiscal_year_end_months is primarily a US field, and security_types are specifically enum values. These enrichments justify a top score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool builds a list of listed companies using filters, not identifiers. It uses a specific verb ('Build') and resource ('list of listed companies'), and explicitly distinguishes from sibling tools by noting it does not accept company identifiers and directing users to resolve_company, qualify_company, or get_reporting_timing for specific companies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool (filter-based universe building) and when not to (company identifier lookups). It names alternative tools (resolve_company, qualify_company, get_reporting_timing) and includes a critical usage warning about setting the limit explicitly to control costs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.