Server Quality Checklist

Tool Scores

• get_filing_sampleA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses what is included (agent_summary, financial_metrics) and excluded (alpha_signals/causality_events), along with the cost context. However, no annotations exist, and the description could further clarify if there are any side effects or idempotency, though not essential.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, using bullet points and clear sections (cost, example arguments). Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and an output schema present, the description covers purpose, usage guidelines, behavioral details, and parameter semantics adequately. It leaves no significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter document_id is fully described in the schema (100% coverage). The description adds value by explaining it comes from search_filings and defaults to a demo filing, which supplements the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it provides a free preview of one SEC filing, specifically a compact row from fi_listings_portfolio_compact, and contrasts with purchase_filing. The verb 'get' combined with resource 'filing_sample' is specific and distinguishes it from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly compares with purchase_filing: 'sample = evaluate quality; purchase = full evidence-backed JSON'. It also mentions the cost ($0) and what is included vs. not included, providing clear guidance on when to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_filingsA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses cost, that it does not return alpha_signals/events/metrics, and that it always returns agent_readiness_score and edgar_url. Also notes required filters for performance.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with front-loaded purpose, then key constraints, workflow, and examples. Each sentence adds value, though slightly verbose. Could trim redundant phrasing, but overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 9 parameters, no annotations, and an output schema (not shown), the description adequately covers return fields and usage context. Workflow integration with sibling tools is clearly explained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds value through examples, cost hints, and usage context (e.g., 'limit default 10 keeps payloads small'), enhancing understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is the 'PRIMARY discovery tool' for SEC filings, a lightweight catalog. It distinguishes from siblings (get_filing_sample, purchase_filing) with specific details about what it does and does not return.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use: as the first step in a workflow, with required filters to avoid full-table scans. Includes workflow steps after the call and examples of minimal and maximal arguments, guiding usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• purchase_filingA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses the tool's behavior: it requires an on-chain payment via 402 workflow, calls must be made in two phases, and the output includes alpha_signals and financial_metrics. It explicitly states what is not returned (agent_bundle or internal pipeline fields).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections: purpose, cost, output, workflow, examples. It is somewhat lengthy but every sentence adds value. Minor redundancy could be trimmed, but overall effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of the 402 payment workflow, the description is highly complete: it explains the multi-step process, expected output, cost, parameter usage, and what not to include. Schema and output schema support completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds significant meaning: document_id origin (search_filings or get_filing_sample), network default and alternative, and crucial instruction to leave tx_hash empty on first call. Example arguments further clarify usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool purchases and delivers one full SEC filing row from fi_listings_portfolio. It distinguishes from siblings get_filing_sample and search_filings, which are for browsing/finding filings, by focusing on the purchase action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides a mandatory 2-step 402 flow with step-by-step instructions, example arguments for minimal and maximal calls, and cost details. It clearly indicates when to use the tool (after finding a filing) and how to perform the payment workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.