Server Quality Checklist

Tool Scores

• cache_statusC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It implies a read-only inspection via 'Report', but does not state whether it mutates anything, whether both parameters may be supplied together, what happens when both are null, or what 'holds' concretely means (e.g., existence, metadata, document segments).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler, and the core idea is front-loaded. It is efficient, though brevity comes at the cost of missing operational context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although an output schema exists so return-value details need not be in the description, the tool is underspecified for a user trying to call it correctly. Key invocation constraints—parameter optionality, exclusivity, and what a cache status report actually contains—are absent, making this incomplete for reliable tool selection and use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the bare schema. It adds only the relationship 'docket or accession', but does not explain the expected identifier formats, whether at least one is required, whether they are exclusive, or what each parameter affects in the report.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies a specific action ('Report') and a specific resource ('what the document store holds for a docket or accession'), which distinguishes it as a cache-status inspection tool among siblings like get_docket and sync_docket. It does not explicitly name a sibling alternative, but the purpose is not tautological or vague.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no explicit guidance on when to use this tool versus alternatives such as get_docket, sync_docket, or list_files. The intended use case (checking cached holdings before fetching or syncing) is only weakly implied, not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sync_docketB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavioral traits. It mentions 'incrementally' and the scope 'missing from the document store,' but it does not state whether the tool writes to or mutates the document store, whether it is idempotent, or whether it may be a long-running operation. The wording is ambiguous about side effects, which is a significant gap for a tool named 'sync_docket.'

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of twelve words, front-loaded with the verb and object. Every word contributes meaning: 'incrementally' clarifies scope, 'missing from the document store' specifies the target set, and 'for a docket' ties it to the parameter. There is no redundant language.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The output schema exists, so return values need not be explained. However, the absence of annotations and the terse description leave important operational context untold: whether the tool mutates the document store, what 'accessions' means in this domain, how 'incrementally' is determined, and whether a prior cache or docket fetch is required. An agent could not fully assess side effects or prerequisites from this description alone.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one required parameter (docket_number) with zero description coverage. The tool description's 'for a docket' implicitly identifies docket_number as the target docket, adding some contextual meaning. However, it does not specify the expected format, example values, or any constraints, so it only partially compensates for the missing schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('fetch') and a precise resource ('accessions missing from the document store for a docket'). It clearly communicates an incremental sync operation, which is distinct from the other listed tools like get_docket or get_filing. It does not explicitly name sibling alternatives, so it stops short of a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'incrementally fetch accessions missing' implies a backfill/sync scenario, giving some sense of when to use this tool. However, it does not explicitly state when to prefer this tool over alternatives such as get_docket or cache_status, nor does it mention any prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_within_documentB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It discloses the core read-only search behavior and the output shape, but it does not mention side effects, extraction prerequisites, pagination, max_hits behavior, or edge cases. It is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence with a leading verb and no filler. Every phrase adds meaning: the search action, the input type (extracted text), and the output (passages with offsets).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has four parameters with no schema descriptions and no annotations, so more context is required. The output schema covers the return shape, but the missing parameter semantics and lack of usage guidance leave the description incomplete for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description only clarifies 'query' by referring to it as a query. It does not explain accession_number, filename, or max_hits, leaving the agent to guess why both identifiers are required and how max_hits limits results.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies a specific verb ('Search'), a resource ('extracted text'), and an explicit output ('passages with page/char offsets'). This makes it clear what the tool does and distinguishes it from siblings like get_filing_text and read_document, which return full text rather than matched passages with offsets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to prefer this tool over alternatives. It does not mention that it is for searching within a single document rather than across filings, and it does not contrast with siblings such as search_filings, get_filing_text, or read_document.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_document_outlineB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. It does convey the key fallback behavior: return PDF bookmarks if available, otherwise a heuristic section map. It does not, however, state side effects, error conditions, or whether the operation is read-only, though 'Return' implies non-mutating.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence, front-loaded with the action and output type, and no filler. This is as concise as possible while still conveying the tool's core behavior and fallback.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The presence of an output schema covers return-value details, and the two required parameters are simple strings. Still, the description lacks parameter semantics and usage guidance, so the definition is only minimally complete for an agent choosing among siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description needed to explain what accession_number and filename mean, but it does not. The phrase 'stored filing' offers only weak context; the parameter names themselves are doing the work.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description names a specific verb ('Return') and a precise resource: 'PDF bookmarks or a heuristic section map for a stored filing.' This makes the output clear and distinguishes the tool from siblings like get_filing_text or read_document, which return content rather than a document outline.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description gives no explicit when-to-use advice and does not mention any sibling alternative, so an agent must infer from the tool name and output type when to select it over get_filing_text or search_within_document. There are no exclusion conditions or prerequisites stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_filesA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the burden of behavioral disclosure. It reveals the operation is a listing action and hints at has_nonpublic_counterpart semantics only via cross-reference, but it does not state whether the call is read-only, what metadata is returned, or whether pagination or limits apply.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact: two short sentences with no filler. The first sentence states the action, and the second efficiently redirects to get_filing for a relevant term instead of duplicating context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter listing tool with an output schema available, the description covers the core action and workflow ordering. It is close to sufficient, though it would benefit from a brief note on expected file metadata or read-only behavior since annotations are absent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no description for accession_number (0% coverage), and the description only ties it to 'an accession' and the download workflow. This adds some meaning beyond the bare parameter name, but it does not specify the expected format or how to obtain the accession number.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and a concrete resource ('files attached to an accession'), making the tool's function immediately clear. It also differentiates from download_file by positioning itself as the step before downloading.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Call this before download_file,' which gives clear sequencing guidance. It also points to get_filing for understanding has_nonpublic_counterpart. It does not fully enumerate when not to use other sibling tools, so it stops short of a complete routing guide.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_filing_textA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the truncation behavior ('at most max_chars', 'reports total_chars when truncated') and the deprecated status, which is meaningful behavioral context. It does not mention side effects or permissions, but the read-only nature is clear enough for a deprecated text-retrieval alias.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no wasted words. It front-loads the deprecation and core behavior, then provides routing guidance to alternatives. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The output schema exists, so return-value details are not required. The description covers deprecation, truncation, and alternative tools well, but incomplete parameter semantics for file_id and accession_number prevent full completeness. It is adequate but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter meaning. It only explains max_chars; the meanings of accession_number and file_id, and their relationship, are left undocumented. This is a notable gap for an agent trying to call the tool correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is a deprecated alias for read_document and specifies the exact behavior: 'Returns at most max_chars of extracted text.' It names the resource (filing text), the operation (bounded read), and distinguishes itself from siblings by framing it as deprecated and bounded.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly steers agents away from this tool for large filings by recommending get_document_outline, search_within_document, and read_document. However, it does not clearly describe when this tool should still be used, only implies it may be acceptable for smaller bounded reads.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_documentA
  4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly states the truncation behavior, the guarantee that the full document is never returned unless it fits within max_chars, and the response metadata (total_chars, truncated, next_char_start/next_page) when clipped. This is strong, concrete behavioral detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three tightly scoped sentences with the primary action front-loaded. Every sentence earns its place: the return type and source, the critical size limitation, and the response navigation contract.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The output schema covers return-value details, and the description provides solid behavioral context. However, the 6-parameter schema has zero description coverage and the description compensates only for max_chars, so an agent still lacks sufficient guidance on pagination/range parameters and how this tool compares to siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% across 6 parameters, and the description only adds meaning for max_chars. It does not explain pages, char_start, char_end, accession_number, or filename, leaving key range-selection and document-identification semantics undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Return bounded plain text from a cached filing attachment,' which names a specific verb, resource, and scope. The 'Never returns the full document' constraint clearly differentiates it from sibling tools like get_filing_text, which likely returns complete document text.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage context is implied: use this when you need bounded plain text from a cached filing attachment. However, it does not explicitly name alternatives or state when not to use this tool, so the agent must infer routing decisions from sibling names and the bounded-text behavior.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_filingA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full disclosure burden. It discloses a meaningful behavioral limitation ('No protected content is ever returned') and explains that has_nonpublic_counterpart is inferred from filer naming conventions rather than authoritative metadata, which is important for interpreting results.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, front-loaded with the core purpose, and every sentence adds value: the first states what the tool does, the second explains the nonpublic_counterpart semantics, and the third explicitly reassures about protected content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a single required parameter, an output schema, and one key behavioral caveat, the description is largely complete. It explains the non-authoritative nature of an important field. It could be more complete by explicitly naming get_filing_text as the tool for content, but that is not essential for invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only declares accession_number as a string with no description, giving 0% schema coverage. The description compensates by providing the exact expected format ('YYYYMMDD-NNNN'), which is crucial for calling the tool correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Fetch'), the resource ('metadata for one filing'), and the key identifier ('accession number (YYYYMMDD-NNNN)'). This distinguishes it from sibling tools like get_filing_text by emphasizing 'metadata' rather than content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving metadata for a single filing and explicitly notes when a nonpublic counterpart would be relevant (moving for access under 18 C.F.R. 388.113). However, it does not explicitly contrast with search_filings or get_filing_text, leaving some routing decisions to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• collect_relatedA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden, and it delivers: it discloses the 10-docket and 50-filing caps, truncation reporting via dockets_capped and filings_capped_per_docket, the download bundle behavior with a 10-file limit, and date-defaulting rules including the docket skip and response fields. This is well beyond surface-level behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well organized: purpose first, then caps and truncation visibility, then download behavior, then shared search semantics and a safety warning. Every sentence adds operational value, with no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For an 11-parameter tool with no annotations, the description is remarkably complete: it covers caps, truncation, download behavior, date defaults, and important response fields. The output schema exists, so return-value shape is covered elsewhere. Minor gaps remain around the precise 'related' criteria and the semantics of category/industry, but overall this gives an agent enough to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains query, document_type, docket, download, match, search_in, and date_field behavior, and references search_filings for date-defaulting rules. However, category, industry, start_date, and end_date are left essentially unexplained, and exact date formats are not addressed, so compensation is only partial.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening sentence states a concrete action—'Search a term or document type, then list related filings by docket'—with a specific resource and scope. It is clearly distinguishable from siblings like search_filings, get_filing, and download_bundle by the 'list related filings by docket' behavior, and it explicitly references sibling tools for shared semantics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives practical guidance: it explains when download mode is appropriate ('rather than fetching each file individually'), points to download_bundle as a related alternative, warns to narrow the query before enabling downloads, and clarifies that supplying a docket skips the 60-day default. It does not explicitly state when not to use this tool versus search_filings, but the context is strong.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• download_fileA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully carries the burden and does so admirably. It discloses the side effect of saving to FERC_DOWNLOAD_DIR, states that file bytes are not returned, explains refused document types, and reveals how format choices change the saved artifact and result metadata.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is detailed yet tightly organized, with each paragraph serving a distinct purpose: primary action, key caveats, format semantics, and result interpretation. No sentence feels redundant or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity, the description covers prerequisites, refusals, format variants, side effects, return-value semantics, and mismatch detection. The presence of an output schema reduces the need to describe return fields, yet the description still adds useful interpretive context about size_matches_metadata and is_bundle.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It thoroughly explains the format enum values, their defaults, and their behavioral differences, and it explains file_id's role via the list_files prerequisite. accession_number is not explicitly explained, though the tool name and context make it reasonably inferable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Download a public eLibrary file to FERC_DOWNLOAD_DIR.' It clearly distinguishes this tool from siblings by focusing on a single file download and by describing the non-return of file bytes, making its role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit operational guidance: call list_files first to pick a file_id, and it warns that privileged/protected/CEII documents are refused. It does not explicitly compare against the sibling download_bundle, so the choice between this tool and that alternative is somewhat left to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_docketA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does so exceptionally. It discloses row duplication and merging by accession number, total_hits semantics, count_basis=distinct_accession, availability_scope='all', the absence of a 60-day default, client-side date filtering, and page=0 handling. This is far more transparent than most tool descriptions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but every sentence earns its place. It is organized into logical chunks: core purpose, docket/subdocket format, pagination, row-merging behavior, comparison to search_filings, sort order, and date behavior. No fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, the description need not explain return values. It covers edge cases (page=0, subdocket lists, multi-docket filings, privileged filings, client-side date filtering) and differentiates behavior from a key sibling. An agent has enough to call this tool correctly and interpret the result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains docket_number format, subdockets values, page indexing, sort_order meaning and default, and date_field/envelope behavior. The only notable omission is the limit parameter, which is left to inference, but the overall parameter guidance is strong.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb+resource: 'Return the docket sheet: related filings, applicants, and accession numbers.' It also gives concrete docket number examples and clearly differentiates itself from search_filings by scope and behavior. An agent can confidently identify this tool as the one that retrieves a docket sheet by docket number.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly contrasts get_docket with search_filings: availability handling, sort order defaults, and date-field behavior. It implies the primary use case is when you have a docket number. It does not include an explicit 'use this when / use search_filings when' rule, but the comparisons provide strong routing guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_filingsA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full disclosure burden and meets it well. It reveals the default 60-day window for open-ended queries, the no-date-filter behavior when docket or accession_number is supplied, and the presence of response flags like date_range_applied and results_may_be_date_limited. It also discloses nuanced behaviors around date_field and match modes that an agent would otherwise have to discover by trial.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with a clear opening and topic-focused paragraphs, each sentence adds useful information. The opening repeats 'public' twice ('public FERC eLibrary filings' and 'Public documents only'), which is minor redundancy; otherwise it is appropriately dense for a 13-parameter search tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex search tool with no annotations, this description is unusually complete: it covers search scope, date defaults, parameter behavior, and response caveats. An output schema exists to define the return shape, so the description provides enough context for correct invocation without missing essential operational details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, and it does. It explains docket, accession_number, date_field, match, search_in, and document_type with examples and usage guidance. Only page, limit, category, and industry are not directly addressed, but the most consequential parameters are richly specified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource: 'Search public FERC eLibrary filings.' It also scopes the tool with 'Public documents only' and lists concrete supported query keys (keywords, docket prefix, accession numbers, document types), making it clearly distinguishable from siblings like get_filing or list_files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives clear context on how to search: which fields to use, date defaulting behavior, and trade-offs between match and search_in modes. It stops short of explicitly saying when not to use this tool versus a sibling like get_filing, so it lacks explicit when-not/alternatives guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• download_bundleA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden, and it does so thoroughly: it discloses filesystem side effects (writing under FERC_DOWNLOAD_DIR/bundles), default folder reorganization, file/size caps, the skipping behavior for restricted/not-found accessions with reasons and categories, and the fact that it does not return file bytes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but every sentence adds value: purpose, alternative comparison, parameter semantics, caps, skip behavior, and the no-bytes return caveat. It is front-loaded with the core purpose before diving into details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given four optional parameters, no annotations, and no schema descriptions, the description covers all necessary operational context: selection semantics, side effects, limits, failure handling, and return caveats. The presence of an output schema means return-field detail is not required in the description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, but the description compensates by explaining each parameter: accession_numbers select all public files on each accession, file_ids target exact attachments, docket selects public files via search, and organize_by_accession controls folder structure with a clear default behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Zip many public files into one archive under FERC_DOWNLOAD_DIR/bundles.' It also explicitly differentiates itself from the sibling tool download_file by saying 'Prefer this over repeated download_file calls,' making the tool's distinct role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly states when to use this tool ('Prefer this over repeated download_file calls') and enumerates valid input combinations. It does not explicitly spell out exclusions like 'use download_file for a single file or restricted accessions,' but the restricted/not-found skipping behavior implies those cases are not this tool's purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.