Server Quality Checklist

Tool Scores

• fiscal_headlineA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare the tool read-only and idempotent, so the description adds value by explaining actuals vs. forecast years, the is_projection flag, and the empty-result note behavior. This goes beyond the structured hints and does not contradict them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and logically structured: purpose, use case, parameter examples, and usage guidance. Each sentence contributes, though the dense multi-subject sentences could be slightly more scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only tool with good annotations and an output schema, the description covers the core purpose, usage context, key parameters, and a notable output flag. It does not mention year range defaults or detailed variable semantics, but overall is sufficiently complete for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% parameter description coverage, so the description must compensate. It provides example values for variable (saldo, einnahmen), household (bund, ktn), and model (fs, gfs), but does not explain the meaning of year_from/year_to or what fs/gfs represent. Partial compensation only.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as a 'headline fiscal time series' covering revenue, expenditure, balance, and debt ratios over a specific period. It distinguishes itself from siblings like fiscal_by_institution and fiscal_budget_breakdown by focusing on aggregate evolution, but lacks a direct action verb in the opening phrase.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides a concrete use case ('track how a federal aggregate evolved over time') and explicitly directs users to call fiscal_list_dimensions first for valid values, which is helpful alternative guidance. Does not explicitly state when not to use this tool versus siblings, but the context implies appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fiscal_by_institutionA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly/idempotent/non-destructive. Description adds data availability 'since 2007', allowed variable values, and a note about empty results, which are useful behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, no fluff. Front-loaded with core purpose, followed by example, allowed values, and edge-case note.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Output schema exists, annotations cover safety. The description covers purpose, usage, data range, variable options, and empty-result behavior. It could mention how to discover department names, but sibling fiscal_list_dimensions likely fills that gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must compensate. It explicitly lists the allowed values for variable, and the example hints at year/department usage. However, it does not explain year_from/year_to defaults or department name format, leaving some ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it returns federal spending by department/administrative unit, and provides a concrete use case. Distinguishes from siblings by emphasizing institution-level filtering.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly gives a use case ('compare personnel, IT or external-services spending across departments') with an example. Does not name alternative tools, so not a 5, but the guidance is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fiscal_budget_breakdownA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint, so the description doesn't repeat those. It adds valuable behavioral details: level semantics (1=total, 2=first breakdown), contains as a path-substring filter, and the note returned for empty results. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences: purpose, use case, and parameter explanations. Every sentence adds value, and the structure is well front-loaded with the definition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With four optional parameters and an output schema, the description covers purpose, usage, parameter meanings, and an edge case (empty result note). The only minor gap is the meaning of year=null, but the schema default and overall context reduce the impact.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates by explaining topic with concrete examples, defining level as hierarchy depth, and clarifying contains as a drill-down filter. Year is only loosely implied, but the schema provides type and range, making it reasonably understandable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Hierarchical federal-budget breakdown for one topic and year,' which clearly states the resource and the specific operation. It also differentiates from siblings like fiscal_headline by emphasizing drill-down and filtering via level and contains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides a concrete use case ('see where the money goes') with an example question, and explains when hierarchy parameters are useful. It does not explicitly list alternatives or exclusions, but the context is clear enough for an agent to select this over fiscal_headline or fiscal_by_institution.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• dump_statusA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide read-only, idempotent, and non-destructive hints. The description adds meaningful context beyond that: the deprecation timeline and the SEC-022 namespace rationale. It does not contradict annotations, and the extra context about the tool's future removal is valuable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is remarkably concise: two sentences, front-loaded with the deprecation warning, and every clause adds value. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter alias tool with an output schema and strong annotations, the description fully covers the essential context: what it does, why it exists, how it relates to the sibling, and what will happen to it. Nothing important is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema is trivially complete. The description needn't document parameters. It adds no param info, but none is needed; baseline for 0 params is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Reports cache freshness and upstream health per dataset.' It also distinguishes itself from siblings by explicitly naming the replacement (`fiscal_status`) and noting it is an alias, which removes any ambiguity about what this tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells users not to use this tool in new code ('DEPRECATED — use `fiscal_status`'), explains its status as a backward-compatibility alias, and warns it will be removed. This is exactly the kind of when-to-use vs. when-not-to-use guidance expected.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fiscal_list_dimensionsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover read-only and idempotent behavior, so the description adds value by disclosing the performance characteristic ('may take a moment on a cold cache') and the scope of data loaded ('all three dumps'). It also clarifies the tool's role in converting free-text to exact filter values, which is non-obvious behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: the first states the exact purpose, the second provides a use case and a performance caveat. Every word earns its place; no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no params, output schema exists), the description fully covers what it does, when to use it, and what to expect. The performance note addresses a key operational concern. No gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and an empty input schema, so the description need not explain individual params. The baseline for 0 params is 4, and the description doesn't contradict or omit anything; it implicitly indicates no input is required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists valid dimension values across datasets, naming specific categories (variables, households, models, budget topics, departments). This is a specific verb+resource and clearly distinguishes it from sibling tools that focus on headline figures, breakdowns, status, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'call this first to build correct parameters for the other tools' and explains it turns free-text guesses into exact filter values. This gives precise when-to-use guidance and implies it should precede other tools, effectively differentiating from alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fiscal_statusA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  While annotations already cover read-only, idempotent, and non-destructive behavior, the description adds a meaningful behavioral guarantee: 'Never returns empty silently.' This is useful context beyond the annotations, although it doesn't discuss other potential behaviors like latency or error semantics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact yet information-rich, with two focused paragraphs. The first sentence states the core purpose, the following sentences provide usage context and a key behavioral trait. No filler or repetition of schema/annotation information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a parameterless health-check tool with a rich output schema and strong annotations, the description provides all necessary context: what it does, when to use it, and one critical behavioral guarantee. The output schema presumably covers return values, so the description needn't repeat them.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes zero parameters, so there are no parameter semantics to explain. The schema coverage is 100% (empty schema), and the description appropriately focuses on behavior rather than inventing unnecessary parameter details. Baseline 4 is appropriate for parameterless tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with a specific verb ('Report') and a clear resource ('cache freshness and upstream health per dataset'), which precisely distinguishes it from data-query sibling tools like fiscal_headline or fiscal_budget_breakdown. It also labels itself as 'the health endpoint of this server,' reinforcing its distinct purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It states an explicit use case: 'check whether the data is fresh, cached or degraded before trusting a figure.' It also frames itself as the health endpoint, implying it should be used for status checks rather than data retrieval, and notes its role in graceful degradation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.