Server Quality Checklist

Tool Scores

• detect_outbreak_signalsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It adds valuable context: a flag means 'high for this time of year' compared to prior years, not merely high in absolute terms. This is beyond what the name/schema convey. It does not mention side effects or edge cases, but the read-only analytical nature is implicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by a compact, well-organized argument list, and ends with a single clarifying interpretation note. Every sentence contributes value with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, the presence of an output schema, and the absence of annotations, the description covers the essential aspects: purpose, parameter meanings, and the seasonal baseline nuance. It does not mention usage alternatives or data limitations, but for a focused detection tool it is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must fully compensate. The 'Args' section adds meaningful semantics: province is case-insensitive, lookback_weeks specifies a default of 12, and sigma is explained as standard deviations above the baseline. This goes well beyond the raw schema properties.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource statement: 'Flag recent weeks running above the same-calendar-week seasonal baseline.' This clearly distinguishes the tool from siblings like compare_provinces, list_provinces, get_incidence_history, and forecast_incidence by focusing on seasonal anomaly detection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a usage context (detecting outbreak signals via seasonal comparison) but provides no explicit when-to-use/when-not-to-use guidance or alternatives. Sibling tools are not referenced, so an agent would rely on the purpose statement alone to infer when to apply it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_incidence_historyA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full disclosure burden. It discloses case-insensitive province matching, optional date bounds, max_weeks cap with most-recent-first ordering, and the returned metrics (incidence per 1,000 and rainfall driver). This is useful behavioral context beyond the schema, though it doesn't address error handling or side effects, which are minimal for a read-only history query.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured with an Args section and Returns statement. Every sentence adds value, and there is no filler or repetition. It is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present, the description need not detail return structures, but it still provides high-level return context. Parameter semantics and scope are adequately covered. A minor gap is the interaction between max_weeks and the date bounds, but this is not critical for a basic query tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain each parameter. It does so fully: province with examples and case-insensitivity, start_week and end_week as ISO date bounds, and max_weeks as a cap with default and ordering. This adds significant meaning beyond the bare schema types and titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a precise verb and resource: 'Return weekly malaria incidence history for one province.' This clearly specifies the tool's function and distinguishes it from siblings like compare_provinces, forecast_incidence, and list_provinces. The scope 'one province' also differentiates it from multi-province tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when to use this tool by stating 'for one province' and focusing on historical incidence. It doesn't explicitly name alternatives or exclusions, but the context is clear: this is for fetching history for a single province, not for comparison, forecasting, or outbreak detection. This is clear context without explicit when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_provincesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It explains that strata are derived from K-means clustering of historical burden and enumerates the possible values ('high', 'moderate', 'low'). This adds meaningful context about the data source and semantics. It doesn't mention side effects, but this is a simple list operation with likely no side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise: two sentences. The first states the primary purpose, the second explains the strata and provides usage guidance. Every word earns its place, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter listing tool with an output schema, the description is complete. It specifies the scope (Angolan provinces under surveillance), the output (epidemiological stratum), the meaning of the stratum values, and the recommended usage order. Nothing crucial is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameters (0 params, 100% coverage vacuously). The baseline for 0 params is 4. The description adds no parameter information, but none is needed. It does clarify the meaning of the output values, which is more relevant here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'List Angolan provinces under surveillance with their epidemiological stratum.' This uses a specific verb (list) and resource (Angolan provinces), and distinguishes it from sibling tools that compare, retrieve history, forecast, or detect outbreaks. The scope is well-defined.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to use the tool: 'Use this first to resolve province names before calling other tools.' This gives strong positioning as a prerequisite, though it does not explicitly name alternatives or exclusions for other tools. Clear context but no explicit when-not-to-use comparison.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_provincesA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations, so the description carries the full burden. It clearly explains the ranking behavior (highest first), parameter ranges, and purpose. It doesn't explicitly mention non-mutating behavior, but the action is read-only by nature and the response format is handled by the output schema. Slightly more detail on edge cases could push to 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three brief sections: purpose, args, usage. No wasted words. Every sentence adds value, and the structure is easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with 2 optional parameters, an output schema exists, and the description provides usage context, parameter semantics, and downstream tools. Fully adequate for an AI agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has zero description coverage, but the description explains both parameters: horizon_weeks ('1 to 8') and top_n ('how many provinces to return, highest forecast first'). This fully compensates for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Rank provinces by forecast incidence to support prioritisation' with a clear verb (Rank) and resource (provinces). It distinguishes itself from siblings like `forecast_incidence` by focusing on cross-province comparison for prioritization.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides an explicit use case ('where should we pre-position resources') and names follow-up tools (`forecast_incidence`, `detect_outbreak_signals`) to drill into a single province. This gives clear when-to-use and alternative context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• forecast_incidenceA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the empirical 80% interval methodology (rolling-origin backtest residuals), explains that out-of-range horizons are refused rather than extrapolated, mentions case-insensitivity for province input, and directs users to the model-card for known failure modes. This is rich, transparent behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the core purpose and confidence intervals. Every sentence adds value: the Args section is directly relevant, and the reference to the model-card is a valuable safety note without padding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the moderate complexity (forecast, uncertainty, validation) and the presence of an output schema, the description covers all essential aspects: what, how far ahead, uncertainty interpretation, validation constraints, and a reference for caveats. The output schema handles return value details, so no further description is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no property descriptions (0% coverage), but the description's 'Args' section adds crucial meaning: province is case-insensitive, and horizon_weeks must be 1-8 with refusal beyond that range. It also clarifies the default isn't stated but appears in schema. The description fully compensates for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Forecast') with a specific resource ('weekly malaria incidence') and clear scope ('1-8 weeks ahead, with 80% intervals'). This clearly distinguishes it from sibling tools that compare, list, retrieve history, or detect signals.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly indicates when to use the tool (for forecasting future incidence) and provides the important context that horizons beyond the validated range are refused. It references the model-card for validation details and failure modes, which is useful guidance, but it does not explicitly name alternative tools or state 'when not to use' beyond the horizon restriction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.