Server Quality Checklist

Tool Scores

• ntd_get_milestonesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool returns milestone dates and progress and can filter by four fields, but gives no information about whether it is read-only, authentication needs, rate limits, or the nature of the response (e.g., list vs single object, pagination, data format). The minimal description leaves significant behavioral unknowns.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences, front-loading the purpose. Every sentence contributes: the first states the core function, the second lists filters. No wasted words. However, brevity sacrifices necessary detail, preventing a perfect score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 4 unannotated parameters, no output schema, and no annotations, the description is insufficient. It lacks information on return format, filter combination logic (AND/OR), what 'progress' means, and how it differs from ntd_milestone_completion. The agent cannot fully understand the tool's behavior from this description alone.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds the meaning that the four parameters are filters, which is not present in the schema. However, it does not describe expected formats, exact matching behavior, or allowed values (e.g., no enums, no examples). The added value is marginal but non-zero, moving above a baseline of 2.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns milestone dates and progress for matching POs, with a specific verb ('Return') and resource ('milestone dates and progress'). It lists filter options, which helps distinguish it from sibling tools like ntd_list_pos or ntd_get_po, but does not explicitly differentiate from ntd_milestone_completion or ntd_days_to_mda.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites, exclusions, or context for choosing this tool over siblings like ntd_milestone_completion or ntd_get_po. The implied usage is simply 'when you need milestones for POs', but no explicit direction is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ntd_milestone_completionC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It does not disclose whether the tool is read-only, whether results are cached, how missing milestones are handled, or performance implications of filtering by many parameters. 'Aggregate' implies a read operation but is never explicitly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single succinct sentence that conveys the core function and filter dimensions. It is not overly verbose, but could be split into two sentences for better readability: one for the operation and one listing the filters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 fully optional parameters, no output schema, and no annotations, the description is incomplete. It does not explain what happens when no filters are provided (potentially system-wide totals), nor does it describe the output format (a table, dictionary, or list). The tool is likely complex enough to warrant more detail.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description only lists parameter names without any details on valid values, formats, or behavior when omitted. For example, 'year' could be a single year or range, and 'region' is ambiguous. The description does not compensate for the schema's lack of semantic information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool aggregates milestone completion rates as percentages of POs with each milestone recorded, and lists the available filters (donor, country, drug, region, year). This distinguishes it from siblings like ntd_get_po (single PO retrieval) and ntd_get_milestones (per-PO milestones), but could be more specific about the output granularity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for aggregate analysis across multiple dimensions, but provides no guidance on when to use this versus ntd_summary_stats (also aggregate) or ntd_list_values (filter discovery). No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ntd_days_to_mdaB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility for behavioral transparency. It does not disclose whether this is a read-only or mutable operation, any authentication requirements, or what happens if no results match the filter. The description adds minimal behavioral context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short at two sentences, with the key action front-loaded. Every sentence adds value. It could be slightly more concise by omitting 'Optionally' since the schema already indicates optionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema and annotations, the description should provide more context about return values (e.g., what fields are in each PO, the sort order) and any side effects. It is adequate for a simple filter tool but lacks details that would help an agent predict behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 20% (1 out of 5 parameters has a description: max_days). The description adds the concept of 'days until planned MDA' as the sort criterion, which is not in the schema. However, four parameters (drug, year, donor, country) have no descriptions in either the schema or the tool description, leaving a gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Find') and resource ('POs sorted by days until planned MDA'), and distinguishes itself from siblings like ntd_list_pos, which likely returns all POs without this sorting. However, it could be clearer that the output is sorted in ascending order.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states 'Optionally filter and/or limit by a maximum day threshold', which gives some context on when to use filtering but does not explicitly distinguish this from siblings like ntd_list_pos or ntd_summary_stats, nor does it mention when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ntd_summary_statsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description carries the full burden of disclosing behavioral traits. It does not mention that the tool is read-only, has no side effects, or requires special permissions. The only hint is the aggregation nature, but critical safety and behavioral information (e.g., that it queries precomputed data) is absent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise: one sentence defining the purpose and a compact list of allowed group_by values. It is front-loaded with the core functionality. However, the structure could be improved by separating the purpose from the parameter enumeration (e.g., using a bulleted list). Still, it wastes no words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one parameter and no output schema, the description is adequate but not complete. It states it returns PO counts and total tablet quantities, which is sufficient to infer the output shape roughly. However, it does not specify if results are returned as a list, what the default ordering is, or whether multiple groupings can be requested at once. Given simplicity, a 3 is appropriate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already lists the possible values for group_by with 100% coverage. The description redundantly repeats these values without adding any deeper semantics (e.g., what 'region' means, how each grouping affects the output). Per guidelines, baseline is 3 when schema coverage is high, and no additional parameter context is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool computes PO counts and total tablet quantities grouped by a field, which is a specific aggregation operation. The resource ('PO counts and total tablet quantities') and verb (implied compute/retrieve) are distinct from sibling tools like ntd_list_pos (which lists individual POs) or ntd_get_po (single PO details), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. There is no mention of scenarios where summary stats are preferred over listing individual POs, nor any exclusions or prerequisites. The agent is left to infer context from the name alone, which is insufficient for effective tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ntd_list_valuesB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool returns 'all distinct values' but does not specify the output format (e.g., array of strings, sorted or unsorted), limits, pagination, or what happens if the column does not exist. It also fails to warn about potentially large result sets for high‑cardinality columns.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two short, front‑loaded sentences with no wasted words. The core action ('All distinct values') is stated first, followed by the use case. It is optimally concise for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one parameter and no output schema, the description covers the main purpose but lacks details about the return value structure (e.g., array of unique values), error conditions, and limitations. Given the simplicity, it is adequate but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of the single parameter 'field' with a clear description and examples. The tool description adds the context that the column is for 'any column' and ties to filter discovery, but it does not add new semantic details beyond the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool retrieves 'All distinct values for any column' and frames it as useful for 'discovering valid filter values'. This indicates a specific verb (list) and resource (distinct values of a column). However, it does not explicitly differentiate from siblings like ntd_schema or ntd_summary_stats, which might also provide column-level information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Useful for discovering valid filter values' implies a common use case—exploring column values before applying filters. But there is no mention of when not to use it (e.g., high‑cardinality columns, performance concerns) or how it compares to siblings like ntd_schema (which might list columns but not values). The guidance is implied but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ntd_get_poA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the tool fetches details and milestones, but does not disclose whether this is a read-only operation, any required permissions, or side effects. Minimal behavioral context beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear, front-loaded sentence with no unnecessary words. It efficiently communicates the tool's purpose and key parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool is moderately simple (1 param, no output schema), the description suffices for basic use. However, missing details like what 'full details' includes, how milestone dates are formatted, or whether the PO number must be a specific format could leave gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions the single parameter po_number by name and implies it is a PO identifier, but does not specify format, required length, or any constraints. The description provides basic context but lacks detail to fully replace schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves full details and milestone dates for a single PO by PO number. It distinguishes itself from siblings like ntd_list_pos (which presumably lists POs) and ntd_get_milestones (which may focus just on milestones).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when you need comprehensive PO details for one PO, but does not explicitly say when to avoid it (e.g., if only milestones are needed, use ntd_get_milestones; if summary stats, use ntd_summary_stats). No alternatives or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ntd_list_posA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears the full burden. It explains that results include milestone progress and mentions valid filter values, but does not disclose mutation safety, required permissions, pagination behavior, or what happens when no filters are applied (e.g., returns all POS). Adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is remarkably concise—only three sentences. The first sentence states the core function and output. The second lists donors and drugs. The third lists regions. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no output schema and the parameter count is moderate (7, all optional), the description adequately covers the tool's purpose and the key filter values. It does not explain default behavior when no filters are applied, or return format details, but for a list tool, this is fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so each parameter already has a description. The tool's description adds context about the purpose (returns milestones) but does not elaborate on parameter behavior beyond what the schema already provides, such as how multiple filters interact or the format of country partial matches.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists POs with optional filters, returns key fields and milestone progress per PO. This distinguishes it from siblings like ntd_get_po (single PO retrieval) or ntd_get_milestones (milestones only).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description lists acceptable values for donors, drugs, and regions, which helps users construct valid filters. However, it does not explicitly tell when to use this tool vs. alternatives like ntd_summary_stats or ntd_days_to_mda, nor does it mention prerequisites or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ntd_schemaA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description bears full responsibility for behavioral disclosure. It lists all returned data elements, giving the agent a clear picture of output. It does not mention side effects or error conditions, but given the tool's read-only nature and simplicity, this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no fluff. It is front-loaded with the key output items and ends with a clear usage directive. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (no parameters, no output schema), the description fully explains what the tool returns. It enumerates five specific output components, which is comprehensive for a schema introspection tool. No gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and schema description coverage is trivially 100%. Per guidelines, 0 parameters earns a baseline of 4. The description adds no extra param detail, which is acceptable since parameters are absent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool returns: column names, row count, milestone column list, known filter values, and a sample row. It also explicitly says 'Call this first,' establishing its role as an initial exploratory tool distinct from siblings like ntd_list_pos or ntd_get_po.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The instruction 'Call this first' provides clear guidance on when to use the tool, implying it is a prerequisite before using other sibling tools. However, it does not explicitly state when not to use it or describe alternative tools, but the context is sufficient for an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.