Server Quality Checklist

Tool Scores

• find_patientsC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It implies a read-only 'find' operation but says nothing about return volume, pagination, sorting, or whether the filters combine as AND/OR. There is no mention of limitations or special behavior beyond the core action, leaving the agent without essential operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, which is concise and easy to parse. However, it front-loads the purpose but omits crucial details like parameter semantics and usage guidance. It earns its place as a purpose statement but lacks the substance needed for effective tool selection.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that this is a filtering tool with a nested filter object and no parameter documentation, the description is incomplete. It doesn't explain what the tool returns (though an output schema exists, it's not referenced), doesn't clarify filter behavior, and offers no guidance on edge cases. For an agent to call this correctly, it must rely on external knowledge or trial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for documenting the 'filters' parameter, but it simply repeats 'matching the given filters' without explaining fields like gender, max_age, min_age, or subject_id. It adds no meaning beyond the schema's structural definition, and the nested object's semantics are entirely unaddressed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Find') and the resource ('patients in the MIMIC-IV Demo cohort'), and mentions 'matching the given filters' which gives a general sense of scope. It distinguishes itself from siblings (get_admissions, get_labs, aggregate) by the patient-centric focus, though it doesn't explicitly name alternatives or contrast them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus the sibling tools, no mention of typical use cases, and no exclusions or prerequisites. The description only states what it does, not when to prefer it over alternatives. An agent would have to infer usage from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• aggregateA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only states the aggregation operation and a constraint on group_by's table membership. It does not mention read-only nature, any side effects, response format, or error conditions. The lack of behavioral details beyond the core operation is a significant gap for a tool with no annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action and followed by a compact enumeration of valid options and a constraint. There is no redundancy or filler. It is efficient and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While an output schema exists, the description omits the 'gender' parameter entirely and does not clarify the mapping between metrics and the two tables (patients vs. admissions) beyond the vague 'same table' note. This leaves an agent unsure which metrics are valid for which table and what the gender field does, making the description incomplete for reliable invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description lists allowed values for metric and group_by, which duplicate the schema enums, but adds the meaningful constraint that both must belong to the same table. However, it completely ignores the 'gender' parameter present in the schema, leaving its purpose unexplained. Since schema description coverage is 0%, the description does not adequately convey the semantics of all parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with a clear verb ('Aggregate') and resource ('patients or admissions'), and immediately specifies the fixed metrics and allowed grouping columns. This is distinct from siblings like find_patients and get_admissions, which suggest raw data retrieval. Even without explicit sibling naming, the purpose is unambiguous and specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for aggregation tasks ('Aggregate patients or admissions') but does not explicitly state when to choose this tool over siblings, nor does it mention when not to use it. There is no guidance on alternatives or scenarios where raw data would be more appropriate. The implied usage is present but not elaborated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_admissionsA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. The verb 'Get' implies a read operation, but the description does not disclose any behavioral traits beyond that—no mention of pagination, return format, edge cases (e.g., empty results), or potential side effects. It is a simple read, but the disclosure is minimal and does not add value beyond the tool name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence. The core action and object ('Get every hospital admission') are front-loaded, followed by the scoping parameter. There is no fluff or redundant phrasing, making it highly concise and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter read tool with an available output schema, the description is reasonably complete: it states what the tool does and which parameter drives it. It does not cover usage nuances or alternatives, but the simplicity of the tool and the presence of an output schema mitigate the need for more elaborate context. A small gap remains in not guiding when to choose this over siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. The description clarifies that subject_id is the patient identifier ('by subject_id'), which adds meaning beyond the bare integer type in the schema. However, it provides no additional constraints or format details (e.g., positive integer, required), so the compensation is partial. For a single-parameter tool, this is a baseline level of semantic clarification.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states a specific verb ('Get'), a specific resource ('every hospital admission'), a clear scope ('for one patient'), and the exact parameter ('by subject_id'). It clearly distinguishes from sibling tools like find_patients (patient demographics) and get_labs (labs), leaving no ambiguity about what this tool returns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: use this when you need admissions for a patient. However, it gives no explicit context about when to prefer this over siblings or any exclusions (e.g., 'for patient demographics use find_patients'). The guidance is minimal, relying on inference rather than direct instruction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_labsA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral burden. It discloses a non-obvious behavior: label resolution against d_labitems can yield multiple candidates, and the error will list them, enabling a retry. This adds genuine value beyond a generic description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, with the core purpose front-loaded. The second sentence adds a valuable behavioral detail without excess, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core purpose and the label resolution nuance, but given four parameters and no schema descriptions, it fails to explain the role of subject_id and the exact usage of window_start and window_end. An output schema exists, so return values need no description, but parameter clarity is lacking.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'label' and 'windowed by date' but never names or explains subject_id, window_start, or window_end. An agent cannot infer which parameter maps to the patient or how the window parameters are formatted, reducing the description's utility for parameter construction.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Get'), a precise resource ('lab results for one patient and one lab test'), and the optional date window. This clearly differentiates it from sibling tools like find_patients, get_admissions, and aggregate, which focus on other aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance is provided. The purpose is self-evident for lab results, but the description does not mention alternatives or exclusions, leaving the agent to infer appropriate usage from the sibling names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.