Server Quality Checklist

Tool Scores

• query_problem_listC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and description does not disclose read-only nature, auth requirements, or any behavioral traits beyond the basic function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, but too brief; key information is missing, achieving brevity at the expense of completeness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no additional details about return format or scope, description is insufficient for an agent to understand the tool's full capabilities.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, and description adds no explanation of the two parameters; agent knows only from schema names which may be ambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns active problems for a patient, but does not differentiate from sibling tools like get_patient_overview which may also include problems.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Lacks any guidance on when to use this tool versus alternatives; no context or exclusions provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_clinical_notesC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description should disclose behavioral traits. It only mentions 'full-text search' but does not describe return format, pagination, ordering, or side effects. The read-only nature is implied but not stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise (one sentence), but at the cost of completeness. It is front-loaded, but lacks critical details that would help an agent use the tool correctly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 3 parameters with no schema descriptions and no output schema, the description is insufficient. It does not explain what 'flat SOAP collection' means, search algorithm, or result structure, leaving agents underinformed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds no meaning beyond parameter names; for example, it does not explain that 'query' is the search term, 'clinic_id' filters by clinic, or 'limit' controls result count.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs full-text search over the flat SOAP collection. It identifies the resource and action, but does not explicitly differentiate from siblings like get_soap_history or query_problem_list, though the function is distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. For example, it does not indicate whether to use this for free-text search vs get_soap_history for structured data or query_problem_list for problem-specific queries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_patient_overviewC
  2.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description implies a read-only, non-destructive operation (aggregating history and events), which is a key behavioral trait. However, it does not disclose any side effects, authentication requirements, or rate limits. It is adequate but not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single phrase with no structure. It is concise but under-specifies the tool, missing critical details. It could be expanded to include parameter hints or result expectations without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of output schema, annotations, and parameter descriptions, the description should compensate by explaining the tool's output and behavior. It only gives a vague summary, leaving the agent uncertain about what 'overview' entails. The sibling list helps but is not leveraged.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain the purpose or usage of 'patient_id' and 'clinic_id' beyond their names and types. This leaves the agent without guidance on how to fill these parameters correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states it provides an 'aggregated patient history and recent events,' which clearly indicates a read operation for a patient overview. This distinguishes it from siblings like get_soap_history (specific clinical notes) or query_problem_list (problem list), but lacks specificity on what is aggregated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives like search_clinical_notes or get_soap_history. The description does not mention any prerequisites, context, or exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• write_soap_draftC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist; description only says 'from AI-generated content' but fails to disclose mutation behavior, idempotency, permissions, or effects on existing data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence is concise but under-informative; no wasted words but fails to provide necessary detail for such a complex tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool has 3 required parameters including a nested object with 5 fields, no output schema, and no parameter descriptions; agent cannot understand how to construct valid input from this definition.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  0% schema description coverage; description adds zero meaning to any parameter (exam_id, soap, clinic_id) beyond their JSON structure. No explanation of what each field represents.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses specific verb 'create' and resource 'draft SOAP note', clearly distinguishing from sibling tools (get_patient_overview, get_soap_history, query_problem_list, search_clinical_notes) which are read-oriented.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs alternatives, no prerequisites or when-not-to-use conditions provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_soap_historyB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full responsibility for behavioral transparency. It only states what the tool returns, but omits any details on read-only nature, authentication needs, rate limits, or effects of invoking the tool (e.g., no side effects mentioned). This is insufficient for an agent to understand safe usage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 10 words, which is concise. However, it is too brief given the complexity of the tool (5 parameters, no annotations, no output schema), sacrificing crucial information for brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters, no annotations, and no output schema, the description is incomplete. It does not explain return value structure, pagination, error handling, or required fields like clinic_id. An agent would need additional context to use the tool accurately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions 'patient/problem' and 'date range', which map to patient_id, problem_id, start_date, and end_date parameters, adding meaning beyond the schema. However, clinic_id is not mentioned, and the schema coverage is 0%, so the description only partially compensates for missing parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool returns SOAP notes for a patient/problem within a date range. The verb 'Return' and resource 'SOAP notes' are specific, and the context distinguishes it from sibling tools like search_clinical_notes or write_soap_draft.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving SOAP history by patient/problem and date range, but it does not provide explicit when-to-use or when-not-to-use guidance, nor does it reference sibling tools or alternatives. Context is implied but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.