Server Quality Checklist

Tool Scores

• appointment_schedulerC
  2.7/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations declare readOnlyHint=true, but the description says 'appointment scheduling' and the schema includes schedule and cancel actions, implying mutation. This directly contradicts the annotation, making the behavioral transparency score 1.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with two short sentences and no waste. However, 'AI-powered' is vague filler and the structure doesn't front-load key details like the tool's multiple actions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite full schema descriptions, the tool description omits the list and cancel actions and fails to explain behavior around API key requirements or response format. The contradiction with annotations further reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter having a clear description. The tool description adds no semantic meaning beyond that, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool schedules appointments, which is a specific verb and resource. It does not explicitly differentiate from sibling tools but the name and description sufficiently convey its core function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description requires a DINGDAWG_API_KEY but provides no context on when to use this tool versus alternatives like patient_intake or hipaa_check. No exclusions or situational guidance are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hipaa_checkA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint annotation, the description discloses that the basic check runs locally and that a deeper audit requires an API key. This adds meaningful behavioral context without contradicting the annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences front-load the tool's purpose and distinguish between free and API-key modes. No filler or redundant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains the return value at a high level and mentions the two modes, but lacks detail on how the API key is supplied given no such parameter exists in the schema. With no output schema, more explicit return or invocation details would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema documents both parameters with full coverage, so the description does not need to add parameter syntax. The description's mention of 'processes and systems' loosely maps to process_description but adds little beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs a HIPAA compliance quick check for processes and systems and returns basic compliance gaps. It also differentiates from the clinical/patient-focused sibling tools by its compliance domain.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides context for when to use the free quick check versus the deeper LLM-powered audit with an API key. No explicit exclusions or named alternatives, but the usage context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• patient_intakeA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations include readOnlyHint:true, and the description adds that it returns structured intake locally and requires an API key for deep ICD-10 coding and clinical reasoning. This provides useful extra behavioral context beyond the annotation, such as local processing and key requirement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two concise, front-loaded sentences. Every phrase adds value, including the free aspect, local processing, and deep LLM features, with no redundant words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description partially compensates by listing triage priority, symptom analysis, and ICD-10 coding as outputs. It lacks details on input prerequisites or error handling, but is reasonably complete for a tool of this simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description mentions symptoms for analysis but doesn't add meaning beyond the schema's parameter descriptions, such as age, gender, or medical_history specifics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as an AI patient intake form performing triage priority and symptom analysis, with additional ICD-10 coding. This specific verb+resource combination distinguishes it from sibling tools like clinical_summary and appointment_scheduler.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for patient intake and symptom analysis, setting clear context. However, it does not explicitly state when not to use or mention alternatives, so it falls short of full usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• clinical_summaryA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint annotation, which indicates a safe read operation, the description discloses an important external dependency: the DINGDAWG_API_KEY for LLM-powered summarization. It also hints at the token/cost implications of using an LLM, which is valuable behavioral context that annotations do not provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no wasted words. The first sentence states the action and the second provides the key prerequisite and output components, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description carries the burden of explaining the return value. It lists the extracted fields, which gives a good sense of the output structure. However, it doesn't clarify the difference between the 'soap', 'structured', and 'brief' formats, and doesn't mention potential errors if the API key is missing, leaving slight gaps for such a complex operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers both parameters with descriptions (100% coverage), so baseline is 3. The description adds value by explaining that the output extracts chief complaint, HPI, assessment, and plan, which clarifies what the 'format' parameter's 'structured' option entails and what the tool does with the clinical_notes parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with a specific verb ('Summarize') and resource ('clinical notes'), and clearly states the goal ('into structured format'). It further specifies the key extracted elements (chief complaint, HPI, assessment, plan), which distinguishes this tool from siblings like patient_intake or appointment_scheduler.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool (to summarize clinical notes) and mentions a critical prerequisite (DINGDAWG_API_KEY). It does not explicitly name alternatives or state when not to use it, but the sibling tools are sufficiently different in purpose that no exclusions are necessary.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.