Server Quality Checklist

Tool Scores

• search_knowledgeA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the behavioral burden. It states the operation and scope, which implies a read-only search, but it does not disclose result behavior, ranking, limitations, or side effects. The cosine-similarity mechanism appears in the parameter schema, not in the tool description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with no filler. It states the action, resource, and content scope efficiently, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parametere search tool, the definition is largely complete: the input schema is fully documented and an output schema exists, so return-value details need not be spelled out. The main gap is the lack of explicit usage guidance, but the resource/scope phrasing partially compensates.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents query_text as a natural-language question embedded and matched by cosine similarity. The tool description adds no parameter-level meaning beyond that, but none is needed given the schema covers it fully.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Search'), identifies the resource ('internal knowledge base'), and narrows the scope ('playbook and policy context'). This makes the tool's purpose clear and distinct from siblings like lookup_lead, check_crm, and score_lead, which target lead/CRM/scoring data rather than knowledge-base content.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended context is implied: use when you need internal playbook or policy context. There is no explicit when-to-use/when-not-to-use guidance and no named alternatives, so an agent must infer selection from the resource type and sibling names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_leadA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden. 'Look up' implies a read-only operation and the description discloses the returned firmographics plus the 'call first' behavior, but it does not explicitly state there are no side effects or describe not-found/error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no filler. The core action is front-loaded, and the critical 'call this first' guidance follows immediately. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple one-parameter lookup with a full output schema. The description names the input, the expected return concept, and the proper call order. Nothing essential is missing for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the description does not need to add parameter details. It does echo 'by email address' and the schema adds exact-match semantics. The description does not add meaningfully beyond the schema, so baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear action ('Look up a lead by email address'), a specific resource (lead), and a scoping detail (company firmographics). It also distinguishes itself as the entry point for qualifying, making it unambiguous against sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit usage guidance: 'Call this first' and explains that other tools consume its return value. It does not name specific sibling alternatives or state when not to use it, but the sequencing instruction is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• score_leadA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It explicitly states that scoring is rule-based, deterministic, that the same lead always yields the same score, and that every point is attributed to 'reasons'. This is strong behavioral context, though it does not address side effects or permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tightly written sentences: the first names the operation and range, the second explains the deterministic and explainable behavior. There is no filler, repetition, or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with a complete input schema and an output schema present, the description adequately explains what the tool does and how the score behaves. Nothing essential is missing for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description itself adds no parameter-level meaning, but schema description coverage is 100% and the lead_id parameter is thoroughly documented: it must be lookup_lead's numeric 'id', not an email or company id, and invalid ids read as not found. Baseline 3 applies because the schema already handles parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Score a lead 0-100' against deterministic qualification rules. It is clearly distinct from the sibling tools, which are about lookup, CRM checks, and knowledge search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the obvious use case—scoring a lead—but does not explicitly say when to prefer this tool over check_crm or search_knowledge, nor does it state when not to use it. The lead_id schema adds a helpful 'call lookup_lead first' cue, but that guidance lives outside the tool description itself.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_crmA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full disclosure burden. It goes beyond a basic summary by revealing exact case-sensitive matching, the indistinguishable {'found': False} failure mode, and instructions to never normalize or guess the name. It does not describe broader return details or explicitly state read-only behavior, but 'Look up' and the output schema partially cover that.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The main description is a concise two-clause sentence with no filler. The parameter description is longer but every sentence earns its place by documenting matching rules, failure behavior, and usage constraints. The critical instruction about using lookup_lead is front-loaded in the parameter documentation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with an output schema, the description is complete. It explains what data is returned conceptually, how to source and format the parameter, what happens on a mismatch, and what failure looks like. No critical operational detail needed to call the tool correctly is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 100%, the description adds substantial meaning: the value must be the exact company.name from lookup_lead, matching is case-sensitive, common invalid forms like 'acme robotics' or 'Acme' are matched exactly and case-sensitively against CRM', and 'a ghost contain' will return {'found': False}' and is indistinguishable from no record. It also says to pass a name, not an id. This is far beyond the schema's basic string declaration.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Look up') and resource ('a company's CRM history') and names the concrete data areas ('deals and support tickets'). It clearly distinguishes check_crm from siblings like lookup_lead, score_lead, and search_knowledge by scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: use this to retrieve CRM history. The parameter description adds strong operational guidance, explicitly requiring the company name to come verbatim from lookup_lead and forbidding guessing or deriving from email domains. It does not explicitly state when not to use it versus specific alternatives, but the purpose and sibling names make the intended use clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.