Server Quality Checklist

Tool Scores

• send_scheduling_emailA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description explicitly states it 'opens' the client rather than sending, which is a useful behavioral trait. However, with no annotations, it doesn't disclose whether sending requires user confirmation, what happens if no mail client is configured, or any security/side-effect considerations beyond opening a program.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence, front-loaded with the action ('Opens the default mail client'), no filler, and every word contributes meaningful information. It is appropriately sized for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with no output schema, the description is mostly complete: it states the action, the content, and the recipient. It loses a point because it doesn't mention the side effect beyond opening the client (e.g., whether this is a draft or automatically sent), and lacks any guidance on when to use it, but overall it's adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters and an empty input schema, the description is the only source of parameter-related meaning. It explains that the email will be 'pre-populated with a scheduling request' and directed 'to contact Matt,' providing useful semantic context that the schema cannot.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Opens') and resource ('default mail client pre-populated with a scheduling request'), clearly distinguishing it from sibling query tools like query_resume_history and get_company_deep_dive. It also names the intended recipient ('contact Matt'), making the purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives, and no exclusions or prerequisites are mentioned. The sibling tools are data-retrieval operations, but the description doesn't explicitly address selection criteria or context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_company_deep_diveA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool 'Fetches deeper details' and gives examples of the content, implying a read-only operation. However, it does not describe the return format, pagination, or any other behavioral constraints, leaving some ambiguity for a fetch tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the core action and resource, with examples adding valuable specificity. No wasted words or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one enum parameter and no output schema, the description is mostly complete: it states the purpose, scope, and typical use cases. It could be more explicit about the returned data's structure, but the examples partially compensate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%: the only parameter 'company' is fully described with an enum and explanation. The description adds no extra parameter-level detail beyond the tool's overall purpose, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the verb 'Fetches' and the resource 'deeper details on specific companies Matt has worked at', with concrete examples (Nomad Go scaling details, TUNE Payout Structures) that clarify scope. This clearly distinguishes it from sibling tools like query_resume_history, which is for general history.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool: when specifics on one of Matt's past companies are needed, as opposed to general history (query_resume_history) or scheduling (send_scheduling_email). It does not explicitly mention alternatives or exclusions, but the 'specific companies' phrasing offers clear context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_resume_historyA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses key behavioral traits: 'Fast, deterministic substring matching' and 'exact keyword or phrase.' This goes beyond the minimal purpose statement but doesn't detail return format or edge cases like case sensitivity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, with the purpose front-loaded and the alternative mentioned succinctly. Every sentence earns its place with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The simple tool has no output schema, and the description covers purpose, behavior, and alternative usage. It does not describe the return format or pagination, but these are less critical for a straightforward search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully describes the query parameter with 100% coverage, so the baseline is 3. The description adds meaning by emphasizing 'exact keyword or phrase' and 'substring matching,' which clarifies how the parameter should be used.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches Matt's work history, role bullet points, and experience highlights for an exact keyword or phrase. It also distinguishes itself from semantic_search_history, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to use this tool for exact keyword or phrase matching and recommends semantic_search_history for conceptual or paraphrased queries. This provides clear when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.