Server Quality Checklist

Tool Scores

• list_applicationsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that status matching is case-sensitive and exact, that returned fields depend on a configurable schema, and clarifies which statuses are server-driven versus manual edits. It explains what gets returned (job_id plus tool-populated fields) and the read-only nature of the operation, providing good behavioral context beyond the minimal schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence, then parameter details, then return-value context. It's reasonably compact for the information conveyed, though the return-fields paragraph is slightly verbose with the documentation reference. No wasted sentences; each paragraph earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-optional-parameter read tool with no annotations, the description covers purpose, filter semantics, return shape (with reference to a configurable schema), and data provenance. An output schema exists to document return structure. It's complete enough for an agent to invoke confidently, though it could note error cases or pagination behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must carry all parameter meaning. It thoroughly explains the 'status' parameter: exact-match semantics, case-sensitivity, example values, and behavior when omitted. The key insight that filtering requires exact tracking-store values is genuinely useful and not derivable from the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'lists tracked jobs from the configured tracking store (Notion)', specifying the verb (list), resource (tracked jobs), and source (Notion tracking store). It distinguishes from siblings by clarifying it reads tracked applications rather than matching or pushing jobs, though it doesn't explicitly name the sibling alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: the optional 'status' filter for exact matches, instructions to omit it to return all rows, and a warning that certain statuses are manual edits made in Notion rather than server-driven. It gives practical when-to-use guidance via the filter semantics, though it doesn't explicitly state when to use a sibling tool instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• match_jobA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It does disclose a key behavioral trait: the tool returns raw retrieval scores and resumes chunks WITHOUT synthesizing strengths/gaps/notes, explicitly telling the agent to reason over the evidence itself. This is valuable transparency about the tool's computational behavior (heuristic cosine similarity) and its non-analytical nature. It doesn't cover auth/permissions or failure modes, but for a retrieval-style tool the core disclosure is present and helpful.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections: purpose sentence, Args list, and Returns paragraph. It's appropriately detailed for a retrieval tool and front-loads the purpose. There is a minor dependency on the job-fit://rubric resource which isn't fully elaborated, and the Args/Returns formatting is somewhat technical, but overall it's concise with no wasted sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With only 2 parameters (1 required), no enums, and an output schema present, the tool is relatively simple. The description explains what the tool returns (retrieval_score, chunks, similarities) and, crucially, what it does NOT do (synthesize strengths/gaps/notes), directing the agent to the job-fit://rubric. The main gap is that the rubric resource is referenced vaguely without stating what it contains, but the output schema and low parameter count reduce the burden on the description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, meaning the schema provides only type names (job_description, source_url) with titles but no descriptive help. The description compensates meaningfully: it explains job_description is 'the full text of the job description to match against' and source_url is 'optional URL the job description was pulled from, for reference.' However, it adds no format details for job_description (e.g., expected length, formatting) or when source_url might matter. This is adequate compensation for low coverage but not exhaustive.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource statement: 'Retrieve resume/experience evidence relevant to a job description.' This clearly distinguishes it from its siblings (push_to_tracker, list_applications), which are evident from context to be write/listing operations. The purpose is unambiguous and specifically tied to retrieval matching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when to use this tool: when you need evidence retrieval against a job description for fit analysis. It explicitly states what the tool does NOT do ('Does not synthesize strengths/gaps/notes') and instructs the agent to reason over retrieved evidence applying the job-fit://rubric resource. This is strong guidance about how to use the output. However, it doesn't explicitly contrast with sibling tools (e.g., when NOT to use this in favor of list_applications), which would push it to a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• push_to_trackerA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden and it delivers extensively. It discloses that only tool-populated fields (per tracking_schema.yaml) are touched while manual fields are left intact, that misconfigured fields are skipped rather than failing the whole write, that warnings appear in the result, and that dry_run makes no Notion API call so its warning coverage is limited. This is exceptionally transparent about side effects and failure modes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense and well-organized with Args, behavior, and edge-case handling clearly sectioned. It's longer than minimal but every sentence adds operational value — dry_run semantics, field-preservation behavior, and warning behavior are all genuinely useful. Slightly long but front-loaded with the core operation first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a write tool with no annotations and no output schema, the description is thorough: it covers the write target, in-place update semantics, field-selection behavior driven by tracking_schema.yaml, error handling for misconfigured fields, and dry_run's limitations. It would benefit from describing the result structure (what warnings/properties are returned) since there's no output schema, but it's largely complete for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, and it does. It explains job_id (Notion page ID of an existing tracked job), verdict (full evaluate_fit output, referencing docs), and dry_run (preview mapped payload without writing). The only minor gap: it references external docs (evaluate_fit_schema.md, adr/0011) rather than describing the verdict structure inline, though reasonable given complexity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Write'), the resource ('an evaluate_fit result to the configured tracking store (Notion)'), and the key behavior — updating an existing row in place, never creating a new one. This distinguishes it effectively from sibling tools like match_job (which produces the fit result) and list_applications (which reads applications).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly explains when to use this tool (after evaluate_fit produces a result, to persist it) and gives critical usage context: job_id must be an already-tracked Notion page ID, whether to use dry_run to preview mapping before writing, and how misconfigured fields are handled. It distinguishes from siblings by positioning this as the write step following the match/evaluate step.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.