Server Quality Checklist

Tool Scores

• get_bill_planC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The description says it 'gets' details but doesn't state the return format, whether it's read-only, failure modes when the contract line doesn't exist, or any pagination/limitations. For a retrieval tool with zero annotation coverage, it should disclose more about behavior beyond the mere action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and has a clear structure with param docs laid out in a docstring format. It's front-loaded with the purpose statement. The param descriptions are very short but readable. No fluff or wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema, no annotations, and three undocumented parameters (0% schema coverage), the description is thin. It doesn't describe the returned bill plan data shape, how it relates to billing setup tools (check_project_billing_setup), or edge cases. Given the tool sits among richer billing-related siblings, more context would help an agent differentiate and use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the undocumented parameters. While it gives terse one-line hints for each param (e.g., 'Business unit code (e.g., 'UCD', 'UCB')'), these are minimal and don't add meaningful formatting or semantic detail beyond what the parameter names already imply. The contract_num and contract_line_num hints are near-tautological.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb+resource: 'Get the bill plan details for a specific contract line.' This clearly identifies the resource scope (a contract line) and distinguishes it from siblings like list_contracts or get_project_contracts. It loses a point because it doesn't clarify how this differs from the general query_peoplesoft_db or list_contracts tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool vs alternatives like query_peoplesoft_db or list_contracts. The description provides no context about why one would choose this specific tool over the broader database query tool or other sibling tools. No exclusions or alternative references are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_project_billing_setupB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden for behavioral disclosure. The description indicates this is a read/audit operation and states the return type (list of rows with effective statuses), but it doesn't disclose whether all parameters are ANDed or ORed, whether omitting all filters returns everything, performance implications of cross-referencing multiple tables, or any required permissions. Given zero annotation coverage, this is thin.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The multiline structure with a clear opening statement, a uses section, and a params/returns section is well-organized and reasonably concise. Every sentence adds value - the examples in the uses section are particularly helpful. It could be slightly tighter but is well-structured for its content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a moderately complex cross-referencing tool (projects × contracts × bill plans × statuses) with no output schema and no annotations. The description covers the input semantics and return shape at a high level, but given the tool's complexity and the presence of sibling tools with overlapping concerns (get_bill_plan, get_project_contracts), it doesn't fully clarify how results aggregate or how filters narrow the result set.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It lists the three parameters with brief inline examples ('UCD', 'UCB' for business_unit) but doesn't explain how filters compose (are they combined with AND?), whether they're mutually exclusive, or how to specify a project (ID format). The parameter names themselves are self-explanatory, but the description adds minimal meaning beyond their names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks how projects are set up for billing, enumerating the specific elements examined (contract lines, bill plan, bill plan type, effective statuses). It's specific about the resource (projects' billing setup) and gives concrete examples for what it's used for, though it doesn't explicitly distinguish from siblings like get_bill_plan or get_project_contracts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit when-to-use purpose: 'Use this to spot setup problems' with concrete examples (active project tied to inactive bill plan, contract line with no bill plan). It doesn't explicitly state when NOT to use alternatives, but the diagnostic/audit framing differentiates it from the more targeted sibling tools like get_bill_plan.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_projectsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The description doesn't mention whether this is a read-only operation (implies so but doesn't state it), authentication requirements, pagination behavior, result limits, or what the return format looks like beyond 'List of matching projects'. For a listing tool that likely queries a database, there's no disclosure about data freshness, access controls, or error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is reasonably concise with a one-line summary followed by compact parameter documentation. Each parameter has a focused one-line entry. However, the return statement is thin, and the docstring-style formatting could be tightened. Slight redundancy between the description line and the parameter list could be trimmed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no annotations, no output schema, and 0% schema coverage, the description needs to be comprehensive. It covers the core purpose and parameters adequately but is thin on return value details, call semantics, and edge cases. Given the sibling tools (query_peoplesoft_db, describe_table) suggest this is a database-backed tool, more behavioral context would be expected for full completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, meaning the description is the only source of parameter documentation. The description does explain each parameter inline (business_unit code examples, eff_status meaning of 'A'/'I', project_type as a code). However, project_type lacks concrete examples and there's no guidance on valid values or format expectations beyond the brief inline descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists projects with optional filtering by business unit, effective status, or project type. It uses a specific verb+resource pattern and distinguishes itself from siblings like list_contracts and list_tables. However, it doesn't explicitly differentiate from get_project_contracts which might overlap in purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context ('List projects, optionally filtered') but doesn't explicitly state when to use this tool versus alternatives like get_project_contracts or query_peoplesoft_db. It explains what filters are available but doesn't provide guidance on when this tool is preferable to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_contractsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions each line includes bill plan type/status, which is helpful, but doesn't disclose aspects like whether unfiltered calls return large result sets, pagination behavior, read-only safety, or field presentation differences from query_peoplesoft_db. For a database-facing listing tool with zero annotations, more behavioral context is expected.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description uses a leading one-sentence summary followed by per-parameter doc lines, which is a clean, front-loaded structure. There is minimal redundancy between the summary and the parameter list. It earns a 4 rather than 5 because the docstring-style parameter layout is slightly verbose compared to a compact inline style.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 3 optional parameters, no output schema, and zero annotations, so the description carries substantial responsibility. It covers all parameters with examples and returns basic return info ('List of matching contract lines with bill plan info'). However, it doesn't mention result ordering, potential large result sets for unfiltered calls, or how this interacts with sibling query tools like query_peoplesoft_db. Adequate but with room to strengthen.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must fully compensate. It does so well: each parameter is described with its semantics (business unit code, contract number, effective status) and enum-like value meanings are explained ('A' = Active, 'I' = Inactive). The description also adds the important detail that each line includes bill plan type/status, going beyond what the schema conveys.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List contract detail lines' and enumerates the optional filters (business unit, contract number, effective status). It distinguishes the tool's scope from siblings that deal with projects and bill plans, though it doesn't explicitly name alternatives. The verb 'list' with specific resource 'contract detail lines' plus filter options gives solid purpose clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is a filtered listing tool by naming each filter parameter and providing example values ('UCD', 'UCB', 'A' = Active, 'I' = Inactive). However, it doesn't explicitly state when to prefer this over siblings like get_project_contracts or query_peoplesoft_db, nor does it note whether it complements those tools for different use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_peoplesoft_dbB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It does convey the query mechanics (SQLite '?' placeholders not Oracle-style), which is useful. However, it gives no indication of whether this is a read-only operation, whether it modifies data, what the return structure looks like, or error handling behavior. It also names helper functions (list_tables, describe_table) as if they're usable within this tool, which could be confusing about actual callable behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is reasonably structured with numbered steps and clear sections (AVAILABLE RESOURCES, then param docs). However, it's somewhat verbose—the AVAILABLE RESOURCES section largely repeats information from the sibling tool names already visible, and the helper-function references (list_tables(), describe_table()) are ambiguous since those are sibling tools, not necessarily callable functions. The front-loaded workflow section is the strongest part; the redundancy at the end is padding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a raw SQL query tool with no output schema and no annotations, the description handles the critical workflow (discover→describe→query) and SQLite-specific syntax well. However, it lacks detail on output format (return dictionary structure), error behavior, and safety profile. The mention of semantic alternative tools helps agents choose the right path but the behavioral gaps keep this from being fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does explain the sql_query parameter with a SQL example (SELECT * FROM PROJECT WHERE BUSINESS_UNIT = ?) and mentions the parameters parameter as 'List of query parameters (optional)'. However, this is minimal—it doesn't explain how the parameters list maps to the ? placeholders in detail, whether they support type conversion, or the expected format of the return dictionary. It adds some value but leaves significant gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool queries the local SQLite database directly with a specific verb+resource ('Query... database directly'). It doesn't fully distinguish from siblings like list_projects or get_project_contracts, but it frames itself as a raw/low-level access layer while the siblings are semantic accessors, which is reasonable differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Excellent usage guidance. The description provides a numbered workflow: discover tables first, describe structure second, then query. It explicitly names alternative tools (list_tables, describe_table, and the semantic accessors like list_projects) and frames the raw query tool as appropriate for low-level access, distinguishing when to use it versus siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_project_contractsA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosure. It states the return shape ('project record, contract detail lines, bill plan') and references internal table names (CA_DETAIL, CA_BILL_PLAN), providing useful behavioral context about what data relationships are traversed. It doesn't describe pagination or what happens when there are many bills, but for a retrieval tool the return-structure framing is a meaningful transparency contribution.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear opening sentence, a purpose paragraph, and a docstring-style parameter listing. It's appropriately sized — not verbose, each sentence adds context. The formatting is clean and scannable. Minor deduction for the docstring boilerplate not being stripped, but it's efficient overall.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description must convey the return structure, which it does (project record, contract lines, bill plans with table names). For a 2-param read tool with a clear scope, this is reasonably complete. It doesn't discuss error cases or edge conditions, but the core purpose and data relationships are sufficiently documented for agent selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. The docstring explains business_unit as 'Business unit code (e.g., 'UCD', 'UCB')' with concrete examples, and project_id as 'The project ID.' This adds some value beyond the bare schema (which has no descriptions), though 'The project ID' is thin and the two params are not deeply characterized. Basline for 0% coverage would be low, so the examples help, but there's limited enrichment.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb+resource: "Get a project along with all contract lines and bill plans tied to it." It names the specific tables (CA_DETAIL, CA_BILL_PLAN) and the broader goal (understanding full billing picture). However, it's not fully distinguished from siblings like get_bill_plan or list_contracts — the relationship to them is implied but not explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description identifies this as 'the primary tool for understanding a project's full billing picture,' which conveys a primary-use context. However, it doesn't explicitly state when NOT to use it or name alternative tools (e.g., when only a single bill plan is needed, use get_bill_plan). The guidance is implied through the 'primary tool' framing rather than explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tablesA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full behavioral disclosure burden. It states listing is read-only implicitly but doesn't explicitly confirm non-destructive behavior. It doesn't mention whether results are sorted, limited, whether the pattern matches against fully qualified names, or what happens with zero matches (empty list vs error). For a discovery tool, these details would help.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: a one-sentence purpose, a clear usage directive, and a concise param table. Every line earns its place with no fluff. Front-loaded with the core purpose statement before the guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple discovery tool with 1 optional parameter and no output schema, the description is quite complete. It covers purpose, when to use, the parameter semantics, and return type. The main gap is lack of explicit safety confirmation (read-only) and example usage, but for this tool's simplicity, the coverage is strong.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the single 'pattern' parameter. The description explains 'Optional substring to filter table names (case-insensitive)' which adds genuine meaning — declaring it case-insensitive and a substring match is valuable. This does add semantics beyond the bare schema (which only specifies type string/null and default null). A 3 is appropriate since it covers the only parameter well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool does: 'List tables available in the local database, optionally filtered by a name pattern.' The verb 'List' is specific, the resource 'tables' in the local database is explicit, and the optional pattern filtering is mentioned. It also distinguishes itself from siblings like query_peoplesoft_db (which queries data) and describe_table (which describes a table's structure).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to 'Use this tool FIRST to discover what tables exist before writing queries.' This is clear usage context for when to invoke it. However, it doesn't explicitly state when NOT to use it or name alternatives (e.g., describe_table for schema details), though the position is implied by sequencing guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_tableA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses the return type (list of columns with properties) which is helpful. However, it doesn't specify whether the operation is read-only, whether it requires schema access permissions, or how tables without descriptions are handled. Non-destructive nature is implied but not stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with a docstring format: clear purpose statement, usage context, and parameter documentation. Slightly verbose with the param and return annotations, but each sentence adds value. Compactly conveys the essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter introspection tool with no nested objects and no output schema, the description is reasonably complete. It explains what's returned (columns, types, primary key) and when to use it. Could benefit from noting what happens with invalid table names or a note that it's a metadata-only operation, but these are minor gaps for a tool of this simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It provides examples of valid table names ('PROJECT', 'CA_DETAIL', 'CA_BILL_PLAN'), giving concrete guidance. With only 1 parameter at low coverage, the examples help but don't fully replace schema documentation of the expected format or valid values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Describes exactly what it does: retrieves the structure of a table including all columns, types, and primary key membership. Clear verb+resource+details. Distinguishes from siblings like query_peoplesoft_db (executes queries) and list_tables (lists tables, not their structure).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  States when to use: 'before writing queries against it' to understand available fields. Provides clear context for use, though it doesn't explicitly name alternative tools for when NOT to use it. The guidance is actionable even if not exhaustive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cost_reimbursable_billable_transactionsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It clearly states this is a summarization (read/aggregate) operation returning summed amounts and date ranges, which is useful behavioral context. However, it doesn't disclose potential pagination/limits, whether zero-result projects are excluded, or any access/permission requirements. For a summarization tool that is inherently non-destructive, the described behavior is reasonably transparent, but it could specify more about filtering semantics and aggregate output details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded: purpose is stated in the first line, then the WHERE criteria, then the aggregation behavior, then the params doc. The parameter section is useful but somewhat verbose - the Python docstring formatting adds lines without much new info beyond the schema titles. It's efficient and free of fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, so the description must explain return values, and it does: 'Per-project sums of billable amounts with min/max accounting dates.' The filter criteria (ANALYSIS_TYPE='BIL', BI_DISTRIB_STATUS='N') are fully documented, and the four parameters are each described with types and formats (e.g., YYYY-MM-DD). For a 4-param summarization tool with no annotations or output schema, this is quite complete. Minor gaps: doesn't describe the exact output structure (field names) or whether it returns zero rows or excludes unmatched projects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description MUST compensate, and it does excellently. Each of the four parameters gets a clear explanation including their types, optionality, meaning (business unit code, project ID, accounting date bounds), and date format (inclusive, YYYY-MM-DD). The description adds meaning beyond the schema's bare null/default defaults, making all filter parameters actionable and self-documenting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening line is a specific verb+resource: 'Summarize undistributed cost-reimbursable billable transactions.' The first sentence precisely scopes what qualifies (PROJ_RESOURCE rows with ANALYSIS_TYPE='BIL' and BI_DISTRIB_STATUS='N') and what the output is (per-project summed BIL amounts and min/max accounting dates). This clearly distinguishes it from the sibling tools (e.g., get_bill_plan, list_contracts, check_project_billing_setup) which operate on different concepts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly explains what the tool does and the filter semantics, which implies when to use it (when you need undistributed billable transaction summaries by project). However, it provides no explicit exclusions or alternatives relative to siblings—there's no statement like 'for distributed transactions use X' or 'for detailed line items use Y.' Given the sibling set includes billing-related tools like get_bill_plan and check_project_billing_setup, explicit differentiation would help agents pick correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.