Server Quality Checklist

Tool Scores

• get_grant_detailC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves 'full detail' but doesn't specify what that includes (e.g., award amount, dates, investigators), whether it's a read-only operation, potential rate limits, or error handling. The description is minimal and lacks essential behavioral context for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste—it directly states the action, resource, and sources. It's appropriately sized for a simple retrieval tool and front-loaded with the core purpose. Every word earns its place without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (2 parameters, no annotations, no output schema), the description is incomplete. It lacks details on what 'full detail' entails, behavioral traits like read-only nature or error conditions, and doesn't compensate for the absence of an output schema. For a retrieval tool with no structured support, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate by explaining parameters. It mentions 'by ID' and sources (NSF, ERC, KRF), which aligns with 'grant_id' and 'source' parameters, but doesn't clarify the format of 'grant_id' (e.g., numeric, alphanumeric) or the meaning of the enum values beyond listing them. The description adds some meaning but insufficiently compensates for the low schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and resource 'full detail of a specific grant by ID', specifying the sources (NSF, ERC, KRF). It distinguishes from sibling tools like 'list_upcoming_deadlines' and 'search_grants' by focusing on retrieving detailed information for a single grant rather than listing or searching multiple grants. However, it doesn't explicitly contrast with siblings in the description text itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search_grants' or 'list_upcoming_deadlines'. It mentions the sources but doesn't explain scenarios where this tool is preferred, such as when you have a specific grant ID versus needing to browse or filter grants. No exclusions or prerequisites are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_upcoming_deadlinesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It describes a read operation ('List') but doesn't cover critical aspects like whether it requires authentication, rate limits, pagination, error handling, or the format of returned data. For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose without unnecessary details. Every word contributes to understanding the tool's function, making it appropriately sized and well-structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of listing deadlines across multiple programs, with no annotations and no output schema, the description is incomplete. It lacks information on behavioral traits, parameter details, and expected return values, leaving the agent with insufficient context to use the tool effectively beyond basic intent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the schema provides no parameter descriptions. The description adds some semantic context by mentioning 'within the next N days' (mapping to 'days_ahead') and 'NSF, ERC, KRF funding programs' (mapping to 'source'), but it doesn't fully explain the parameters' roles, defaults, or enum values. It compensates partially but not completely for the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'List upcoming grant application deadlines within the next N days across NSF, ERC, KRF funding programs.' It specifies the verb ('List'), resource ('grant application deadlines'), and scope ('within the next N days across NSF, ERC, KRF funding programs'). However, it doesn't explicitly differentiate from sibling tools like 'get_grant_detail' or 'search_grants', which might also involve grants but serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings ('get_grant_detail' and 'search_grants'). It implies usage for listing deadlines within a time frame and specific programs, but lacks explicit instructions on alternatives, exclusions, or prerequisites. This leaves the agent to infer usage without clear direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_grantsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the search functionality and return fields but doesn't disclose important behavioral traits: whether this is a read-only operation, if there are rate limits or authentication requirements, how results are ordered, if pagination is supported beyond the 'limit' parameter, or what happens when no results are found. For a search tool with 5 parameters and no annotation coverage, this leaves significant gaps in understanding how the tool behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that efficiently conveys the tool's purpose, scope, search criteria, and return information. It's appropriately sized for a search tool and front-loaded with key information. There's no wasted text, though it could potentially be split into two sentences for even clearer structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, no output schema, no annotations), the description is incomplete. It covers the basic purpose and some parameter context but misses behavioral transparency, full parameter explanations, and output details. For a search tool that returns grant information, the description should ideally mention result format, ordering, or error handling to be complete enough for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions searching 'by keyword, PI name, or institution' which partially explains the 'query' parameter, and lists data sources (NSF, ERC, KRF/NRF) which maps to the 'source' enum. However, it doesn't explain 'year_from', 'year_to', or 'limit' parameters at all. With 5 parameters and only partial coverage of 2, the description adds some meaning but doesn't adequately compensate for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches research grants across specific funding agencies (NSF, ERC, KRF/NRF) using various criteria (keyword, PI name, institution) and returns specific grant information (titles, award amounts, PIs, institution info). It distinguishes from 'get_grant_detail' (which likely retrieves details for a specific grant) and 'list_upcoming_deadlines' (which focuses on deadlines rather than searching existing grants). However, it doesn't explicitly mention how it differs from 'get_grant_detail' beyond the search functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by listing searchable criteria and data sources, suggesting this tool is for finding grants across multiple agencies. However, it doesn't explicitly state when to use this versus 'get_grant_detail' (e.g., 'use this to find grants matching criteria, then use get_grant_detail for detailed information on a specific grant') or 'list_upcoming_deadlines' (e.g., 'use this for active grants, use list_upcoming_deadlines for future opportunities'). The guidance is present but not explicit about alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.