Server Quality Checklist

Tool Scores

• search_issuesC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'supports pagination', which is useful context, but lacks details on permissions, rate limits, error handling, or what the output looks like. For a search tool with 6 parameters and no annotations, this leaves significant gaps in understanding how it behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—two sentences with zero waste. It's front-loaded with the core purpose and efficiently lists key features. Every word earns its place, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (6 parameters, no annotations, no output schema), the description is incomplete. It doesn't cover parameter interactions, return format, error cases, or authentication needs. While concise, it fails to provide enough context for an agent to use the tool effectively without additional guesswork.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It lists filter types (project, status, assignee, keyword) and mentions pagination, which maps to some parameters (e.g., 'query', 'limit', 'offset'), but doesn't explain semantics like 'project_id' vs 'project', 'status_id' values, or how 'query' interacts with other filters. This adds minimal value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('search') and resource ('issues in Lychee Redmine'), making the purpose immediately understandable. It distinguishes itself from siblings like 'create_issue' or 'update_issue' by focusing on search functionality. However, it doesn't explicitly differentiate from other search-like tools (none exist in siblings), so it's not a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions filtering by project, status, assignee, or keyword, which implies when to use it, but provides no explicit guidance on when to choose this tool over alternatives like 'get_projects' or 'get_users'. There's no mention of prerequisites, exclusions, or specific scenarios where this tool is preferred.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_scheduleC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool returns data (start/end dates, progress, etc.) and mentions optional critical path analysis, but lacks details on permissions needed, rate limits, error conditions, or response format. For a read operation with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose and lists return data. There's no wasted text, but it could be slightly more structured (e.g., separating purpose from return details).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (returns multiple data types like milestones and dependencies), lack of annotations, and no output schema, the description is insufficient. It doesn't explain the return structure, error handling, or how to interpret the data (e.g., what 'progress' means). For a tool with rich output and zero structured context, more completeness is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the schema provides no parameter descriptions. The description mentions 'optional critical path analysis,' which implies the include_critical_path parameter, but doesn't explain project_id or detail what critical path analysis entails. It adds minimal semantic value beyond the bare schema, failing to fully compensate for the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get schedule information for a project in Lychee Redmine' with specific verb ('Get') and resource ('schedule information'), and distinguishes it from siblings like get_project or get_projects by focusing on schedule data. However, it doesn't explicitly differentiate from all siblings (e.g., search_issues might also return schedule-related data).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like get_project (which might include schedule info) or search_issues (which might filter schedule-related issues). It mentions the tool returns schedule information but doesn't specify prerequisites, exclusions, or comparative contexts with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_issueB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses that partial updates are supported, which is useful behavioral context. However, it doesn't mention authentication requirements, rate limits, error conditions, whether changes are reversible, or what the response looks like. For a mutation tool with zero annotation coverage, this leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise - just two sentences that directly convey the core functionality. Every word earns its place: first sentence establishes purpose, second adds crucial behavioral detail about partial updates. No wasted words or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a mutation tool with 7 parameters, 0% schema description coverage, no annotations, and no output schema, the description is insufficient. It covers the basic operation and partial update feature but lacks information about authentication, error handling, response format, parameter meanings, and relationships with sibling tools. The agent would struggle to use this tool effectively without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'only specify fields to change' which explains the partial update behavior, but doesn't provide any semantic information about individual parameters (id, subject, description, status_id, priority_id, assigned_to_id, due_date). The description doesn't clarify what these fields represent, their formats beyond schema constraints, or relationships between them.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Update'), target resource ('an existing issue in Lychee Redmine'), and scope ('Partial updates supported'). It distinguishes from 'create_issue' by specifying 'existing issue' and from 'search_issues' by focusing on modification rather than retrieval. However, it doesn't explicitly differentiate from other potential update operations that might exist.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for modifying existing issues ('Update an existing issue') and mentions partial updates as a feature. It doesn't provide explicit guidance on when to use this tool versus alternatives like 'create_issue' for new issues or 'search_issues' for finding issues. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_issueB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'Requires project_id and subject' which hints at prerequisites, but doesn't disclose critical behavioral traits like authentication requirements, rate limits, what happens on success/failure, or whether this is a destructive/mutative operation beyond the obvious 'Create' implication.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise - a single sentence that front-loads the core purpose and efficiently lists parameter requirements. Every word earns its place with zero redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with 6 parameters, 0% schema description coverage, no annotations, and no output schema, the description is inadequate. It doesn't explain what the tool returns, error conditions, side effects, or provide enough parameter semantics. The agent would struggle to use this tool correctly without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It lists all 6 parameters and indicates which are required vs. optional, adding meaningful context beyond the bare schema. However, it doesn't explain parameter meanings (e.g., what priority_id values represent, format expectations beyond due_date pattern) or provide examples, leaving gaps in semantic understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Create') and resource ('new issue in Lychee Redmine'), making the purpose immediately understandable. However, it doesn't explicitly distinguish this tool from its sibling 'update_issue' beyond the obvious create vs. update distinction, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides basic context by listing required and optional parameters, which implies when to use it (when you have project_id and subject). However, it doesn't explicitly state when NOT to use it or mention alternatives like 'update_issue' for modifying existing issues, leaving usage guidance incomplete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_projectB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the return values (project name, description, status, and members), which is helpful, but lacks details on error handling (e.g., what happens if the ID is invalid), authentication needs, rate limits, or whether it's a read-only operation. This leaves behavioral gaps for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the purpose and key details without unnecessary words. Every part earns its place by stating the action, resource, identifier, and return values clearly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no nested objects) but lack of annotations and output schema, the description is moderately complete. It covers the basic purpose and return values, but for a read operation with no structured output, it could benefit from more behavioral context like error scenarios or data freshness, leaving room for improvement.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter with 0% description coverage, so the description must compensate. It adds meaning by specifying 'by ID', clarifying that the 'id' parameter is used to identify a specific project, but doesn't provide further details like format examples or constraints beyond the schema's 'exclusiveMinimum'. This partially compensates but doesn't fully address the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get project details'), resource ('from Lychee Redmine'), and scope ('by ID'), distinguishing it from sibling tools like 'get_projects' (plural) and 'get_project_members'. However, it doesn't explicitly contrast with all siblings, such as 'get_schedule' or 'get_users', which slightly limits differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying 'by ID', suggesting it's for retrieving a single project when the ID is known, but it doesn't explicitly state when to use this versus alternatives like 'get_projects' for listing all projects or 'get_project_members' for member details. No exclusions or prerequisites are mentioned, leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_project_membersB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds useful context: it specifies that the tool retrieves both user and group memberships and supports pagination, which are behavioral traits not inferable from the schema alone. However, it lacks details on permissions, rate limits, error handling, or response format, leaving gaps for a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and front-loaded: the first sentence states the core purpose, and the following two sentences add essential behavioral details without redundancy. Every sentence earns its place by providing distinct value, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, no output schema, no annotations), the description is somewhat complete but has gaps. It covers the purpose and key behaviors (membership types, pagination), but lacks details on response format, error cases, or usage context. This makes it adequate for basic understanding but insufficient for full agent guidance without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate for undocumented parameters. It mentions 'Supports pagination,' which implicitly relates to the limit and offset parameters, adding some semantic meaning. However, it doesn't explain the project_id parameter or provide details on pagination behavior (e.g., default values), so it partially compensates but not fully, aligning with the baseline for moderate coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get members and their roles for a specific project in Lychee Redmine.' It specifies the verb ('Get'), resource ('members and their roles'), and scope ('for a specific project'), distinguishing it from siblings like get_users (general users) or get_project (project details). However, it doesn't explicitly differentiate from all siblings (e.g., get_projects might imply member info), keeping it at 4 rather than 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions 'Includes user and group memberships' and 'Supports pagination,' but these are behavioral details, not usage instructions. There is no mention of prerequisites, when to choose this over get_users or get_project, or any exclusions, resulting in minimal guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_projectsB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool supports pagination, which is useful behavioral context beyond the input schema. However, it doesn't mention other traits like whether it's read-only, requires authentication, has rate limits, or what the output format looks like, leaving significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences with zero waste: the first states the purpose, and the second adds key behavioral context (pagination). It's appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (list retrieval with pagination), no annotations, no output schema, and low schema coverage, the description is incomplete. It covers the basic purpose and pagination but lacks details on permissions, output format, error handling, or how it differs from siblings, making it only minimally adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 2 parameters with 0% description coverage, so the description must compensate. It mentions 'Supports pagination,' which implies the use of 'limit' and 'offset' parameters, adding some semantic meaning. However, it doesn't explain what 'accessible' means or provide details on parameter usage beyond this hint, so it partially compensates but not fully.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get list') and resource ('accessible projects from Lychee Redmine'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_project' (singular) or 'search_issues', which might retrieve similar data, so it falls short of a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_project' (for a single project) or 'search_issues' (which might involve projects). It mentions pagination support, which hints at usage for large datasets, but lacks explicit when/when-not instructions or named alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_usersB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool supports pagination, which is a useful behavioral trait. However, it lacks details on permissions required, rate limits, response format, or whether it's read-only or has side effects. For a tool with zero annotation coverage, this is insufficient to fully inform the agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose and includes key features (filtering and pagination) without unnecessary details. Every word earns its place, making it highly concise and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, no annotations, no output schema), the description is minimally adequate. It covers the basic purpose and hints at behavior (pagination), but lacks details on output format, error handling, or integration with sibling tools. Without annotations or output schema, more context would be helpful for the agent to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning by explaining that 'status' filters by active, locked, or all users, which aligns with the enum in the schema. However, it doesn't cover 'limit' and 'offset' parameters beyond mentioning pagination support, leaving their specific roles unclear. The description provides some value but doesn't fully compensate for the low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get list') and resource ('users from Lychee Redmine'), making the purpose specific. It distinguishes from siblings like get_project or get_projects by focusing on users rather than projects or issues. However, it doesn't explicitly differentiate from get_project_members, which might also involve users, so it's not a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by mentioning filtering by status and pagination, suggesting it's for retrieving user lists with those controls. However, it doesn't provide explicit guidance on when to use this tool versus alternatives like get_project_members or search_issues, nor does it specify prerequisites or exclusions, leaving room for ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.