Server Quality Checklist

Tool Scores

• get_readmeB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavior. It mentions 'decode' but does not explain what decoding means, whether authentication is required, what errors occur if the README is absent, or what the output structure looks like. This leaves significant uncertainty for a tool that performs an action beyond a simple fetch.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with no redundant words. It efficiently states the action and target, making it a model of conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The basic action is clear, but given no output schema or annotations, the description should provide more detail about return format, error cases, and any side effects. It is adequate for a simple tool but leaves gaps that could confuse an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not mention owner or repo at all. Parameter meaning is left entirely to the property names, which is insufficient compensation for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses the specific verb 'fetch and decode' and names the resource 'README file contents for a repository', clearly distinguishing it from siblings like search_repositories or list_issues.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or alternatives are mentioned, but the tool's purpose is clear enough that an agent can infer it should be used when README contents are needed. Lacks exclusions or guidance about when other repository tools might be preferable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_issuesA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the burden. It discloses sort order ('most recently updated first') and state filtering ('open or closed'), but omits default state, limit behavior, and pagination details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise, front-loaded sentence. It communicates the essential purpose and ordering without redundant words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and no annotations, yet the description omits default state (open), the limit constraint, and return structure. For an autonomous agent, this leaves important gaps in understanding what the tool returns and how defaults work.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds minimal context for the 'state' parameter but does not explain owner, repo, or limit semantics. The default and constraints are only in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists issues for a repository, with a specific verb and resource. It distinguishes from sibling tools (search_repositories, get_repository, get_readme) which focus on different entities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the usage context ('for a repository') but does not explicitly state when to prefer this tool over alternatives or provide exclusions. It is clear enough for issues but lacks explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_repositoryA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that this is a fetch/read operation and lists the metadata returned, which is helpful. However, it does not mention potential error cases (e.g., 404), authentication needs, or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently communicates the purpose and key output fields. No unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with two well-described parameters and no output schema, the description is mostly complete. It lists the returned metadata fields, giving the agent a clear idea of the result. However, it could briefly mention error handling or authentication requirements for full completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description does not add extra meaning to the owner and repo parameters beyond what the schema already provides; it only lists the metadata fields returned, which is not directly parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches metadata for a single GitHub repository and lists the specific fields (description, stars, forks, etc.). This specific verb-resource combination distinguishes it from sibling tools like search_repositories and list_issues.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for when you have a known repo and need metadata, but it does not explicitly state when to use this tool versus alternatives such as search_repositories or get_readme. Sibling tool names provide context, but no direct guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_repositoriesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. It states that only public repositories are searched and results are 'top matches', implying ranking. However, it does not detail ordering criteria, pagination, or specify that it is read-only, though 'search' implicitly indicates a non-mutating operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the action ('Search public GitHub repositories by keyword') and directly lists the return fields. Every word adds value, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (2 parameters, no output schema, no annotations), the description covers the primary use case, scope, and return fields. It does not specify sorting or result count, but these are inferable from 'top matches' and the 'limit' parameter. It is sufficient for an agent to correctly select and invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both 'query' and 'limit'. The description adds minimal extra meaning beyond tying 'query' to 'keyword' and implying the limit affects 'top matches', but this is redundant with the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'search' with a specific resource ('public GitHub repositories') and a scope ('by keyword'), and lists the output fields. It distinguishes itself from sibling tools like 'get_repository' (specific repo) and 'list_issues' by focusing on search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for use: searching public repositories by keyword. It does not explicitly mention alternatives or exclusions, but the phrase 'public GitHub repositories' implies it is for finding repositories, not for other operations like issues or readmes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.