Server Quality Checklist

Tool Scores

• list_issuesC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears the full responsibility for behavioral disclosure. It conveys a read-only list operation, but it says nothing about default state filtering, pagination, authentication, response contents, or access limitations. The description provides little behavioral context beyond what the name already implies.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no wasted words and the core target is accurately fronted. It is concise and easy to parse, though it lacks peripheral details that would make it more informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no annotations and no output schema, the description leaves too much implicit. It does not explain response behavior, default state behavior, or how this listing interacts with discussion. The agent can invoke the tool based on the schema, but not confidently understand the full behavior from the description alone.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description should compensate for the undocumented parameters. The phrase 'de um repositório' loosely aligns with owner and repo, but the description does not clarify their meaning or relation, and it entirely omits state semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear action, 'list', and a clear resource, 'issues', scoped to a repository. This makes the tool's basic purpose understandable and distinguishes it functionally from sibling tools like list_repos or create_issue. However, it does not explicitly differentiate itself from alternatives or mention filtering behavior.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance about when to use this tool versus alternatives such as list_repos, search_code, or create_issue. There are no preconditions or exclusions stated, so the agent must infer usage entirely from the tool name and schema.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_repoC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full behavioral burden. It only says 'gets details', implying a read-only operation, but it does not explain error behavior (e.g., non-existent repo), return contents, authorization needs, or any constraints the agent should know before invoking it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with no wasted words and communicates its basic intent up front. However, it is very terse and needs more detail to truly stand alone, though it is not overly verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The lack of an output schema and annotations means the description should explain what 'detalhes' realistically includes or what the agent can expect as a result, but it does not. While the 2-parameter schema is simple and clear, the behavior around failures and return shape is left entirely to the agent to guess.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, since both 'owner' and 'repo' are already described in the JSON schema. The description does not add any parameter-level details beyond what the schema already provides, so the baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a clear verb ('obtém' / 'gets') and a specific resource ('detalhes de um repositório específico'), which adequately conveys this tool retrieves a single repository's details. It implicitly distinguishes from list_repos via the word 'específico', but it does not name that sibling or specify which details are returned.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance about when to use this tool versus list_repos, list_issues, or search_code. No alternatives are mentioned, and there are no usage conditions, prerequisites, or exclusions for the agent to follow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_file_contentB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Nenhuma anotação foi fornecida, então a descrição precisa explicar o comportamento. Ela apenas descreve a leitura, sem mencionar formato de retorno, limites, autenticação, codificação do conteúdo ou condições de erro. O agente não obtém visibilidade sobre como a resposta será entregue.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A descrição é uma frase curta e direta, sem conteúdo redundante. Ela entrega o objetivo principal sem incluir informações desnecessárias para a compreensão básica do uso.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Sem annotations e sem schema de saída, a descrição deveria compensar com mais informações para chamadas corretas. O texto atual é muito enxuto para um cenário onde o formato de resposta, possíveis limites e condições de disponibilidade não são explicitados. Ainda que a ferramenta seja simples, o agente fica sem contexto suficiente para cenários não triviais.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  A cobertura do schema é de 50% e a descrição não acrescenta significado aos parâmetros owner, repo e ref. path já é explicado no schema, mas owner e repo permanecem vagos e a descrição geral não ajuda a entender o que cada parâmetro representa no contexto da chamada.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  A descrição afirma claramente o recurso e a ação: 'Lê o conteúdo de um arquivo em um repositório'. Isso distingue a ferramenta dos irmãos list_repos, get_repo, list_issues, create_issue e search_code, pois o agente identifica que o objetivo é recuperar o conteúdo de um arquivo específico.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Não há orientação explícita sobre quando usar esta ferramenta nem alternativas para evitar. O uso precisa ser inferido pelo verbo 'ler', sem indicar diferenças de contexto em relação a ferramentas semelhantes. Falta contexto de seleção.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_issueB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It states that the operation creates, but does not mention permissions, return type, or side effects beyond the creation itself.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence with no wasted words. The action and resource are stated immediately and clearly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no annotations and no output schema, and the description is too minimal to give an agent complete context for executing the operation, especially for identifying owner/repo and understanding expected outputs or permission dependencies.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 60%, leaving owner and repo undocumented. The description does not clarify these required parameters; 'um repositório' is vague and does not compensate for the gaps in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Cria') and resource ('nova issue em um repositório'), distinguishing it from siblings like list_issues and get_file_content. There is no ambiguity or tautology.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the use case: when a new issue should be created. However, it does not explicitly state when not to use this tool or how it compares to alternatives such as list_issues.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_reposA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. 'Lista' implies a read-only operation and the authenticated-user scope is disclosed, but it does not mention pagination behavior (only one page returned per call, default 20), authentication requirements, rate limits, or error conditions. Acceptable but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, precise sentence in Portuguese that leads with the action and resource. No filler, no redundancy with the schema. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple listing tool with 2 well-documented parameters this is largely adequate, but the output shape is not mentioned (no output schema) and pagination semantics are left implicit. It lacks any annotation backstop, so slightly more disclosure would be appropriate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so both parameters are already documented with constraints: 'sort' has an enum and default, 'per_page' has min, max, and default. The description adds nothing beyond the schema, which is fine — baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Lista') and resource ('repositórios do usuário autenticado'), clarifying both the action and the scoping to the authenticated user's own repos. This clearly differentiates it from siblings like get_repo (single repo), list_issues (issues), and search_code (code search).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use is implied by the purpose: call this when you need the authenticated user's repository list. However, it gives no explicit exclusions or nod to alternatives, e.g., no pointer to get_repo for a single repository or search_code for broader discovery.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_codeA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, all behavioral disclosure falls on the description. It usefully mentions that the tool relies on GitHub's search syntax rather than natural language, which is an important behavioral nuance. It does not mention authentication, result shape, pagination, or rate limits, but the query-syntax note adds meaningful transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action and includes the most important usage detail: GitHub search syntax. There is no wasted or redundant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter, simple search tool with no output schema, the description is largely complete: it tells the agent what the tool searches, where, and the syntax to use. It could additionally clarify what kinds of results are returned, but this is not a serious omission for such a tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the parameter description already provides concrete examples using GitHub search syntax. The tool description reinforces that the query uses GitHub syntax, but adds little beyond what the schema already provides, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Busca código') and resource ('no GitHub'), and clarifies that it uses GitHub's specific search syntax. This distinguishes it from sibling tools like list_repos and list_issues, which operate on different GitHub entities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use is implied: an agent would call this when needing to find code, as opposed to listing repositories or issues. However, the description gives no explicit guidance about when to choose this tool over siblings, nor any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.