Server Quality Checklist

Tool Scores

• list_directoryC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full burden. It only states basic listing behavior without disclosing if it is recursive, shows hidden files, or has output limits. The output schema exists but is not described here, so agent lacks insight into return structure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. For a simple tool with one parameter, it is appropriately concise, though it could include more useful details without being verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple but the description lacks completeness. It does not specify whether the listing includes hidden files, whether it is recursive, or how errors (e.g., invalid path) are handled. An output schema exists but its content is not hinted.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage. The description adds value by stating the default behavior (current directory when path is null). However, it does not clarify path format (absolute/relative) or any constraints beyond that.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it lists files and folders with a default of current directory. The verb 'list' and resource 'directory' are specific. Sibling tools (change_directory, get_working_directory, etc.) are distinct, but the description does not explicitly differentiate itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. There is no mention of prerequisites, limitations, or context where this tool is preferred over siblings like get_working_directory or run_command.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• change_directoryB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It only states the function but does not disclose error behavior (e.g., invalid path), effects on session, or any side effects. Minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise with no wasted words. It front-loads the verb 'Muda'. While brevity is good, it could benefit from a slightly more structured format (e.g., separate sentence for clarification).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (1 param) and presence of an output schema, the description need not detail return values. However, it lacks behavioral completeness for a mutable tool (no error or confirmation info), and the language mismatch (Portuguese description with English tool name) may hinder comprehension.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds the key detail that the path can be relative or absolute, which goes beyond the schema's type-only specification. However, it omits formatting examples, allowed characters, or default behavior, leaving gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool changes the session's working directory, specifying it can be relative to the current directory or absolute. This distinguishes it from sibling tools like get_working_directory (read-only) and list_directory (listing contents).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for changing directory but provides no explicit guidance on when to use versus alternatives, nor any exclusions or prerequisites. It relies on tool name and context for inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_working_directoryA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Sem anotações, a descrição não revela se a operação é somente leitura, se requer permissões especiais ou se tem efeitos colaterais. Apenas afirma o que faz, sem aprofundar.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Descrição em uma única frase, direta e sem redundâncias. Ocupa o espaço mínimo necessário para transmitir a função.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Para uma ferramenta simples sem parâmetros e com esquema de saída presente, a descrição é adequada mas poderia mencionar o formato do diretório retornado (caminho absoluto) ou possíveis falhas (ex.: sessão não inicializada).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Não há parâmetros; a cobertura do esquema é 100%. A descrição não precisa adicionar informações sobre parâmetros, e o valor base para 0 parâmetros é 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Descreve claramente o propósito: retornar o diretório de trabalho atual. Diferencia-se dos irmãos (change_directory, list_directory) por ser específico para obter o diretório atual.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Não fornece orientação sobre quando usar esta ferramenta em vez das alternativas, como get_system_info ou list_directory. Não menciona contextos de uso ou exclusões.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_system_infoA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It correctly indicates the return format (JSON) but does not disclose that the tool is read-only or has no side effects. For a zero-parameter tool, this is minimally adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, concise and front-loaded. Could be slightly improved by linking to output schema or siblings, but overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Output schema exists to document return values, and description covers the high-level return content. No missing critical information given the simplicity of the tool and the presence of output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters in schema, so baseline is 4. Description does not need to add parameter details, and the schema coverage is 100%, making any additional param info unnecessary.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns system, shell, and session info in JSON format. This differentiates it from siblings like change_directory and run_command, which operate on directories and commands.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. Basic purpose is implied but no exclusions or context about when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_commandA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so description bears full burden. It discloses persistent session, directory state preservation, and timeout mechanism. But lacks details on error handling, output format, or security implications.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise paragraphs. Front-loaded with core purpose. No unnecessary fluff, though could be slightly more compact.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and a simple schema, the description covers key aspects like persistent session, timeout, and purpose. Output schema exists, so lack of return format detail is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has no descriptions (0% coverage). Description clarifies that 'timeout' is in seconds and that the command is executed as a string, but does not elaborate on command syntax or expected format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool executes a local terminal command and returns output. It distinguishes from sibling tools (e.g., change_directory, get_system_info) by focusing on running arbitrary commands.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides context about persistent session and directory maintenance, and explains timeout usage. However, it does not explicitly state when to use this tool versus alternatives like change_directory or list_directory.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.