Server Quality Checklist

Tool Scores

• list_dirC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states what the tool lists, but doesn't disclose whether it's recursive, whether it includes hidden files, whether it returns absolute or relative paths, or whether it errors on invalid paths. Given the empty annotations, this is insufficient transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately brief (single sentence in Portuguese), but it borders on under-specification rather than conciseness. For a simple list-directory tool, one sentence is acceptable, but the sentence lacks useful detail that would justify the brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a straightforward directory listing with an output schema and a single simple parameter. The low complexity mitigates some gaps, but the 0% parameter coverage combined with no behavioral details about recursion, hidden files, or error handling make the description incomplete for reliable agent usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the undocumented parameter. The only parameter, 'rel_path', has no description text explaining its format, whether it accepts slashes, whether it's a directory or a file path, or how the default '.' behaves. The description adds zero parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb+resource ('Lista arquivos e pastas dentro do projeto' - lists files and folders within the project). It clearly distinguishes from siblings like read_file, write_file, and change tools, as it is a directory-listing operation. However, it doesn't explicitly state scope or contrast with similar navigation concerns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. It doesn't mention when listing directories is appropriate, how it relates to read_file, or any exclusions/limitations. There is no context about whether it works on directories vs files only, or any behavioral constraints.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states it reads content from a project file but doesn't disclose whether the file must exist, what happens on missing files, encoding behavior, or whether it's a read-only operation (which it clearly is by implication). The description adds minimal behavior beyond the obvious.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single efficient sentence with zero waste. It communicates the core purpose in the fewest words needed. However, it may be under-specified rather than concisely complete.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema and only 1 parameter, the tool lacks annotations entirely and has zero schema description coverage. For a file-reading tool serving as a sibling to write_file and change-proposal tools, the description should note relative-path grounding and typical usage flow, but it doesn't. The simplicity keeps it from being a 1.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. The single parameter rel_path is vaguely implied by 'arquivo do projeto' but its semantics (relative to what root?) and format requirements are not explained. There is no output schema detail needed for params, but the one parameter's meaning is left to inference.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Lê o conteúdo de um arquivo do projeto' (reads a file's content from the project) clearly states the verb (read) and resource (file content). It distinguishes itself from siblings like write_file, list_dir, and the change-proposal tools. Slightly below 5 because it's minimal and doesn't add scoping details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No when-to-use guidance is provided. With siblings list_dir and write_file, an agent might benefit from knowing when to use read_file (e.g., before proposing changes) versus these alternatives, but the description offers no such context or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_pending_changesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It never states whether this is read-only vs. mutating, whether it returns a list, what the output contains, or how filtering behaves. The output schema exists but the description adds no behavioral traits beyond the three filter values. For a listing tool, the read-only nature is implicit but unstated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, efficient sentence in Portuguese that conveys the purpose and filter options. No waste, appropriately brief.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists, so return-value details are covered elsewhere. The tool has only one optional parameter. However, the description doesn't convey whether the list is read-only, ordering, pagination, or any caveats. Given the simplicity (1 param) and presence of an output schema, this is adequate but thin.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It lists the three valid status values, which partially documents the 'status' parameter's allowed values. However, it adds no detail on format, case sensitivity, or default behavior beyond the schema's own 'default: pending' — so the value added over the schema is marginal.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description says it lists plans by status with three allowed values ('pending', 'approved', 'rejected'). Verb+resource is clear ('lista planos por status'), but it only vaguely distinguishes from siblings like approve_change/reject_change — those are action tools while this is a listing tool, but the description doesn't make that contrast explicit. It does communicate the core function adequately.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The status values ('pending', 'approved', 'rejected') imply the usage context — an agent filtering across plan states. However, there are no explicit when-to-use instructions, no exclusions, and no mention of sibling alternatives. The context is reasonably implied but not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reject_changeC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. It reveals the action (rejects/cancels) but says nothing about reversibility, whether there are conditions for rejection (only pending), what side effects occur (does it notify someone?), or what the output represents. For a mutation tool with no annotation coverage, more behavioral detail is needed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short sentence in Portuguese, which is efficient and front-loaded. Every word earns its place. It could add a bit more context, but it is appropriately sized for a simple single-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no annotations, minimal parameter info in the description, and only one simple parameter. An output schema exists, so return values are documented elsewhere. Given the low complexity, the description is near-adequate, but it lacks guidance on preconditions (must be pending) and success/failure implications, which would improve completeness for a mutation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There is one parameter (plan_id) with 0% schema description coverage, so the description must compensate. The description does not mention plan_id at all, though the tool name and description context ('plan pendente') make it reasonably inferable that plan_id identifies the plan to reject. Baseline is 3 given the single, self-evident parameter name, but the description adds no explicit meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb+resource ('rejects/cancels a pending plan'), which clearly distinguishes it from siblings like approve_change and propose_change. However, the term 'plano pendente' (pending plan) is ambiguous - it's unclear if this means a pending change, proposal, or another entity type. The purpose is understandable but terminology is vague.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The name 'reject_change' and description imply it's the counterpart to approve_change, which gives implied usage context. However, there is no explicit guidance on when to reject vs. alternatives, no mention of caveats (e.g., can only reject pending plans, not approved/executed ones), and no stated relationship to propose_change or list_pending_changes for discovering plan_ids.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• write_fileA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the content gets a traceability comment prepended (feature, plan, date/time) and that the operation overwrites. However, it doesn't mention whether it requires specific permissions or whether overwrites are destructive/reversible—useful context not stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Compact 3-sentence paragraph in Portuguese, front-loaded with the core action. Every sentence adds value: what it does, the prerequisite, and the automatic behavior. Minor waste: 'Escreve/sobrescreve' is slightly redundant phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 5 required params, 0% schema coverage, no annotations, but a present output schema, the description gives enough to use the tool safely (prerequisite, auto-comment). It could better explain what the output schema returns and the relationship between description param and the plan, but overall minimally complete for a mutation tool with a gating workflow.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate, and it partially does: it explains plan_id's semantic role (must be approved, scope must include rel_path) and how feature relates to the traceability comment. However, rel_path, content, and description semantics remain entirely delegated to the schema with no added meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it writes/overwrites a project file (specific verb+resource). It doesn't explicitly distinguish from siblings like read_file, but the write action is unambiguous and the sibling names make the differentiation reasonably clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Establishes a clear prerequisite: a pre-approved plan_id (via propose_change + approve_change) whose scope must include rel_path. This explicitly tells the agent when it's valid to use this tool and what precondition must be satisfied, though it doesn't explicitly list alternatives or when NOT to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• approve_changeA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses the key behavioral aspect (unlocks write_file for declared files), which is the most important side effect. However, it doesn't state whether approval is one-time or persistent, what happens on re-approval, or any error conditions (e.g., plan not found, already approved). For a state-changing workflow tool with no annotations, this is somewhat thin but captures the essential effect.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two compact sentences, front-loaded with the purpose and effect, followed by the usage precondition. Zero filler; every phrase adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple one-parameter, workflow-gated tool with an output schema present, so complete documentation burden is low. The description tells the agent what it does, its side effect, and when to call it. It could note the relationship to pending-changes listing or rejection flow, but for a single-param approval action the coverage is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for plan_id, but it doesn't elaborate on what plan_id refers to (the plan identifier, presumably from propose_change or list_pending_changes). The single parameter is reasonably self-explanatory given the schema (a string ID), but the description could connect it to the sibling tools that generate/return such IDs. Baseline 3 is appropriate for zero coverage with an obvious param name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource ('Aprova um plano previamente proposto') and clarifies the effect (liberating write_file for declared files). It distinguishes clearly from siblings like reject_change since it's the approval counterpart. However, it doesn't explicitly name the sibling alternatives, though the context makes this obvious.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to call this ('Só chame isso depois de aprovação explícita do usuário na conversa'), providing a clear precondition and guardrail. It doesn't mention when-not-to-use or name alternative tools explicitly, but the call condition is well-defined for a workflow-gated tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• propose_changeA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that this tool returns a 'plano pendente de aprovação do usuário' (pending user approval plan) and its mandatory-before-write_file role, which is good behavioral context. However, no annotations are provided, so the description carries full burden. It doesn't explain what happens if changes are made without proposing first, validation behavior, or the approval workflow details beyond a brief mention.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact — a few sentences in Portuguese covering purpose, usage context, required content, and return behavior. It's reasonably front-loaded with the mandatory-step framing. Slightly dense but each sentence earns its place, though no structural formatting (bullets, headers) is used.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains the workflow position (mandatory pre-write_file step), what data to include, file path convention, and the pending-approval return. With an output schema present, return details are partially covered. It could elaborate on approval mechanics and error cases, but for a declaration tool it's reasonably complete for a 3-param, no-nested-object tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for all 3 parameters. The description explicitly names 'feature', 'description' (descrição objetiva do propósito), and 'files' (lista de paths relativos à raiz do projeto). The files parameter gets useful semantics (relative to project root). However, feature and description just restate their schema names without deeper nuance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is 'Primeiro passo, obrigatório antes de qualquer write_file' (first step, mandatory before any write_file), declaring a feature/alteration with files. It specifies what the tool does: declare a feature, objective description, and list of paths to be created or modified. However, it doesn't distinguish from siblings beyond write_file context, and the purpose is somewhat implicit in the tool name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states this is the mandatory first step before any write_file operation, gives clear guidance on what to include (feature, objective description, list of relative paths), and establishes a precondition/ordering relationship with sibling tools. It also states it returns a pending plan awaiting user approval, clarifying the workflow sequence with approve_change/reject_change.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.