Server Quality Checklist

Tool Scores

• searchB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description must carry the full burden of behavioral disclosure. It states the core search behavior and optional value matching, but omits details such as case sensitivity, result ordering, pagination behavior, or the shape of the response. For a search tool, this lacks important behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler or redundant content. Every word earns its place, making it appropriately concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 4 parameters, no annotations, and no output schema. The description only conveys the basic purpose, leaving users to guess how limit, locale, and searchValues interact, and what the response contains. Given the minimal structured context, the description is incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides descriptions for all 4 parameters (100% coverage), so the baseline is 3. The description adds little beyond the schema—'optionally values' echoes the searchValues parameter description without providing new meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'substring search' and a clear resource ('translation keys'), and even notes the optional scope expansion to values. This distinguishes it from sibling tools like 'keys' (likely listing all keys) and 'get_key' (exact lookup).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a search use case but does not explicitly state when to prefer this over sibling tools like 'keys' or 'get_key', nor does it mention any exclusions. The intent is inferable from the name and phrasing, but there is no direct guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• keysA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full transparency burden. It states it lists key paths and can filter, but it does not disclose pagination behavior, default limits, or explicitly state that it is a read-only operation. This is minimal for a list tool and does not add context beyond what the schema already provides.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded with the action ('List translation key paths'), and includes the key filtering capability with no superfluous words. It is concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with optional filters, and the description covers the primary use case. However, with no output schema or annotations, it does not explain return format, default behaviors, or edge cases. It is adequate but not fully complete for an agent that might need more context on what the tool returns or how it behaves.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers all three parameters (limit, locale, prefix) with descriptions, so the baseline is 3. The description adds that locale and prefix are filters, but this is already implied by the schema's parameter descriptions. No new semantic meaning is provided beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' and clearly identifies the resource as 'translation key paths'. It also mentions filtering by locale and/or prefix, which distinguishes it from sibling tools like get_key (single key) and search (likely more complex). The purpose is unambiguous and directly actionable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use this tool (listing translation key paths, optionally filtered), but it does not explicitly contrast with alternatives such as search or get_key. There is no when-not-to-use guidance or mention of alternative tools, leaving usage somewhat implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• missingA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the full burden of behavioral context. It adds the comparison against a base locale with default 'en' and lists three audit checks, which is useful. However, it does not state whether the tool is read-only, what the output format looks like, or any operational constraints, leaving important behavioral details undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single compact sentence that front-loads the main verb 'Audit' and packs in the key checks and default locale. It is dense but not overly verbose, with every phrase contributing to the meaning. A minor deduction for the slightly list-like enumeration, but overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of an output schema, the description should indicate what the tool returns (e.g., a list of issues and their locations). It only states what it checks, not the response format or behavior, leaving a significant gap for an agent to understand the tool's full behavior and how to consume its results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides descriptions for all three parameters (100% coverage), so the schema carries the parameter semantics. The description's mention of 'base locale (default: en)' restates the baseLocale parameter without adding new meaning beyond the schema. Thus, it neither compensates for gaps nor enhances understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action (audit) on keys and enumerates three distinct checks (missing per locale, empty values, placeholder mismatches), distinguishing it from sibling tools like get_key or search. It is a specific verb+resource combination with clear scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage context is implied: it is for auditing translation consistency relative to a base locale. However, there is no explicit guidance on when to use this tool vs alternatives, nor any mention of exclusions or edge cases, so the guidance remains implicit rather than direct.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_keyA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses the side effect of pruning empty parents and the default locale scope, which is useful. However, it omits other destructive-operation details like reversibility or return values, though the verb 'Delete' signals irreversibility.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that immediately states the action and resource, then adds a concise behavioral note. No wasted words, earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 params, no output schema, no annotations), the description adequately covers the purpose, scope, and a key side effect. A minor gap is the lack of explicit error or return behavior, but this is not required without an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds no extra parameter meaning beyond what the schema already contains; the 'default: all' for locales is duplicated from the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool deletes a key from locales, with a specific verb and resource. It distinguishes itself from siblings like get_key, set_key, and search by the 'Delete' action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (when you want to delete a key) and notes the default behavior (all locales), but does not explicitly compare alternatives or state when not to use it. No exclusion criteria or alternative tool references are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_keyA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. The word 'Get' clearly signals a non-destructive read operation, and the added detail about returning the file each value lives in provides useful output behavior context. However, it does not mention error handling or sorting, but these are not critical for such a simple tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys purpose, scope, and output detail without any redundant words. It earns its place entirely.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple tool complexity, a 2-parameter schema with full coverage, and no output schema, the description is reasonably complete. It explains what the tool returns (values and the file), which partially compensates for the lack of an output schema. Minor gaps like missing-key behavior are acceptable at this simplicity level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% coverage with descriptions for 'key' (dot-separated key path, e.g. 'roles_scope.super_admin') and 'locales' (default: all locales). The description's phrase 'across locales' aligns with the 'locales' parameter but adds no new semantic detail beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get a key's values across locales, with the file each lives in' clearly states the verb ('Get'), resource ('a key's values'), scope ('across locales'), and an additional detail ('with the file each lives in'). This distinguishes it from sibling tools like 'keys' (listing all keys) and 'search' (fuzzy lookup).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used to retrieve a specific key's translations, but it does not explicitly state when to use it over alternatives like 'search' or 'missing'. The context is clear enough for a simple read operation, but there are no explicit when-to-use or when-not-to-use directions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• localesA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It implies a read-only list operation, but does not explicitly state that it performs no modifications or mention any potential side effects or performance characteristics. Adequate for a simple list tool but with gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no filler. It front-loads the action and resource, followed by the specific output details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (no parameters, no annotations, no output schema), the description adequately explains what the tool returns and the context in which it is used. Nothing critical is missing for a simple list operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the schema provides no semantic information. The description doesn't need to explain parameters because none exist. Baseline for 0 params is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List'), the resource ('locales'), and the specific data included (file/key counts and parse problems). It is distinct from sibling tools like 'keys' or 'missing', which focus on other aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for getting an overview of locales, but it does not explicitly say when to choose it over siblings. No alternatives or exclusions are mentioned, though the context makes it fairly obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_keyA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses file placement logic and warns on placeholder mismatch, which is useful. However, it doesn't mention overwriting behavior, error handling, or permission requirements, leaving some important behavioral traits undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short sentences, each contributing distinct value: purpose, file selection logic, and placeholder warning. There is no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a moderate-complexity tool with no output schema and no annotations, the description covers the core behavior and important edge cases (file selection, placeholder warning). It doesn't explain return values or error states, but for a mutation tool this is acceptable. The description is sufficiently complete for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds extra meaning by explaining how the optional 'file' parameter falls back to the current file, nearest prefix, or locale default. It also hints at validation of 'values' via placeholder mismatch warnings.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource pair ('Create or update a key per locale') and clearly distinguishes this from sibling tools like get_key, delete_key, and search. It also explains the file selection behavior, which further clarifies the tool's scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys clear context for when to use this tool: when creating or updating a key per locale. It doesn't explicitly name alternatives or exclusions, but the purpose is unambiguous enough for an agent to select it over read/search/delete tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.