Server Quality Checklist

Tool Scores

• kaeris_list_languagesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the return format (mapping of language code to English name) and provides an example. This makes the behavior predictable, though it could be more explicit about being a read-only operation with no side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences plus an example, with zero waste. The purpose is front-loaded in the first sentence, and the example clarifies the output format. This is an ideal length for a simple list tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description is quite complete. It explains what is returned and how to use the result with other tools. It could additionally mention that the list is static or that no authentication is needed, but that is likely inferred from the context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, so the schema conveys no additional information. The baseline for 0 params is 4, and the description doesn't need to add parameter details. It correctly mentions nothing about parameters, which is fine since there are none.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('List every target language') with a specific resource ('KAERIS can translate into'). It includes a concrete example ('{"es": "Spanish", ...}') and distinguishes itself from sibling tools like kaeris_translate or kaeris_add_language, which perform different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage instructions: 'Use the codes with kaeris_translate / kaeris_translate_file.' This tells the agent when to use this tool's output. However, it does not mention alternative tools or explicitly say when not to use it, which would be needed for a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kaeris_list_missing_keysA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses what the tool returns (missing keys, missing values, extra, placeholder issues), notes that JSON works offline while other formats go through the backend '/api/parse' at no cost, and implies a read-only operation by listing keys rather than modifying files. It lacks explicit side-effect or permission information, but the read-only nature is strongly implied. This is solid disclosure for a list-type tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a lead purpose sentence, a brief note on formats/alternatives, then an 'Args' section and 'Returns' section. Every sentence provides value, and the most critical information is front-loaded. The length is justified by the need to document args and return shape, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 3 parameters (all explained in detail), no output schema (but the Returns section fully describes the response shape), and a moderately complex task (finding missing/placeholder-broken keys). The description also covers format compatibility and next-step alternatives, so an agent has all necessary context to select and invoke the tool correctly. No gaps are evident.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate, and it does thoroughly. It explains 'source' with examples ('locales/en.json'), 'lang' with a code example ('de'), and 'out' including its default behavior and expected file naming ('<out>/<lang><source-ext>'). This adds meaning well beyond the bare schema fields, making the tool fully usable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool does: 'The exact keys missing or placeholder-broken in ONE target locale... so an agent knows exactly what to translate/fix.' It uses a specific verb ('list' implied) and resource, and distinguishes from translation tools by noting 'use kaeris_add_language or kaeris_translate to actually fill them in.' This is unambiguous and differentiates from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear use context: to identify what needs translation/fixing. It explicitly names alternatives for the next step: 'use kaeris_add_language or kaeris_translate to actually fill them in.' However, it does not explicitly contrast with other sibling tools like kaeris_scan_repo or kaeris_list_languages, leaving some ambiguity about when to choose this over those. Still, the primary usage is well communicated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kaeris_translateA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does so excellently. It discloses placeholder-safe handling, pass-through of non-strings, Translation QA with placeholder and overflow warnings, back-translation behavior, translation-memory reuse, and API key fallback logic. It even explains token cost for verify=True.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Although lengthy, the description is meticulously structured: a concise opening, a brief feature-highlight paragraph, a linear Args breakdown with one parameter per line, and a clear Returns block. Every sentence adds necessary information and no words are wasted.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 11 parameters, nested objects, no output schema, and no annotations, the description is remarkably complete. It explains all parameters, return fields, edge cases (failed_languages), and even provides a direct instruction to the agent ('You are reading the repo, so you know this: pass it.'). It leaves virtually no ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the Arduino-style Args section provides thorough explanations for all 11 parameters, including types, examples, defaults, and nuanced semantics (e.g., what 'context' is for, how 'reuse' works, how 'icu' affects processing). This fully compensates for the sparse schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action and resource: 'Translate a set of UI strings into one or more languages.' It clearly distinguishes from sibling tools like kaeris_translate_file by specifying it operates on a set of key-value pairs rather than a file.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly indicates when to use this tool (when you have a set of UI strings) and even references kaeris_list_languages for valid language codes. However, it does not explicitly exclude alternatives like kaeris_translate_file or provide explicit when-not-to-use guidance, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kaeris_translate_fileA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses side effects (file writes, merging), the lock-file mechanism, conditions for retranslation, and the return value. It even explains how context affects translation sense with concrete examples, going far beyond a typical definition.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Although long, it is structured with an Args/Returns layout and front-loaded with purpose and supported formats. Every sentence contributes necessary information; there is no fluff or tautology.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex 13-parameter tool with no annotations and no output schema, this description covers everything: return values, file formats, mode constraints, parameter semantics, and example usage. It is fully self-contained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet the description compensates fully by explaining every one of the 13 parameters with examples, defaults, constraints, and interactions (e.g., context max 300 chars, only_new JSON-only, reuse non-incremental).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb-resource pair: 'Translate a strings file on disk and write the results next to it.' It further clarifies scope by listing supported formats, and distinguishes itself from siblings by focusing on file-based translation with incremental options.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Clear conditions are given: only_new is JSON-only and incremental, verify and reuse are non-incremental-only. While it doesn't explicitly name sibling alternatives, it provides strong when-to-use context through these restrictive clauses.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kaeris_add_languageA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that this 'CALLS THE TRANSLATION API (same as kaeris_translate_file)' and is not free unlike local-only siblings. It explains parameter effects (tone, icu, keep) and the meaning of context with a concrete example. It also states the return format. This is richly transparent beyond just saying it translates.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is moderately long but every sentence earns its place due to the tool's complexity (9 params, cost implications, alternatives). It is structured with a clear 'Args:' section and a 'Returns:' section, front-loading the key usage guidance and API call warning. No redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 9 parameters, no output schema, and no annotations, the description provides complete context: purpose, usage triggers, alternatives, parameter semantics, and return value. It even includes guidance on how to obtain context (from reading the repo). An agent can select and invoke this tool correctly without additional information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It does: every parameter gets an explanation and often an example (source: 'locales/en.json', lang: 'de', tone: 'formal'/'casual', icu: 'preserves plural/select syntax', keep: 'brand/product names', context: with the 'Bank' example, api_key/openrouter_key as auth overrides). This fully covers all 9 parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Bootstrap a BRAND-NEW target locale by translating the entire source file into it and writing the result,' which is a specific verb+resource action. It further distinguishes from siblings by stating when to use it (when a language has no locale file at all in 'missing_files') and contrasts it with kaeris_translate_file for existing files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly provides usage context: 'Use this when kaeris_scan_repo/kaeris_status show a language with no locale file at all yet.' It also gives an alternative: 'For an EXISTING locale that's just missing a few keys, prefer kaeris_translate_file(..., only_new=True)' and highlights the cost difference (API call vs local-only). This is clear when/when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kaeris_scan_repoA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility. It discloses that the tool is 'Purely local — reads the filesystem, makes no API call,' and explains that it 'Skips node_modules, .git, build/dist output and other noise; bounded so it's safe on large repos.' It also clarifies that outputs are 'best guess' or null when ambiguous, providing a clear behavioral model.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear opening statement, usage guidance, supported frameworks, and a return schema. Each section earns its place; no redundant text. Despite its length, it remains focused and informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (supporting many frameworks and scanning a repo), the description covers purpose, use cases, safety, parameters, and return values in detail. The absence of an output schema is compensated by an explicit return JSON structure with field explanations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, 'path', is described as 'directory to scan (default: current directory).' This adds semantic meaning beyond the schema, which lacks any description. For a single simple parameter, this is fully sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Discover a repo's i18n setup' and lists the exact outputs (locale files, base language, target languages, framework). It distinguishes itself from siblings by positioning itself as the initial discovery step, explicitly naming tools like kaeris_status and kaeris_add_language as subsequent steps.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'Use this FIRST when asked to "look at/audit/fix this repo's i18n" and you don't already know the source file.' It also provides an exclusion condition and points to specific alternative tools, giving strong guidance for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kaeris_statusA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does so excellently. It discloses that JSON is checked fully offline while other formats hit the /api/parse endpoint, that there is no translation or cost, and details the exact outputs: missing keys, extra/stale keys, placeholder mismatches, and missing files. This provides a complete behavioral profile.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but densely packed with valuable information, structured logically from summary to format handling to usage guidance to args to return value. Every sentence serves a purpose, and the lead sentence immediately clarifies the tool's role. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity and the absence of an output schema, the description covers all necessary context: supported formats, offline vs. backend behavior, parameter defaults, expected file layout, and a detailed return object. An agent could confidently select and invoke this tool without ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates richly. It defines source with concrete examples, explains langs as target codes with fallback to kaeris.json or auto-discovery, and details the out directory with default behavior and expected file naming. This adds far more meaning than the sparse schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb (completeness/health check) and resource (every target locale against a source file). It explicitly distinguishes itself as the 'i18n firewall' and contrasts with sibling tools like kaeris_list_missing_keys and kaeris_add_language, making its unique purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance: 'Use this to find out what's broken before deciding whether to call kaeris_list_missing_keys... or kaeris_add_language.' It also explains fallback behavior for langs and format coverage, giving clear context and alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.