Server Quality Checklist

Tool Scores

• get_lemmaA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adequately describes the core functionality (returns all possible base forms and word classes) and output structure, but lacks details on error handling, rate limits, authentication needs, or performance characteristics that would be helpful for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with a clear purpose statement, parameter documentation, and return value explanation in just three sentences. Every sentence adds essential information with zero wasted words, making it easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no annotations and no output schema, the description provides good coverage of purpose, parameters, and return structure. However, it lacks explicit error cases or limitations (e.g., handling of non-Icelandic words, empty inputs), which would make it fully complete for agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must fully compensate. It explicitly documents the single parameter 'word' with clear semantics ('The word form to analyze'), provides examples, and explains what it represents (inflected Icelandic form), adding significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific verb ('find') and resource ('lemma(s) and word class(es)'), explicitly distinguishes from siblings by focusing on Icelandic word form analysis, and provides concrete examples ('hestana', 'laga', 'færi') to illustrate its unique function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context ('Given any inflected form') and implies usage for Icelandic language analysis, but does not explicitly state when to use this tool versus the sibling tools (get_variant, lookup_word), nor does it mention any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_variantA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes what the tool does (converts words to specified grammatical forms), provides examples of input-output behavior, and outlines the return structure. It doesn't mention error cases, rate limits, or authentication needs, but covers core functionality well given the annotation gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with a clear purpose statement, explanatory sentence, examples, and well-organized parameter documentation. Every sentence earns its place by providing essential information without redundancy, and it's appropriately front-loaded with the core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of grammatical transformation with three parameters and no output schema, the description does an excellent job explaining inputs and providing return value documentation. It could be more complete by explicitly mentioning error conditions or limitations, but it covers the essential context needed for effective tool use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing detailed semantic explanations for all three parameters. It defines 'word' with examples, explains 'word_class' with common values and meanings, and thoroughly documents 'target_form' with multiple examples and tag explanations, adding significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Get', 'converts') and resources ('grammatical variant of an Icelandic word'), distinguishing it from siblings like 'get_lemma' (which likely returns base forms) and 'lookup_word' (which likely provides definitions or general information). It provides concrete examples ('convert "hestur" to dative plural') that illustrate its unique functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool through examples and parameter explanations, suggesting it's for grammatical transformation rather than lemma retrieval or general lookup. However, it doesn't explicitly state when NOT to use it or name alternatives like 'get_lemma' or 'lookup_word', which would be needed for a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_wordA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does so well by detailing the tool's behavior: it performs a lookup, returns a structured dict with specific fields (found, search_key, entries), and explains the search key adjustment (z->s replacement). It covers the output format comprehensively, though it lacks information on error handling or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the core purpose, followed by detailed explanations of arguments and returns in a structured format. Every sentence adds value, with no redundant information, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (2 parameters, no annotations, no output schema), the description is largely complete, covering purpose, usage, parameters, and return values in detail. However, it could benefit from mentioning potential limitations or error cases, slightly reducing completeness for a tool with no structured output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate, which it does excellently. It adds meaning beyond the schema by explaining what 'word' is (e.g., 'Icelandic word form' with examples like 'hestur') and clarifies the purpose of 'at_sentence_start' (checking lowercase forms for capitalized words at sentence start), providing practical context not in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Look up an Icelandic word form'), the resource ('BÍN'), and the outcome ('return all matching entries'). It distinguishes from siblings by specifying it finds 'all possible interpretations' including lemma, word class, and grammatical tags, unlike get_lemma or get_variant which likely focus on specific aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use this tool (e.g., for looking up word forms and their interpretations) and includes a practical example for the 'at_sentence_start' parameter. However, it does not explicitly state when not to use it or name alternatives among siblings, though the distinction is implied by the detailed functionality described.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.