Server Quality Checklist

Tool Scores

• score_promptB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool costs 1 Kongen Token and returns specific outputs. However, it does not mention whether the operation is read-only, has side effects, or requires permissions. It adds some value beyond the schema but is not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences that front-load the core purpose and outputs. Every sentence adds value with no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with 2 parameters, one required. The description explains the purpose, outputs, and cost. There is no output schema, but the description lists return values. Missing error cases or prerequisites, but overall adequate for the complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of parameters with descriptions. The tool description does not add any additional meaning beyond what is already in the schema for the parameters. Baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool scores a text prompt for reasoning complexity using a specific engine. It uses a specific verb ('score') and resource ('prompt'), and lists the returned values. However, it does not explicitly differentiate from sibling tools like check_usage or route_model, though the purpose is distinct enough.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There is no mention of context, exclusions, or when not to use it. The agent must infer usage from the purpose alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• transfer_scoreA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries full burden. It discloses return values (pattern classification, confidence, adjustment, evidence) and token cost (50 Kongen Tokens). It also mentions that the optional parameter 'source_domain' excludes same-domain patterns. However, it does not explicitly state whether the operation has side effects or requires authentication, but as a scoring function it is likely safe.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences covering purpose, return values, and token cost. Every word is meaningful and front-loaded. No fluff or repetition. It is well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 8 parameters and no output schema. The description provides a high-level overview of output fields but lacks details on their structure or types. It covers the essential purpose and cost, but could be more complete given the complexity of the tool. It is adequate but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with all parameters described in the JSON schema. The tool description adds no parameter-specific information beyond the schema. Since the schema fully documents parameters, a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: scoring a 7-dimensional signal vector against reference patterns. It uses a specific verb ('score') and specifies the resource ('7-dimensional signal vector'). The output is also listed: pattern classification, confidence, etc. This distinguishes it from siblings like 'score_prompt' which likely scores prompts, not signal vectors.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No usage guidelines or comparisons to sibling tools are provided. The description does not indicate when to use this tool over alternatives like 'score_prompt', nor does it specify prerequisites or context. The implied usage is for signal vector scoring, but no explicit guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_usageA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses 'No token cost' which is a positive behavioral trait, but does not mention authentication, rate limits, or data freshness. For a read-only check, this is adequate but not exceptional.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, consisting of two short sentences. Every word adds value, and the structure is front-loaded with the main purpose followed by a key benefit. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description provides sufficient context for an AI agent to understand what the tool does. It could be improved by specifying the return format, but it's largely complete for its purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the baseline is 4. The description adds no parameter details, but the schema coverage is 100% by default. The description's mention of checking balance and usage compensates for the lack of parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks token balance, usage in the current billing cycle, and active plan. It uses a specific verb 'Check' and resource 'Kongen Token balance/usage/plan', and is easily distinguishable from the sibling tools which involve routing, scoring, and transferring.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for checking token status but does not explicitly state when to use this tool versus alternatives or provide any exclusions. Given the simplicity and distinct purpose, the lack of explicit guidance is acceptable but not above average.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• route_modelA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool internally scores using 'Logic', maps to a model, and provides cost savings, and costs 1 token. This is fairly transparent, though it could mention the exact output format (e.g., a string recommendation or structured data).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long, contains no redundant information, and front-loads the core purpose. Every sentence adds distinct value: purpose, internal mechanism, and cost. It is optimally concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (1 param, no output schema, no annotations), the description covers purpose, internal logic, and cost. However, it could be enhanced by specifying what the output looks like (e.g., a model name string) and how cost savings are presented. Overall, it is sufficiently complete for a basic recommendation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema coverage is 100% with one parameter 'text' described as 'The prompt text to evaluate'. The description adds value by explaining the purpose (routing based on reasoning complexity) beyond the schema's minimal description. It provides semantic context that aids in parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool recommends which Claude model to use based on reasoning complexity, and distinguishes itself from siblings like score_prompt (which likely only scores) and check_usage (which likely checks usage limits). The verb 'recommend' and resource 'model' are specific, and the description explains the internal process.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (when you need a model recommendation) but does not explicitly state when to use this tool versus alternatives like score_prompt or check_usage. No exclusions or when-not-to-use guidance is provided, which is a gap given the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.