Server Quality Checklist

Tool Scores

• get_jambonz_schemaA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description bears full burden. It only states the return type (JSON Schema) but does not disclose behavioral traits like whether it's read-only, requires authentication, or possible side effects. Minimal transparency beyond the operation's nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description front-loads the purpose but then becomes verbose with a long list of items. While the list is useful, it could be better structured (e.g., grouped by type) for readability. The length exceeds what is necessary for a concise description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description does not detail the return value structure. However, 'JSON Schema' is self-explanatory for those familiar. The comprehensive input list provides completeness for parameter values, but missing output info slightly reduces overall completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameters, and description adds significant value by listing all valid values for the 'name' parameter (e.g., verbs, components, callbacks, guides), which goes beyond the schema's brief example. This enriches parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get the JSON Schema for a jambonz verb or component.' It identifies the specific verb-resource relationship and distinguishes the tool from siblings like get_sdk_example (examples) and list_jambonz_tools (tools listing).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool over alternatives. Implicitly, if you need a schema, use this; but no mention of prerequisites or when not to use. Does not exclude alternative uses.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_sdk_exampleA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states 'Get full source code', which is a safe read operation. It does not disclose rate limits or auth, but for a simple retrieval tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence listing examples, which is concise but could be better structured (e.g., bullet list). No redundancy, but front-loading could improve scanning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter, no output schema, and no annotations, the description covers purpose and available options adequately. It lacks details on return format but is sufficient for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with the 'name' parameter described and listing examples. The description repeats the same list, adding no new meaning beyond the schema. Baseline is 3 due to high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get full source code for an SDK example' with a specific verb and resource. It also lists all available examples, making the tool's purpose unambiguous. It is distinct from sibling tools like get_jambonz_schema or list_jambonz_tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need SDK example source code, but lacks explicit guidance on when to use versus alternatives or exclusions. Since siblings are unrelated, the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• jambonz_developer_toolkitA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Describes the output but does not discuss side effects, auth needs, or state modifications. For a read-only knowledge retrieval tool, the disclosure is moderately adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with key requirement, no wasted words. Perfectly concise and structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and no output schema, the description thoroughly describes the returned items (guide, schemas, tools). Could mention output format but is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so baseline is 4. The description adds value by explaining the output content beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the developer guide, schemas, and pre-built tools. It distinguishes from siblings by being a prerequisite and advises checking pre-built tools before implementing custom handlers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'REQUIRED: call this before writing jambonz code' and 'check these before implementing custom toolHook handlers,' providing clear when-to-use guidance. Does not directly name sibling tools as alternatives but implies alternatives exist.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_jambonz_toolsA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided but the description fully discloses behavior: it lists tools with their schemas. The tool is read-only and non-destructive, appropriately conveyed for a listing operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences: purpose, usage guidance, and enumeration. Front-loaded with the core action, no redundancy, every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and no output schema, the description is fully complete. It tells the agent what to expect (list of tools with schemas) and includes actionable advice, leaving no gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has zero parameters, so description's mention of listing tools with schemas adds meaningful context. It emphasizes the value of the output without needing parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: listing pre-built tools from @jambonz/tools with full parameter schemas. It uses a specific verb 'list' and resource 'pre-built tools', distinguishing it from sibling tools like get_jambonz_schema and get_sdk_example.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises to check here before hand-rolling a toolHook handler, encouraging use of ready-made tools if one fits. Also lists available tools, providing clear context for when to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.