Server Quality Checklist

Tool Scores

• server_infoA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds meaningful behavioral context beyond the readOnlyHint annotation, notably that it never returns credentials and that it verifies whether the connection works. This reassures an agent about safety and expected operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, stating the primary purpose first, then the practical usage hint, then the safety guarantee. Every sentence contributes useful information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read-only informational tool, the description covers what the agent needs: what is reported, how to use a reported value, and a safety boundary. No output schema exists, but the description sufficiently enumerates the expected information categories.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool accepts no parameters, and schema description coverage is 100%, so there is no parameter documentation burden. The description appropriately focuses on what the tool reports rather than parameter details that do not exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool does: report connection details for the Matrix42 instance, the authenticated account, and connection status. This distinguishes it from the sibling tools, which involve discovery and data querying, by making the server/account introspection purpose explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context for when the tool is relevant, especially the instruction to use the reported user fragment id for answering 'my items' questions. It does not explicitly list exclusions or compare with sibling tools, but the use case is sufficiently clear for an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• data_queryA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Adds rich behavioral context well beyond the readOnlyHint/openWorldHint annotations: ID is auto-added to queries, DisplayString is returned automatically and 'cannot be selected explicitly', 'without [sort], page boundaries are not stable', validation is cheaper than a failed query, and guessed attribute names fail. deep_link's verify parameter behavior is disclosed in the schema, and the description's read-only claim is consistent with readOnlyHint=true. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Long (~400 words) but proportionate to a 10-action, 21-parameter tool; the core read-only purpose is front-loaded and the action enumeration is compact. Every clause earns its place, though it could be tightened into a structured list rather than a dense paragraph.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Very complete for a tool this complex: covers 9 of 10 actions with prerequisites, failure modes ('guessed names fail'), and best-practice warnings, while the query action's return shape ('rows... with typed column metadata') is at least named despite no output schema. The sole gap is action='deep_link', absent from the description; its behavior is partially recovered by the schema's parameter descriptions (verify, view_type, embedded), so the omission is not disabling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with already-detailed parameter descriptions (examples for where, enum meanings for view_type, verify's default and opt-out case), so the baseline is 3. The description adds value above that by giving each value of the central 'action' parameter a purpose clause and by stating cross-parameter rules — pass sort whenever you page; never request columns not reported by schema_discovery — that the schema alone does not convey.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Opens with a specific verb and resource — 'Read records from the connected Matrix42 instance (read-only)' — and enumerates its nine named sub-actions (asql_guide, validate_asql, query, get_fragment, get_object, list_views, run_view, list_journal, list_attachments), making the tool's scope unmistakable. The read-only data-access framing implicitly separates it from schema discovery and service operations, so an agent can tell what this tool is for at a glance.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives explicit when-to-use guidance with named alternatives: read asql_guide 'before writing a filter', validating 'is cheaper than a failed query', and 'prefer a matching view over hand-written ASQL' for run_view. It also names a sibling tool as a prerequisite — 'NEVER guess attribute names: read them with schema_discovery(describe_data_definition)' and use get_pickup_values for pickup filtering — which properly routes the agent across the toolset.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• webservice_discoveryA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint annotation, the description details per-action behavior, including return shapes for list_operations, the nature of api_overview content, and the required inputs for describe_operation. It also clarifies that the tool is metadata-only and references authentication conventions as discoverable content, providing meaningful behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but well-structured, front-loading the core purpose and then systematically covering each action, its parameters, and the typical flow. Every sentence carries actionable information without repetition or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a multi-action discovery tool with no output schema, the description is exceptionally complete: it defines all actions, their return content, parameter usage, and a recommended call sequence. An agent has enough information to select the right action and parameters for a discovery task.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents all five parameters with a high coverage level, so the baseline is 3. The description adds value by mapping each parameter to the action it affects, explaining search and service_id filtering semantics, and noting operation_id's requirement for describe_operation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies a specific verb ('Discover'), a specific resource ('Matrix42 REST API of the connected instance'), and its read-only metadata nature. It enumerates four distinct actions with concrete outputs, so an agent can understand the tool's scope and distinguish it from data-query or schema-related siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when this tool is useful: reasoning about the API or writing standalone integration code. It also provides a typical discovery flow, but it does not explicitly name sibling tools or state when not to use it, so it stops short of a full when/when-not comparison.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• schema_discoveryA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and openWorldHint, and the description reinforces read-only status in the first sentence. It adds valuable behavioral context: list_data_definitions excludes pickups by default, describe_data_definition defaults to attributes because relations can exceed a hundred, and get_pickup_values resolves pickups via two alternative routes. This goes well beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but perfectly structured: a one-sentence scope statement, a per-action breakdown, and a closing typical flow. Every sentence adds information, and the most important routing guidance ('read this first') is placed early. No filler or redundant restatement of the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a multi-action tool with six distinct operations and eight parameters, and the description covers every action's purpose, the parameters each action consumes, defaults, and a recommended sequence. With no output schema, the description still tells the agent what kind of result each action returns. Nothing needed for correct invocation is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds substantial meaning beyond the schema: it explains the purpose of each action, which parameters apply to which action, the default behavior of include, and the two ways to resolve pickup values. The 'read this first' guidance for schema_overview and the typical flow description give parameters operational context the schema alone lacks.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a precise statement of what the tool does: 'Explore the Matrix42 data model of the connected instance (read-only metadata).' It then enumerates six concrete action verbs and their resources, making the tool's scope unmistakable and distinguishing it from the data-querying purpose implied by siblings like data_query.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit internal routing: schema_overview should be read first, followed by the typical flow overview → list_* → describe_* → get_pickup_values. It also explains when to use pickup_class versus name plus attribute. It does not explicitly contrast against sibling tools, but the read-only metadata framing and detailed action-level guidance are strong enough to guide selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• service_deskA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool as readOnly and openWorld, and the description reinforces this ('Read the service desk'). It adds substantial operational nuance beyond the annotations: Matrix42 silently ignores certain filter parameters and the tool refuses them, get_ticket answers null for fragment ids, fields absent from a module are reported as unavailable rather than failing, and search_tickets covers all ticket kinds through one contract.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Long but justifiably dense: seven actions, thirteen browse domains, and multiple caveats are packed into a well-organized, front-loaded description. Each sentence carries operational value, from 'read it first if you are unsure' to the closing note about per-instance column resolution. There is no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a polymorphic read-only tool with 22 parameters and no output schema, the description is remarkably complete. It covers every action, the domain list for browse, the relationship to data_query, the exact filter caveat, id-type semantics for get_ticket, and instance-specific schema resolution. The sibling tools are addressed where relevant, and the absence of an output schema is mitigated by explaining what each action returns or how failures surface.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3; the description builds on that by clarifying which parameters actually do something ('Only subject, category_name and states actually filter'), which parameters are refused, and how action-specific parameters map to actions like sla_times or get_ticket. It does not exhaustively walk through all 22 parameters, but the schema already describes each one, so this is appropriately complementary.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear 'Read the service desk and the business objects around it' and then enumerates each action with a specific verb and resource: search_tickets, get_ticket, sla_for_ticket, sla_times, browse, find. It explicitly distinguishes find from search_tickets ('It does not cover tickets') and points to data_query for person/ticket-number/asset filtering, so an agent can tell it apart from sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives direct routing guidance: read data_model first if unsure where something lives, use find when you do not know where something lives, use search_tickets for tickets, and use data_query with an ASQL where clause to filter on person, ticket number, or asset. It also warns which parameters are ignored/refused, preventing wasted calls.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.