Server Quality Checklist

Tool Scores

• list_duplicatesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the responsibility for behavioral transparency. The description explains the classification outcome (accepted layered pairs vs genuine smells) and the consequence (explicit import needed). It does not disclose details like return format or whether it filters based on the 'smells_only' parameter beyond what the schema implies, but the behavior is reasonably described.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, concise and front-loaded with the core purpose. It includes necessary context (accepted layered pairs definition) and consequence (explicit import). No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple with one optional parameter, and the description covers the purpose and outcome. An output schema exists, so the description does not need to explain return details. The main gap is the lack of explanation for the 'smells_only' parameter, but overall it is adequate for its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one parameter, 'smells_only' with a default of false. Schema description coverage is 0%, so the description must compensate. The description does not explain the parameter's effect (e.g., filtering results to only genuine smells). The baseline is 3 given the single parameter with default, but the description adds little about its semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: listing duplicate class names and distinguishing between accepted layered pairs and genuine smells. It also explains the implication (explicit import needed). Distinguished from siblings because it focuses on duplicates, while siblings like search_classes or get_usages serve different functions, but does not explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool: when checking for duplicate class names and needing to know which duplicates are acceptable vs smells. It provides the criterion for accepted layered pairs (ORM model + domain model sharing a name across layers). It doesn't explicitly state when not to use it, but the purpose is clear enough to guide selection among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_snippetA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It describes a 'check' but does not explicitly state whether it is read-only (no side effects), what happens on success/failure, or if any state is modified. The note about disambiguation is useful, but the lack of explicit side-effect or return behavior leaves room for assumptions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, with only two sentences. The first sentence states the purpose and timing, and the second adds a key parameter hint. No redundant or unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, 0% schema coverage), the description adequately explains the main purpose, the file_path parameter's purpose and when to use it, and the context (before writing). It does not describe the return value, but an output schema exists, so that is presumably covered there. The description is sufficient for an AI agent to understand when and how to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, and the description adds meaning to the 'file_path' parameter by explaining its role in disambiguation. However, it does not clarify the 'code' parameter beyond context (it is inferred as the snippet content). The description compensates partially but not fully for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: verifying a draft snippet's constructor calls against a registry before writing to a file. It uses a specific verb ('check') and a specific resource ('draft snippet's constructor calls against the registry'). It distinguishes from sibling tools (e.g., get_signature, list_duplicates) by focusing on validation pre-write, though the term 'check' could be more explicit about the action's outcome.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides usage context by saying 'before writing it to a file' and instructs to pass the file_path when known, explaining that imports and target location disambiguate duplicates. However, it does not explicitly mention when not to use this tool or compare it to alternatives, so it earns a 4 rather than 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_signatureA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that it reports arguments, keywords, and null for unknowable constructors, implying a read-only operation. However, it does not explicitly rule out side effects or discuss error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences and fairly concise, but the phrase 'EVERY definition if the name is duplicated' is ambiguous and could be streamlined. Overall, it's not overly verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has one required parameter that is not described in the schema or description. While the output schema likely covers return values, the input format and handling of duplicate names are left unclear. This makes the description incomplete for a simple but potentially ambiguous use case.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, and the description does not explain what class_name should be (e.g., fully qualified vs. simple name). The mention of duplicate definitions hints at disambiguation needs but does not clarify how to specify which class. This is a significant gap for a one-parameter tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states it returns a constructor contract (required arguments, accepted keywords, null for open-ended). This clearly distinguishes it from sibling tools like get_usages or search_classes, which focus on other aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Call this before writing an instantiation,' giving clear timing guidance. It does not mention when not to use it or alternatives, but the context is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_classesA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavior burden. It discloses case-insensitivity and the fuzzy-match fallback, which are non-obvious and valuable. It does not mention result ordering or empty-result behavior, but the presence of an output schema likely covers return structure. It adds meaningful behavioral insight beyond the bare schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tightly packed sentences. The first front-loads the primary action and key behaviors (case-insensitive, fuzzy fallback), and the second gives usage guidance. No filler or redundant phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a straightforward search tool with an output schema present and only two params, the description is nearly sufficient. It explains the search behavior and primary use caseable gaps: it doesn't mention behavior when no results match beyond the fallback, or any limit semantics. These are minor for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It implies 'query' is a name fragment via 'Find classes by name fragment', giving some meaning to that parameteraine, but it never mentions the 'limit' parameter or its semantics. The description covers only one of two parameters, leaving the other undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (Find), the resource (classes), and the search mechanism (name fragment, case-insensitive). It also distinguishes itself from siblings by framing the operation as a fuzzy search fallback, which an agent can immediately differentiate from listing or other operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit when-to-use guidance: 'Use this when unsure of the exact class name.' However, it does not explicitly name alternatives or state when NOT to use it. This leaves exclusion logic to the agent's judgment, preventing a top score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• server_infoA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description must carry the full transparency burden. 'Report' implies a read-only, non-mutating behavior, which is helpful, but it does not specify whether it hits a live endpoint, any error semantics, or output shape. It is not misleading, but it is minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence with a front-loaded verb and concise object list. There is no filler, and all three reported elements are enumerated efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters, the presence of an output schema, and the high-level nature of the report, the description is fully adequate for an agent to select and invoke this tool. It covers the main informational content, and no missing detail would confuse a capable agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero properties, so the description does not need to explain parameters. It profitably defines what the tool reports (status, project, registry size) and thereby gives the agent meaningful context beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('Report') with a concrete resource ('server') and specifies the exact output categories: status, project, and registry size. This distinguishes it unambiguously from the code-focused sibling tools such as get_signatures or list_duplicates.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use is implied—call this when you need server status, the associated project, or registry size—but no explicit guidance is given about when to prefer it over alternatives or what action it might precede. There are no usage conditions, exclusions, or context cues beyond the obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_usagesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool includes usages through import aliases and that it should be run before a rename, which implies it is a read-only operation. However, it does not mention pagination or the limit parameter's behavior, which is a minor gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, front-loaded with the core purpose, and every sentence adds value. It uses a clear example of import aliases and a practical instruction (run before rename) without unnecessary fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 parameters, output schema present), the description covers the main use case and provides actionable guidance. It lacks details on limit and include_tests, but the output schema likely clarifies return values. The description is sufficient for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains the class_name parameter implicitly by describing what usages are found, but it does not explain the limit or include_tests parameters. The description adds some context for class_name but leaves the other two parameters undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to find every place a class name is used, listing specific usage types (imports, inheritance, instantiations, annotations, references) and including import aliases. It distinguishes itself from siblings by focusing on usages rather than searching or listing classes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to run this tool BEFORE a rename to enumerate what must change, providing clear when-to-use guidance. It also implies when not to use it (e.g., for searching classes, which is handled by search_classes).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.