Server Quality Checklist

Tool Scores

• searchC
  2.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, so safety is covered. The description adds useful behavioral context: 'Source symbols are evidence only', 'executable=true only for Molexp bind', and 'sources is intersected with knowledgeScope when set'. These go beyond annotations, but the phrasing is technical and ambiguous.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short, which is efficient, but the cryptic phrasing (e.g., 'Molexp bind', 'evidence only') makes it less communicative than a clearer, slightly longer explanation would be. It is not front-loaded in a way that aids quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 7 parameters, 0% schema coverage, and an output schema present, the description is insufficient for an agent to correctly call the tool without external context. It doesn't explain parameter formats, defaults, or the meaning of 'knowledgeScope', leaving critical gaps for a complex tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With schema description coverage at 0% and 7 parameters, the description must compensate for the missing schema documentation. It only touches on 'sources' (via 'source=' and the intersection note) and leaves mode, path, kinds, limit, and namespaces completely unexplained. This is a significant gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies the tool as an 'Index helper' to 'find refs', which gives a specific verb and resource, but it doesn't clarify what kind of references or how they differ from siblings like 'outline' or 'packages'. The purpose is distinguishable from 'info' or 'suggest' but the wording is cryptic and assumes domain knowledge.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It says 'prefer after packages/outline, with source=' which is a clear recommendation about when to use it relative to siblings, and hints at a required parameter. However, it doesn't explicitly state when NOT to use it or when alternatives are better, leaving the guidance incomplete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• outlineB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already disclose read-only, idempotent, non-destructive behavior, so the description's burden is lower. It adds a small amount of scope context ('one source', 'optional path prefix') but does not explain the meaning of 'L1' or how top_symbols_limit affects results. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short and front-loaded; every word earns its place. It is not a perfect 5 because it is a sentence fragment and depends on the unexplained 'L1' abbreviation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with full schema descriptions, annotations, and an output schema, this is minimally adequate. It lacks usage guidance and contextual semantics around what 'L1' means and what the returned directory/page actually contains.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description only echoes the 'optional path prefix' from the schema and adds no additional meaning for source or top_symbols_limit.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies the resource ('module directory'), the scope ('for one source'), and a key option ('optional path prefix'). However, it is a noun phrase rather than an explicit verb+resource statement, and 'L1' is unexplained, leaving some ambiguity about exactly what is returned.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to choose outline over sibling tools like info, packages, or list_planes. 'for one source' hints at scope but does not state conditions, exclusions, or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• composeB
  3.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnly, openWorld, idempotent, non-destructive), the description discloses a concrete behavioral trait: sources is intersected with knowledgeScope when set. This adds useful operational context that the annotations do not provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with the core purpose and followed by a critical operational caveat. No filler or redundancy; every sentence contributes information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with four undocumented parameters, no usage guidance, and an output schema that does not compensate for input semantics, the description is incomplete. An agent could guess the overall purpose but would not know how to set refs or task, or how budget_chars affects the composed result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain parameter meaning. It only mentions 'sources' and indirectly hints at 'budget_chars' through 'budgeted pack'; 'refs' and 'task' remain completely unexplained, leaving the agent without enough information to populate the inputs confidently.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description identifies a specific composite action: binding packages, suggestions, and explore/open operations into a single budgeted pack. This clearly differentiates it from individual siblings like search, suggest, and open, though the phrase 'explore/open pages' is somewhat informal.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is used when multiple operations should be combined into one budgeted pack, but it does not explicitly state when to use it versus calling the sibling tools individually, nor does it mention exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• infoA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, non-destructive, and open-world behavior, so the safety profile is covered. The description adds only a conceptual Ops/health framing and a discovery exclusion, not operational behaviors like scoping or result characteristics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with the key purpose, and the exclusion is in parentheses so the core message is immediate. No filler or redundant restatement of the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple optional-parameter tool with rich annotations and an output schema, the description gives enough positioning: Ops/health, sources and registry, not the discovery path. A small gap is the meaning of 'workspace' and what 'sources and registry' refer to, but the schema and output schema fill part of that.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description does not mention the optional 'workspace' parameter or how it affects the result. The parameter name is somewhat self-explanatory, but the description does not compensate for the missing schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies the tool as an 'Ops/health view of sources and registry,' which conveys a clear subject and scope. It also distances itself from 'the main discovery path,' helping an agent distinguish it from search-oriented siblings like search and suggest, though it does not name a specific alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says this is not the main discovery path, which tells an agent when not to use it and suggests discovery is handled elsewhere. The wording provides clear context for an ops/health use case, though it does not explicitly point to a sibling for discovery.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• routeA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the description doesn't need to restate those. It adds useful behavioral context: this tool only routes and does not perform science, and it warns against inventing tools. There is no contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the core purpose. The additional sentences about routing-only and not inventing tools are purposeful guardrails. It is slightly cryptic but contains no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's role and key constraints, and an output schema exists, but the task parameter semantics are underspecified. An agent may understand the routing concept yet still lack enough detail to invoke it correctly with real task values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain the 'task' parameter, but it only mentions '*task*' without defining what values it accepts, expected format, or how it determines routing. This is a meaningful gap for a single required parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool's function: decide which optional provider plane(s) to connect for a task. It also disambiguates itself with 'Routing only — no science' and warns against inventing domain MCP tools, which helps separate it from science/chemistry-related siblings. However, 'provider plane(s)' is left somewhat vague and no sibling is explicitly named.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives clear context for when to use the tool: when routing/connecting provider planes, not for doing science. The note 'molcrafts is already this connection' implies a condition where routing may not be needed, and 'Do not invent domain MCP tools for chemistry APIs' is an explicit exclusion. It does not name alternative sibling tools, but the guidance is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• suggestA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already establish read-only, idempotent, open-world, non-destructive behavior. The description adds only that this is a shortcut for page selection and does not explain what the result set looks like; no contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One short, front-loaded sentence with no filler; every word adds either scope or usage context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, has one required parameter, an output schema, and annotations covering safety/behavior. The main remaining gap is the lack of an example or clarification of what a task looks like, but for its complexity the definition is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% coverage and just names `task` as a string. The description clarifies that `task` is the input driving the page suggestions, which is meaningful, but it does not specify format or expected phrasing; this partially compensates for the empty schema description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool recommends which package pages to read for a given task, naming a concrete resource and purpose. It does not explicitly contrast with any sibling tool, so it lacks the sharp differentiation seen in the best definitions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  'Optional shortcut' implies this is a convenience alternative to a fuller workflow, and 'for *task*' describes the trigger. However, there is no explicit when-to-use or when-not-to-use guidance relative to siblings like search or packages.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• openA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover read-only, idempotent, and non-destructive behavior, so the bar is lower. The description adds valuable context beyond annotations: miss responses return ok=false/SYMBOL_NOT_FOUND, and empty examples are meaningful rather than missing data. No contradiction with annotations exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely compact: two sentences that front-load the core purpose and timing, then add failure and edge-case semantics. Every sentence earns its place with no filler or redundant restatement of the title.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides the core purpose, timing, failure behavior, and empty-example semantics, while annotations and output schema cover safety and return shape. However, the undocumented parameters remain a notable gap: the agent is left guessing what format ref should take and what include_source controls.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the lack of parameter documentation, but it does not. The meaning of 'ref' is only loosely implied by 'L2 symbol page', and 'include_source' is completely unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies the tool as opening an 'L2 symbol page' with signature, doc, examples, and tests, which is much more specific than the generic title 'Open'. It still lacks an explicit verb and does not differentiate from siblings like info or outline, but the resource and contents are clearly stated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a concrete use context with 'inject before coding', telling the agent when to invoke it. It also documents the miss behavior, but it does not state when not to use this tool or explicitly compare it to sibling tools such as search or outline.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• packagesA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, openWorld, idempotent, and non-destructive behavior. The description adds useful behavioral details beyond those: it is a catalog page for context injection, includes every package plus summary, is not a ranked result, and is scoped by knowledgeScope. This gives the agent a clearer behavioral model of what the tool returns and how to treat the output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the core purpose. The second and third lines add meaningful guidance about treating the output as a catalog and about knowledgeScope filtering. Slight jargon like 'L0 directory page' costs a little clarity but does not seriously hurt the message.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only directory tool with an output schema and rich annotations, the description conveys the essential information: what is returned, how to use it, and what changes when knowledgeScope is set. It could be more explicit about the exact output shape, but the presence of an output schema reduces that burden. Minor ambiguity about where knowledgeScope comes from prevents a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description mentions knowledgeScope as a scoping influence, which gives additional semantic context even though it is not a declared schema parameter. There is no param ambiguity to resolve since the input schema is empty.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states this is a directory page listing every package with a summary, which clearly identifies it as a catalog/list operation rather than a search or detail view. It uses resource-specific language (packages, data.packages[].summary) that distinguishes it from sibling tools like info or outline, though it lacks an explicit verb like 'list'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives clear usage context: read the markdown or summaries and choose sources yourself, emphasizing this is a catalog rather than a ranking. It also explains behavior when knowledgeScope is set. It does not explicitly name sibling alternatives or state when not to use it, but the 'not a ranking' guidance and knowledgeScope condition provide meaningful selection cues.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_planesA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already disclose readOnlyHint, idempotentHint, and destructiveHint, covering safety profile. The description adds valuable behavioral context beyond annotations: it explains that molcrafts is always on and only provider planes can be dropped, which informs the agent about data invariants. No contradiction with annotations; the added context is relevant and useful.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and front-loaded with the purpose. The first sentence states what it does; the second explains row structure and a key rule. Every sentence contributes to understanding, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a no-parameter list tool with an output schema (not shown), the description provides sufficient context: it enumerates the fields of each row and clarifies the distinction between core and provider planes. An agent can call this tool without ambiguity about inputs or expected outputs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With zero parameters, the baseline is 4 per the rubric. The description doesn't need to explain parameters; however, it does describe the output fields, which indirectly clarifies what the tool returns. No compensation needed for missing parameter info.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists 'the core connection and optional provider planes' with a distinct resource and scope. It differentiates from siblings by specifying the exact content, and the name 'list_planes' aligns with the purpose without tautology.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention any sibling tools, conditions for substitution, or context that would help an agent decide between list_planes and other operations. The information given is purely descriptive, lacking explicit usage or exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.