Server Quality Checklist

Tool Scores

• editC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Minimal behavioral disclosure: mentions page_break emits \clearpage and lint check for replace_raw. Lacks details on side effects, error conditions, auth needs, or success/failure behavior. No annotations to supplement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Relatively concise with bulleted actions, but structure could be improved by front-loading key details and grouping parameter descriptions. Current format is acceptable but not optimal.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given high complexity (18 params, no annotations, output schema exists but not described), the description is insufficient. It fails to explain many parameters, return values, or provide comprehensive usage context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage and 18 parameters, the description only explains a few (action, section, position, page_break). Most parameters like content, title, level, headers, rows, etc. are not described, leaving significant gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Manipulate document content structurally' and lists specific actions. The purpose is clear but does not explicitly differentiate from sibling tools like 'document', though the action list distinguishes it reasonably.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives (e.g., 'document' for reading). The description lists actions but does not explain contexts or situations where each action is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• layoutB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses the tool is a configuration tool (non-destructive implied) and explains the partial update behavior, but does not describe return values, side effects, or authentication needs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences, front-loaded with the main purpose, and each sentence adds value. No redundant or verbose phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 19 optional parameters, no annotations, and an output schema that presumably describes return values, the description covers basic behavior but lacks detail on many parameters and usage context. It is minimally adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description only adds meaning to the 'section_break' parameter with a detailed explanation. The other 18 parameters rely solely on their names for interpretation, which is insufficient given no enum values or descriptions in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool configures document typesetting and layout, using a specific verb ('Configure') and resource. While it doesn't explicitly differentiate from siblings like 'edit' or 'document', the purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description notes that only provided parameters are changed (a usage hint), but does not specify when to use this tool versus alternatives like 'edit' for page breaks. It implies usage for layout configuration but no explicit when-not-to.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• documentB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It does so partially: 'reset: Clear the current document and saved state' reveals state mutation, and 'update: Update document metadata' implies a side effect. However, it does not specify whether actions are irreversible, require authentication, or have rate limits. The description adds some behavioral context but is not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a front-loaded summary followed by a bullet list of actions. Each action has a concise one-line description. The length is appropriate for the complexity (multiple actions). Minor redundancy ('Create, ingest, and inspect documents' and then each action) but overall efficient and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multiple actions, 8 parameters, no annotations), the description covers actions but not the full context. It lacks return values for each action (though output schema exists), error handling, and lifecycle guidance (e.g., document state before actions). The description is adequate but has notable gaps in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It links some parameters to actions: e.g., 'create' implies document_class, title, author; 'read' uses section. This adds meaning beyond the raw schema. However, it does not explain all parameters (e.g., date, abstract for update) or provide formats/constraints. The description partially fills the gap but leaves several parameters underspecified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs and resources: 'Create, ingest, and inspect documents.' It lists distinct actions (e.g., create, ingest, read, update, reset) that define its scope, and the verb 'document' differentiates it from sibling tools like edit, layout, and render.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives or when not to use it. It lacks prerequisites, context for action selection, or exclusions. The list of actions implies usage context but does not clarify ordering or state dependencies (e.g., must create or ingest before reading).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• renderA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must convey behavioral traits. It discloses return values for each action (PDF path, file path/dimensions, LaTeX content) but omits potential side effects (e.g., file overwriting, permission requirements) or dependencies on prior document state.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, front-loading the overall purpose and using a clear bulleted list for each action. Every sentence adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's multiple actions and lack of annotations, the description covers return values but not parameter usage or contextual dependencies. The presence of an output schema (not shown) may mitigate completeness needs, but gaps remain in parameter semantics and behavioral context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, leaving the description to explain parameters. It does not explicitly map parameters (output_path, page, dpi) to actions; the action names hint at their use but lack explicit constraints or format details beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Compile and export the document', and enumerates three specific actions (compile, preview, tex) with distinct functionalities. It differentiates well from sibling tools (e.g., document, edit) that handle other aspects of document management.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through action descriptions but does not explicitly state when to use this tool versus alternatives or exclude scenarios. No guidance on prerequisites or conflict with other tools is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• queueA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description must disclose behavior. It mentions sequential execution, single disk write, and error stopping. But it does not cover rollback, side effects, or partial execution details, leaving some behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with a clear front-loaded purpose, followed by behavioral details and a well-structured example. No extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists (context signal), the description does not need to cover return values. It sufficiently explains the two input parameters and the batching concept, making it complete for a tool of moderate complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description adds significant meaning: it provides an example for the operations parameter explaining the dict structure, and clarifies continue_on_error with a default behavior. This compensates well for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Execute multiple operations in a single call' and provides a detailed example showing how to batch operations from sibling tools. It distinguishes itself by acting as a batching mechanism, not a specific tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that operations run sequentially and disk is written once at the end, implying efficiency for multiple related tasks. It also describes the continue_on_error parameter. However, it lacks explicit when-to-use or when-not-to-use guidance relative to calling individual tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• referenceA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It reveals that the tool performs lookups and analysis (e.g., check_style) but does not explicitly state that it is read-only or have side effects. The listed actions imply safe query behavior, so transparency is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the overall purpose, followed by a clear bulleted list of actions. Every sentence adds value, and the structure is easy to parse for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (7 parameters, multiple actions, presence of output schema), the description covers the main behaviors adequately. It could include more explicit parameter-action mappings and return value hints, but the output schema likely compensates for the latter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must compensate. It maps actions to likely parameters (e.g., symbol uses description, error_help uses error), providing meaningful guidance. However, not all actions have clear parameter mapping (e.g., styles), preventing a perfect score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches LaTeX documentation, symbols, packages, and error solutions. It lists distinct sub-actions, establishing it as a multi-purpose reference tool distinct from siblings like document, edit, layout, queue, and render.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description enumerates specific use cases (search, symbol, package, etc.), allowing an agent to infer when to invoke this tool. However, it does not explicitly state when not to use it or compare with alternatives, which would earn a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.