Server Quality Checklist

Tool Scores

• manic_save_to_projectA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses required authentication (MANIC_API_KEY with specific scopes) and revision behavior on existing paths. However, it does not mention the return value on success/failure, potential side effects, or whether the operation is idempotent. The disclosure is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long with zero wasted words. The first sentence states purpose and outcome; the second adds revision behavior and auth requirements. Every sentence earns its place, and no irrelevant details are included.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has two mandatory parameters, no enums, no nested objects, and no output schema. The description covers purpose, effect, revision behavior, and auth needs. One minor gap: it does not describe what the agent should expect as a response or how to confirm success, but for a save tool this is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% – both parameters (path, source) are documented with descriptions and constraints in the schema. The description does not add any additional meaning beyond what the schema provides, meeting the baseline expectation for high-coverage schemas.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool saves '.manic source as a document in the user's default Manic project' and explains the effect (appears in Manic Create and Workbench). The verb 'saves' and resource '.manic source' are specific, and the purpose is distinct from all sibling tools (manic_check, manic_render, etc.), which perform different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool (to save a .manic file to the project) and provides important context like revision behavior on existing paths. However, it does not explicitly state when not to use it or suggest alternatives among siblings, though the sibling tools are sufficiently different that no confusion is likely.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manic_render_statusA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full weight. It clearly identifies the operation as a read (durable status) and specifies the output when succeeded. It also includes a rate-limiting guideline (poll interval). Could be improved by mentioning possible error states.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place: first defines purpose and output, second adds polling guideline, third states auth requirements. Likely efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple status check tool with one parameter and no output schema, the description covers purpose, prerequisites, behavioral constraint (poll interval), and output on success. Missing details about possible intermediate statuses or error responses, but overall sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single parameter job_id, so baseline is 3. The description does not add additional semantic depth beyond what the schema already provides (i.e., it is the job ID returned by manic_render).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads the durable status of a render job and returns the artifact URL on success. It distinguishes from siblings like manic_render (which starts jobs) and manic_save_to_project (which persists results).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises polling no more than every 10 seconds and mentions the required API key and scope (jobs:read). However, it does not explicitly describe when to avoid using this tool or compare with the sibling `manic_check`.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manic_checkA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses the need for MANIC_API_KEY (scope: check) and signals that checking is cheap (low cost). While it doesn't explicitly state idempotency or read-only nature, the verb 'validates' and the contrast with rendering imply no side effects. Minor gap but generally transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences: purpose, usage guidance, and requirement. No wasted words. The most critical info (what it does, when to use) appears first. Perfectly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given one simple parameter and no output schema, the description covers purpose, usage, and a key requirement. It does not describe the return format of diagnostics (e.g., list of errors with severity), which could be helpful but is not critical for selection. Slightly incomplete but adequate for a straightforward validation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There is only one parameter with 100% schema description coverage. The tool description does not add extra meaning beyond what the schema already states ('Complete .manic source to validate'). Baseline score of 3 is appropriate since schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool validates '.manic source' against the Manic platform's checker and returns diagnostics. It distinguishes itself from siblings by contrasting with rendering (which spends credits), making the purpose highly specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance: 'Always check before rendering' because checking is cheap while rendering costs credits. It also instructs to 'Fix every error diagnostic and re-check until clean.' This clearly defines when and why to use the tool, with no ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manic_renderA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are absent, so the description carries the full burden. It clearly discloses the asynchronous nature (returns job id immediately, rendering can take minutes), cost implications (spends export credits), and prerequisites (MANIC_API_KEY with scopes). It could mention idempotency or error behavior but covers the key behavioral traits effectively.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place: first defines action and async return, second sets usage context and cost, third lists auth requirements. No redundancy, front-loaded with key info, and efficient for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (async, cost, auth) and the absence of output schema, the description provides sufficient guidance for expected usage and follow-up (polling). It doesn't cover error cases or response format details, but the tool's return (job id) is implied. A minor gap is the lack of instruction on how to interpret or use the job id beyond polling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description does not add details beyond the schema; for example, it doesn't explain 'plan ceiling' or valid values for 'canvas' beyond the example. However, it contextually links the tool's purpose to parameters like 'source' needing validation, which adds minor value, keeping score at baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'submits' and the resource 'validated .manic source to the Manic hosted renderer'. It distinguishes from siblings by mentioning 'manic_render_status' for polling and 'manic_check' for validation, providing explicit differentiation from other tools with similar themes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidelines: use 'manic_check' first for validation, then this tool only when user wants output due to credit cost. It specifies when to render and when to poll with 'manic_render_status', creating a clear workflow. No sibling tool is an alternative for rendering itself, so exclusion is handled well.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manic_authoring_guideA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It fully discloses essential behavior: the tool has zero parameters, requires no authentication, returns a complete guide, and is safe to call repeatedly despite being designed for one-time use. No behavioral surprises.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, zero waste. Every sentence serves a distinct purpose: what it does, how to use it, and behavioral note (no auth). Front-loaded with the verb and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters, no output schema, and no annotations, the description is complete. It fully explains the tool's purpose, usage pattern, and behavior. There are no missing details — the agent can use this description to correctly select and invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with zero parameters, so the baseline is 4. The description adds no parameter-level detail because there are none, but it clarifies the nature of the returned content (complete guide, source of truth), which serves as context for what the tool provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Fetches the complete, current Manic language authoring guide'), identifies the exact resource (the system prompt Manic's own AI uses), and distinguishes itself from sibling tools by being a one-time reference tool, unlike manic_check, manic_render, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance: call ONCE before writing any .manic source, follow it exactly, and explains why (Manic is a precise DSL and the guide is the source of truth). This implicitly advises against calling it repeatedly or relying on alternatives for authoring rules.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.