Server Quality Checklist

Tool Scores

• mylar_issues_queueA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the full burden of explaining side effects, but it only discloses potentially significant behavior for two operations ('may snatch/download' for force_search and queue_issue). Other mutating operations, like mylar_force_process and mylar_unqueue_issue, do not mention permissions, reversibility, external effects, or other behavioral consequences.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than average but necessarily so given the seven distinct operations. The per-operation bullet list with signatures and short comments is well-organized and scannable, with no significant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the high complexity (seven operations), minimal input schema, and no annotations, the description does a solid job of covering the purpose and key parameters of each operation. An output schema exists, so not explaining return values is acceptable, but a few operation side effects remain under-specified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is generic (operation plus a free-form arguments object), so the description's detailed function signatures add substantial meaning. It lists parameter names and provides allowed values for several (e.g., comicid: single, list, 'all', 'missing', or 'refresh-missing'), although some optional parameters like ddl and oneoff remain unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies this as the Mylar3 issues-queue tool and enumerates seven specific operations (e.g., mylar_force_process, mylar_queue_issue) with concise explanations. This distinguishes it from sibling tools like mylar_comics and mylar_providers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Each operation is accompanied by a brief usage context, such as 'Used by SABnzbd/NZBGet-style callbacks' and 'Trigger a wanted-issue search across the library.' It does not explicitly say when not to use the tool or name alternatives, but the per-operation guidance gives clear situational context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mylar_lists_discoveryA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must handle behavioral disclosure, and it does disclose parameter-specific behaviors (e.g., customOnly='1', include_downloaded_issues='Y'). It also explicitly marks `mylar_add_story_arc` as 'Add/create', signaling mutation. Still, it does not give a high-level summary of which operations are read-only versus mutating, nor does it mention permissions, side effects, rate limits, or error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description opens with a crisp one-line overview and invocation pattern, then uses a consistent bullet list for all seven operations. Each entry follows the same 'signature — explanation' format without redundancy or filler. Although the description is lengthy due to the number of operations, every sentence carries necessary information and is well-structured for scanning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers all operations and their parameters, and an output schema exists to define return values, reducing that burden. The main gaps are the lack of explicit read-only vs mutation distinction (though inferable) and the misleading 'discovery' umbrella that doesn't fit the add operation. These minor omissions prevent perfect completeness for agent decision-making.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is minimal, with only `operation` and a generic `arguments` object (0% coverage), so the description fully compensates. It documents every operation's parameter signature, required vs optional arguments (e.g., storyarcname required when creating, id to extend), and explains the meaning of special flags. This adds substantial meaning beyond the schema and enables correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as handling Mylar3 list operations, enumerating all supported operations and their purposes. It differentiates from sibling tools by its 'lists' scope. However, the term 'discovery' is contradictory because `mylar_add_story_arc` is a mutation operation, not a read-only discovery, which slightly weakens clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by providing the invocation pattern ('Pass `operation` and an `arguments` dict...') and listing all operations, which tells an agent exactly what actions are possible. However, it offers no explicit comparison to sibling tools, no when-to-use/when-not-to-use guidance, and no mention of prerequisites or context for choosing this tool over alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mylar_comicsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses important behaviors like background execution ('Runs in a background thread') and destructive delete warning ('WARNING: pass directory='true' to also delete the comic folder from disk (rmtree)'). However, it lacks details on side effects for other mutating operations (e.g., change_book_type, change_status), permission requirements, or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured as a bullet list with a clear opening instruction. Each operation is documented in one line with its parameters and brief explanation. It is concise yet thorough, front-loading the general usage pattern and itemizing operations without unnecessary prose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (12 operations, 0% schema coverage), the description covers all operations with signatures and basic behavior. An output schema exists, so return values need not be detailed. Some parameter semantics and behavioral details are incomplete, but the description provides enough to invoke the tool correctly for most operations, making it reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage and only exposes operation and arguments, so the description must compensate. It provides per-operation parameter lists and explains many key semantics (e.g., booktype values, id='all' for bulk, type_='story_arc' for arcs, directory='true' for folder deletion). Some parameters remain cryptic (e.g., issue, pageSize, serinfo in mylar_find_comic), but overall it adds substantial meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it handles 'mylar comics operations on Mylar3' and lists all supported operations with specific verb+resource pairs (e.g., 'mylar_add_comic(id) — Queue adding a series'), distinguishing this tool from sibling tools focused on issues queue, lists, providers, and system.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies clear usage context: use this tool for comics-related operations (add/delete/get/pause etc.) on Mylar3. It does not explicitly mention when not to use this tool or name alternatives, but the scope is well-defined through the operation list and sibling context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mylar_providersA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It provides operation semantics (add/change/delete/list), notes the constraint that torznab requires categories, and specifies that delete requires either name or prov_id. However, it does not discuss side effects like persistence, permissions, idempotency, or error behavior, which are important for a tool that mutates provider configuration.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-organized: a one-line overview followed by four operation signatures with descriptive notes. Each line adds specific value, with no filler or redundant repetition. The structure makes it easy to scan and understand the tool's capabilities quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a dispatcher tool with four operations, the description covers all operation parameters and even highlights the return structure for list_providers. The presence of an output schema reduces the need to document return values further. However, additional context about side effects (e.g., whether changes persist to Mylar3's config) would improve operational completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Since schema description coverage is 0%, the description is the primary source of parameter meaning. It enumerates each operation's parameters, defaults, and required fields, and explains the operation vs. arguments pattern. It does not explicitly type every parameter (e.g., 'enabled' as boolean) or describe the arguments dict structure beyond the signatures, but it compensates well overall.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly defines the tool's role as managing Mylar3 providers, explicitly listing all four supported operations (add, change, delete, list) with detailed function signatures. This distinguishes it from sibling tools that target comics, issues, lists, or system functions, making the tool's purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The operation list effectively tells the agent when to use the tool (whenever provider management is needed), and each signature clarifies the exact scope of each operation. However, there is no explicit 'when not to use' or direct comparison with alternative tools, leaving the agent to infer the appropriate context from the sibling names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mylar_systemA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and succeeds: it discloses side effects (e.g., mylar_shutdown 'stops the server', mylar_update 'may restart the app'), persistence ('Applies immediately and persists to config.ini'), and auth requirements. It also notes when a function does NOT require the API key, which is valuable behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the general invocation pattern, then uses a clear bulleted list to cover each operation. Every line supplies necessary information without fluff; the length is appropriate for a 10-operation dispatcher and remains scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is complete for its purpose: it explains the dispatch mechanism, lists all operations, specifies their arguments, flags side effects, and notes credentials. The output schema exists but doesn't reduce the need for behavioral details, which are provided. Sibling tools are clearly different domains, so no further contextual clarification is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description thoroughly explains parameters for each operation in prose, such as 'Pass operation and an arguments dict matching that operation's parameters' and explicitly details argument shapes (e.g., 'Takes a dict of checkbox/text settings keyed by their config.ini name'). This fully compensates for the generic input schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states this tool handles 'mylar system operations on Mylar3' and enumerates all supported operations with specific verbs and targets (e.g., 'Check GitHub for updates', 'Clear Mylar3's in-memory log buffer'). It distinguishes from sibling tools by focusing exclusively on system operations rather than comics, issues, lists, or providers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for each sub-operation, including when to use them (e.g., 'Bootstrap helper: fetch the API key' for mylar_get_api) and prerequisites (e.g., requires MYLAR_WEB_USERNAME and MYLAR_WEB_PASSWORD for set_config). It does not explicitly discuss alternatives or exclusion cases, but the dispatcher nature makes such guidance largely unnecessary.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.