Server Quality Checklist

Tool Scores

• get_media_detailsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must convey behavioral traits. It only says 'Get media details', which implies a read operation but does not disclose side effects, idempotency, auth requirements, rate limits, or any other behaviors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is efficient and front-loaded. It conveys the core concept without extra words, though it might be too brief given the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 7 parameters (including batch and format control) and no output schema, the description omits important details like how to specify single vs. batch format, the meaning of format/fields/language, and the output structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is high (86%), so baseline is 3. The description adds value by clarifying single vs. batch usage and the levels (basic/standard/full), but does not address fields, format, or language parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get media details') and resource, and mentions the key capabilities (single/batch, level control). However, it does not distinguish this tool from siblings like 'get_service_details' or 'search_media', which might also return media-related information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It lists capabilities but does not explain the context (e.g., retrieving details vs. searching) or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_mediaC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description should fully disclose behavior. It mentions dedupe returns actionable statuses but fails to indicate if tool triggers requests (autoRequest) or is read-only. No side effects or auth needs disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise: two sentences covering purpose and dedupe statuses. No redundant information; every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite 13 parameters and no output schema, description omits output format for single/batch modes, mode selection criteria, and how to interpret results beyond dedupe. Incomplete for complex tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds context about dedupe statuses but does not significantly enhance parameter understanding beyond what schema descriptions provide.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search movies/TV with single/batch/dedupe modes,' specifying the verb 'Search' and resource 'movies/TV.' It distinguishes from siblings like manage_media_requests or request_media, which involve different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives (e.g., get_media_details for details, request_media for requesting). The description lacks when-not usage or explicit context for choosing modes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_service_detailsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It states it retrieves data but does not disclose auth requirements, side effects, rate limits, or whether the tool is read-only (implied but not explicit).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence, front-loaded with key purpose, no fluff. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given simple parameters and no output schema, description adequately lists the types of data returned. Could mention that servers from get_services are needed, but it's implied. Minor gap for moderate completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (both parameters described). The description adds no extra meaning beyond the schema; it lists output types but not parameter details. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resources: quality profiles, root folders, tags, and language profiles for a Radarr/Sonarr server. It distinguishes from siblings like get_media_details (media items) and get_services (list servers).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs siblings like get_media_details or search_media. No mention of prerequisites, when not to use, or how it fits in a workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• manage_media_requestsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It mentions batching and filters but does not disclose destructive nature of 'delete', side effects of 'approve'/'decline', or required permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise: two sentences delivering purpose and filter options. No filler, front-loaded information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 9 parameters, multiple actions, batch support, and no output schema, the description is inadequate. Missing explanations for format, summary, batching mechanics, and action specifics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 44% (only action, summary, requestId, requestIds have descriptions). Description adds filter enum values and batching hint but does not explain skip, take, sort, format, or their interactions. Partially compensates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool manages media requests with specific actions (get/list/approve/decline/delete). It distinguishes from siblings like 'request_media' (creation) and 'search_media' (searching) by focusing on request management.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description implies usage for request management but does not explicitly guide when to use this tool vs siblings like 'request_media' or 'get_media_details'. No exclusion or alternative mention.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_servicesA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden. The description indicates a read operation ('List'), which is inherently non-destructive, but it does not explicitly state read-only behavior, permissions, or side effects. For a simple list operation, this is acceptable but could be more explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loading the core action and then listing return fields. Every sentence adds value, and there is no redundant or unnecessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (list operation with one optional parameter, no output schema), the description provides sufficient context: what the tool does and what fields are returned. No additional details (e.g., pagination) are necessary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully describes the single optional parameter with an enum and default behavior ('Omit for both'). The description does not add meaning beyond the schema, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'List configured Radarr/Sonarr servers.' It uses a specific verb ('List') and resource ('servers'), and lists the fields returned (ID, name, isDefault, etc.). This distinguishes it from sibling tools like 'get_service_details' which likely retrieves a single server.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for retrieving all configured servers and mentions the fields returned. However, it does not explicitly contrast with sibling tools like 'get_service_details' or state when to use this versus alternatives, e.g., for an overview vs. detailed info.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• request_mediaA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses auto-confirm logic, validation, dry run, and TV seasons behavior. Missing details are what happens after a request is made (e.g., submission status).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, uses newlines for structure, and every sentence provides essential information. It is front-loaded with the core behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 11 parameters and no output schema, the description covers key invocation details: auto-confirm, seasons, validation. Missing are clarifications for some params like serverId, profileId, rootFolder, and the output format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is high (73%). The description adds significant context beyond schema, especially the auto-confirm rule and seasons interpretation. This goes beyond the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('request media'), the resource, and the key auto-confirm behavior for TV ≤24 episodes. It distinguishes from siblings like search_media and manage_media_requests by specifying single/batch and validation details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided on when auto-confirm applies (movies and TV ≤24 eps) and when confirmed:true is needed (TV >24 eps). Seasons handling is clearly explained. However, no explicit alternatives or when-not-to-use scenarios are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.