Server Quality Checklist

Tool Scores

• browse_movies_by_genreC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must disclose behavioral traits. It mentions pagination support but does not state whether this is a read-only operation, whether it requires authentication, or what happens with invalid genres. The mutation safety profile is unclear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the core purpose efficiently. It contains no unnecessary words, but could benefit from a second sentence on when to use this vs getMoviesByGenre.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description does not explain return values (e.g., movie details format, total result count). For a paginated browsing tool with three parameters and sibling overlap, more context is needed to guide correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters. The description adds minimal value beyond listing genres and pagination, but it does not clarify cursor semantics (e.g., whether it is a page number or token). Baseline 3 is appropriate given full schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Browse' and resource 'movies in a genre' with pagination support. While it differentiates from siblings like getMoviesByGenre (similar purpose) and explainMovieData (different purpose), it could be more explicit about how browsing differs from getting movies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool vs getMoviesByGenre, which appears to have overlapping functionality. It does not specify exclusions or alternatives, leaving the agent to infer usage from the description alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getMoviesByGenreC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description alone must disclose behavioral traits. It only states 'Get movies by genre' without confirming read-only behavior, authentication needs, or pagination behavior (though the schema reveals a default limit of 10). This minimal disclosure leaves significant behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence of 8 words, with no superfluous information. It is front-loaded with the core action. However, it could be slightly expanded with additional context (e.g., limit behavior) without becoming verbose, so it does not achieve a perfect score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description should explain what the tool returns (e.g., list of movie objects, format). It does not. Additionally, it fails to differentiate from the sibling tool 'browse_movies_by_genre', making the overall context incomplete for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for both parameters, so the baseline is 3. The description does not add any extra meaning beyond what the schema provides (e.g., it does not clarify whether genre matching is exact or fuzzy). It scores neither higher nor lower than the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get movies by genre') and the data source ('Neo4j database'). It is specific and provides a direct understanding of the tool's function. However, it does not differentiate itself from the sibling tool 'browse_movies_by_genre', which appears to have a very similar purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description offers no guidance on when to use this tool versus alternatives like 'browse_movies_by_genre'. It lacks any context about prerequisites, limitations, or typical use cases. Without such guidance, an agent may select the wrong tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• graphStatisticsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden. It states a read operation (count), but doesn't disclose whether the count is real-time, cached, or if it requires permissions. For a zero-parameter tool, more context on performance or scope would help.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence, perfectly sized for the tool's simplicity. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters, no output schema, and simple purpose, the description is largely adequate. However, it doesn't mention the format or granularity of the counts (e.g., separate numbers for nodes vs. relationships), leaving minor ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has zero parameters with 100% coverage, so the description doesn't need to add param details. The baseline is 4 due to schema fully covering the (empty) parameter list.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Count') and the resources ('nodes and relationships in the graph'). It differentiates from siblings (which filter by genre or explain data) by being a global count tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for getting graph size, but it doesn't explicitly say when to use this vs. siblings (e.g., 'Use this for an overview; use getMoviesByGenre for filtering'). No when-not or alternatives mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explainMovieDataA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description bears full responsibility for behavioral disclosure. It mentions 'using LLM sampling', which implies non-deterministic generative behavior, but omits details on authorization, rate limits, or potential side effects. The minimal context is acceptable for a read-like tool but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 10 words, front-loading the core functionality. Every word contributes meaning, and there is no redundant or irrelevant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given low complexity (1 required parameter, no output schema), the description adequately states the tool's purpose but fails to specify what aspects of movie data are explained (e.g., plot, cast, ratings) or the nature of the 'natural language explanation'. The output format is left entirely to inference.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with a single parameter 'movieTitle' described as 'The title of the movie'. The description adds no additional parameter semantics beyond the schema, so it meets the baseline for well-documented parameters without extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get', the resource 'movie data', and the method 'natural language explanation using LLM sampling'. It distinguishes itself from siblings like getMoviesByGenre and graphStatistics, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings. It does not mention when-not-to-use, prerequisites, or alternatives, leaving the agent to infer based solely on the tool name and sibling list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.