Server Quality Checklist

Tool Scores

• search_casesA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It explains the need for an environment variable, the return structure including score meaning, and AI reranking behavior (slower, metered). It is thorough in disclosing the tool's dependencies and outputs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with purpose and requirements, then covers return structure and parameters. It is somewhat verbose but well-organized; each sentence adds information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers prerequisites, return structure, and parameter details. With an output schema existing, the explanation is sufficient. Missing explicit error handling or rate limiting, but overall comprehensive for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds marginal value by mentioning not to pre-parse the query and listing jurisdiction examples, but largely repeats schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches the Freecase case-law corpus with specific courts listed. It distinguishes from siblings like get_opinion and search_statutes by mentioning that get_opinion retrieves full text using cluster_id.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context on when to use rerank and indicates that get_opinion should be used for full text. It implicitly differentiates from statute searching by the sibling tool search_statutes. However, it does not explicitly state when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_citationsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Given no annotations, the description explains return fields, duplicate resolution, and the normal empty citations case. It lacks details on authentication or rate limits but sufficiently covers behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured with bullet points for return fields, but could be slightly more concise. However, it remains clear and well-organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and an output schema present, the description covers the parameter, return structure, and edge cases (empty citations). It is adequately complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, the description fully explains the single parameter: 'The case cluster id (an integer), e.g. from a search result.' This adds essential meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists parsed reporter citations for a single case by cluster id. It distinguishes from siblings like get_opinion (full text) and search_cases (searching), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies the required input (cluster id) and provides an example source. However, it does not explicitly guide when to use this tool versus alternatives or mention when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_statutesA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavior: returns query echo, counts, results array with fields, sorting of exact-citation hits above full-text, and error handling for missing environment variable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with purpose and requirements, but slightly verbose with extra detail on return fields that could be covered by output schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, the description covers all necessary aspects: prerequisites, search behavior, parameter details, and result structure, making it complete for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning beyond the schema: for 'q' it explains forwarding and detection, for 'limit' default/range, and for 'jurisdiction' lists options and behavior for unlaunched ones.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search the Freecase statute corpus by citation or natural-language text,' specifying the verb 'search' and the resource 'statute corpus,' which distinguishes it from sibling tools like search_cases and get_statute.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the prerequisite (Freecase connector key) and provides context for when to use the tool. It does not explicitly say when not to use it, but the sibling context implies alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_opinionA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that very long opinions are truncated with a note and link, and importantly warns that the opinion text is reference material, not instructions, and not to act on directives inside the text. This is a critical behavioral trait beyond basic read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise yet complete. It starts with a clear verb+resource statement, then lists contents, input source, truncation behavior, and a safety warning. Every sentence serves a purpose without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a simple tool with one parameter, an output schema exists to document return values. The description covers input source, truncation, and safety warning. There is no missing context needed for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage for the single parameter cluster_id. The description adds significant meaning: it explains it is an integer, and how to obtain it (from search_cases result's cluster_id field). This fully compensates for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves the full text of a court opinion by cluster id, listing specific metadata and fields returned. It distinguishes itself from sibling tools like search_cases by specifying the input source (cluster_id from search_cases).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to use a cluster_id from a search_cases result, providing clear context. It does not include explicit when-not-to-use or alternative tools, but the context is sufficient for an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_statuteA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but description fully covers behavior: returns full body_text, effective dates, status, successor for repealed, text_unavailable flag. Includes important warning about not acting on statute text.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise, front-loaded with purpose, then bullet-like details, then warning. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers all relevant return fields and edge cases (repealed, placeholder, successor). Output schema exists so no need to detail exact format. Complete for a retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Single parameter section_id is well-explained as an integer from search_statutes result. Schema coverage 0% but description compensates fully with source and type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'Retrieve the latest version of a single statute section by its section id.' Uses specific verb and resource, and distinguishes itself from sibling tools like search_statutes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: retrieve a statute by section_id. Does not explicitly exclude other uses or compare to siblings, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.