Server Quality Checklist

Tool Scores

• get_judgmentB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the disclosure burden and does add one behavioral detail: the judgment is assembled from chunks in chunk-index order. It does not mention error behavior, what happens if chunks are missing, or any access constraints, so transparency is partial.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence with no filler, front-loaded with the action, and all words earn their place. It is an appropriate length for such a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although an output schema exists, the definition lacks parameter semantics, usage guidance, and clarification of what 'chunks' means. An agent may infer the call shape from names, but it is not fully equipped to invoke the tool correctly in ambiguous cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not compensate by explaining how case_id or court are used. The property names are self-explanatory at a basic level, but the description adds no parameter-specific meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Reassemble') and a clear resource ('full judgment'), and explains the mechanism of ordering by chunk index. This is distinct from siblings like search_judgments (searching) and get_act (acts).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It implies use when a complete judgment text is needed, but it does not state when to prefer search_judgments or other alternatives, nor give explicit exclusions or prerequisites. The usage context is only inferable from the purpose statement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_regulationsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It adds useful context about provision-level granularity and shared schema semantics, but it does not disclose pagination, result ordering, whether only published instruments are returned, or other runtime behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three tight sentences front-load the tool's purpose and example, then point to the relevant sibling for schema semantics and list_catalog for valid inputs. Every sentence earns its place with no redundant phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The core invocation path is covered: required body parameter, example values, where to find valid slugs, and the relationship to search_legislation. Gaps remain around the optional parameters and behavior, but the output schema and self-explanatory param names reduce the risk of incorrect calls.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for all four parameters. It gives an example for 'body' but does not explain 'query', 'limit', or 'in_force_only'; the reference to 'same query semantics as search_legislation' is helpful only if the agent already knows that tool's parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Search one regulator's instruments at the provision level.' It also distinguishes itself from sibling search tools by emphasizing 'regulator' and by explicitly linking its query semantics to search_legislation while still framing it as a distinct corpus.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It tells the agent when to use this tool—for regulator instruments—and points to list_catalog() to discover valid body slugs. It does not explicitly say when not to use it versus search_legislation or search_judgments, but the regulator/provision framing makes the intended domain clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_schemaA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral burden. It conveys read-only intent through 'Return' and adds useful context about undocumented schemas, but it does not mention error cases, authentication, or side effects. The core behavior is transparent enough for a schema lookup, but not richly detailed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler: the first sentence states the action and input source, the second provides usage guidance and context. Every clause earns its place and the key information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity, one required parameter, and the presence of an output schema, the description covers what the tool does, where the input comes from, when to use it, and why it may be needed. No critical missing information prevents correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only provides a string title for file_uri, and schema description coverage is 0%. The description compensates by explaining the parameter is an hf:// URI obtained from list_catalog and refers to a parquet file. It lacks an explicit example or format validation, but the single parameter's meaning is effectively conveyed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Return the real column names and types for a parquet file.' It also clearly distinguishes itself from siblings by referencing the hf:// URI from list_catalog and being a schema-introspection step before run_sql.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says to use this before run_sql when unsure a field exists, and notes that plain court-judgment files lack documented schemas. It gives a concrete condition but does not detail exhaustive when-not-to-use scenarios or all alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_actA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It discloses the non-obvious state inference behavior, gives a concrete example, and explains when to override it. It could add more about pagination or errors, but the core behavior is clear and consistent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and front-loaded with the core purpose. Every sentence earns its place: the first gives the action, and the second paragraph clarifies an important optional parameter behavior without filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter fetch with an output schema, the description covers what the tool returns, the ordering, and the state inference rule. It does not explicitly point to search_legislation for broader search use cases, but that is a minor gap given the get-by-id nature of the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does well for state, explaining that it is optional, inferred from act_id, and overridable. The required act_id is only illustrated via example rather than formally described, but the example makes its structure sufficiently inferable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Fetch every section of one Act, in section order.' It clearly names the scope (every section), the resource (one Act), and the ordering, and it is readily distinguishable from sibling search tools like search_legislation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The first sentence gives clear context: use this when you need all sections of a specific Act in order. It does not explicitly name alternatives or state when not to use this tool, so it stops short of full routing guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_catalogA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It reveals useful behavior: the tool reflects the current state of the repo, groups results, returns file URIs, and caches results for 15 minutes with a refresh option. This goes well beyond a simple list operation, though it does not discuss failure modes or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, then explains why to call it first, and ends with the parameter behavior. Every sentence earns its place with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter listing tool with an output schema, the description covers the discovery workflow, the returned URI format, grouping semantics, and caching behavior. An agent has everything needed to decide to call it and to understand what it will get back.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must add parameter meaning. It does: refresh=True bypasses the 15-minute cache and re-checks the repo. The default false value is provided by the schema, so the description meaningfully supplements the schema rather than repeating it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists parquet files that actually exist in the dataset, grouped into judgments, regulations, and legislation. It also distinguishes itself by framing it as the entry point for discovering names and URIs that other tools expect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Call this first' and explains the tool provides the exact court/state/body names and hf:// URIs needed by run_sql and sibling tools. It gives clear when-to-use context but does not enumerate when not to use it or compare against each sibling directly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_legislationA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the full burden and does so well: it discloses that query is an ILIKE substring match rather than ranked relevance, that state is fuzzy-matched, that in_force_only excludes repealed/spent acts by default, and that each result includes an authoritative source_url.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with a clear summary sentence followed by tight bullet points. Every bullet adds meaningful information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, matching behavior, jurisdiction handling, default filtering, and result provenance, which is sufficient for most calls. It does not mention limit/pagination behavior or what happens when neither query nor act_id is supplied, though the output schema likely covers return structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions are absent, so the description compensates for most parameters: query, state, act_id, and in_force_only all receive added semantics with examples. Only limit is left undocumented, which is a minor gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb and resource: 'Search Indian legislation at the section level for one jurisdiction.' This immediately distinguishes it from siblings like get_act, search_regulations, and search_judgments.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for using the tool, including what type of search it performs and that state values come from list_catalog. It does not explicitly state when not to use it or name alternatives, but the purpose is clear enough for an agent to choose correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_sqlA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden and delivers richly: read-only execution, 'only a single SELECT/WITH statement is allowed -- no DDL/DML, no multiple statements,' the hard 'limit is capped at 200 rows' stop, and the scan-cost rationale for tens-of-GB files. No contradiction with annotations exists since none are present.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Though long, every sentence earns its place: purpose, a fully executable example, hard constraints, the row cap, and performance guidance. Each element of the example teaches a required convention (URI format, read_parquet invocation, filter patterns), so no portion is filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The output schema covers return values, freeing the description to focus on behavior, and it covers everything else an agent needs: prerequisites (list_catalog URIs), structural constraints, the 200-row cap, and an optimization strategy for avoiding slow scans. Nothing needed for correct invocation is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for both bare parameters, and it does. It explains the required shape of `sql` (single SELECT/WITH, hf:// URI + read_parquet pattern, example query) and the `limit` cap behavior at 200 rows. This fully makes up for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource immediately: 'Run a read-only SELECT against one or more Open India Law parquet files.' The concrete example with hf:// URIs and the reference to `list_catalog` make the scope unambiguous and distinguish it from the higher-level search_* siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear operational context: URIs come from list_catalog, and it advises narrowing 'with a specific file (one court/state/regulator), a year range, and an indexed-looking equality filter (act_id, case_id).' However, it never explicitly states when NOT to use this tool in favor of sibling search/get tools, so routing to alternatives is implied rather than exclusionary.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_judgmentsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden, and it delivers: it reveals that court values are fuzzy-matched against list_catalog()['courts'], that year filters prune row groups for efficiency, and that results are chunks rather than complete judgments. This goes well beyond the structured schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with the core action, and every sentence adds meaning: required parameter, performance tradeoff, filtering guidance, and result shape. No filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with five parameters, no annotations, and an existing output schema, the description covers the essential decision context: which parameter is required, how to scope the search, why narrowing matters, and how to follow up on a hit. Nothing critical for correct invocation is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description compensates for court and year_from/year_to by explaining requiredness, fuzzy matching, and performance behavior. Query and limit are less elaborated, but their purposes are reasonably inferable from the full-text-search framing and the schema defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Full-text search over one court's judgment chunks.' It clearly distinguishes from the sibling search tools (search_legislation, search_regulations) by focusing on judgment chunks and requiring a court.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says court is required, explains the fuzzy-matching behavior, warns against searching all 26 courts due to cost, and recommends using year_from/year_to when possible. It also directs the agent to get_judgment for reassembling full judgments from result chunks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.