Server Quality Checklist

Tool Scores

• legalize_list_countriesB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint. The description merely restates the list action without adding behavioral context. It does not contradict annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. However, it could include a brief note about the output format or purpose without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (1 optional param, no required fields, annotations present, output schema exists), the description is minimally adequate but lacks additional context that could help an agent, such as mentioning that this tool returns country names or codes.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the single boolean parameter is already documented. The description does not add any extra meaning or usage details beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists jurisdictions in the legalize-dev corpus. It uses a specific verb (list) and resource (jurisdictions), and distinguishes from sibling tools that deal with laws, reforms, and metadata.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No usage guidelines are provided. The description does not indicate when to use this tool versus alternatives like legalize_get_meta, nor does it provide any context on prerequisites or limitations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• legalize_list_reformsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint, openWorldHint, idempotentHint, destructiveHint. The description adds context by specifying 'reform history (commits)' and 'legislative timeline,' but does not elaborate on ordering, pagination, or potential nuances beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence that is front-loaded with the core action. No redundant or unnecessary words; every part contributes to clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (not shown but indicated), the description is fairly complete. It could hint at the return structure (e.g., 'reform history includes dates and changes'), but the output schema likely covers that.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 100% coverage with each parameter having a description (e.g., 'ISO-3166 alpha-2 code', 'filename stem'). The description adds no additional meaning to parameters beyond what the schema provides, so baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List the reform history (commits) of a law - the legislative timeline,' which specifies the verb (list) and resource (reform history of a law). It effectively distinguishes from siblings like legalize_get_law (likely gets current law) and legalize_search_laws (search).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for viewing legislative timeline, but does not explicitly state when to use this tool versus alternatives. No exclusion or when-not-to-use guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• legalize_search_lawsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly, idempotent, and non-destructive. The description adds value by disclosing the token requirement and the underlying GitHub code search mechanism, which are not present in annotations. However, it does not detail pagination behavior or error handling beyond the limit parameter.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded with the action, and includes the critical token note in parentheses. No extraneous words, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has three parameters, full schema coverage, and an output schema (not shown), the description provides sufficient context for the core operation. However, it could briefly mention result scope (e.g., returns matching law snippets) to complete the picture.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with each parameter having a description. The description does not add new semantic information beyond the schema, so it meets the baseline for full schema coverage without additional parameter insight.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches a jurisdiction's laws by keyword, specifies it uses GitHub code search, and notes the token requirement. This distinguishes it from siblings like legalize_get_law (retrieve a specific law) and legalize_list_countries (list jurisdictions).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies search use but does not explicitly guide when to use this tool over alternatives. The token requirement is mentioned but no when-to-use or when-not-to-use guidance is provided, leaving the agent to infer based on tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• legalize_get_lawA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide safety profile (readOnly, idempotent, non-destructive). The description adds that it returns both metadata and full text, which is beyond what annotations cover. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no wasted words, front-loaded with action and resource. Every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the input schema has full descriptions and an output schema exists, the description is complete for the tool's purpose. It mentions the key optional feature (historical commit) and the fact it returns both metadata and full text.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already describes all parameters. The description reinforces the purpose but does not add new meaning beyond stating 'by id' and the optional historical commit, which is already clear from the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Fetch' and identifies the resource 'a law's metadata and full text' with a clear method 'by id (optionally at a historical commit)'. It distinguishes from siblings like legalize_get_meta by indicating it returns both metadata and full text, and from legalize_list_reforms by noting the optional commit SHA.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: when you have the law ID and optionally want a historical version using the SHA from legalize_list_reforms. However, it does not explicitly state when not to use or provide alternatives beyond the implied sibling context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• legalize_get_metaA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and no destructive hint. The description adds valuable behavioral context by noting the tool is 'cheap', indicating low cost or fast performance, which is beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is front-loaded with key action and resource, no wasted words. Every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the existence of an output schema, return values need no explanation. The description fully covers the tool's role: fetching only metadata, cheap, for citation checks, and distinguishes from siblings. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameters are well-documented. The description does not add further meaning to the parameters beyond the schema, staying at baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (fetch), resource (law's frontmatter metadata), and distinguishes it from sibling tools like legalize_get_law by explicitly saying 'no body'. The phrase 'cheap for citation checks' further clarifies the purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for citation checks and distinguishes from legalize_get_law (full body). It provides clear context but does not explicitly state when not to use or list alternatives beyond the implied distinction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.