Server Quality Checklist

Tool Scores

• fr_articleA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true and idempotentHint=true. The description adds behavioral context beyond annotations, such as tolerance in provision_ref format (e.g., 'Article L. 1233-15' equals 'L1233-15') and error codes (missing_arg, not_found, corpus_error). No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, front-loading the main purpose and including error codes efficiently. However, it is in Polish which may reduce clarity for non-Polish agents; still, it earns its space.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description states it returns the full verbatim text. It mentions error codes, which is helpful. Annotations cover safety. Additional details about return format or encoding are missing, but overall it is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already describes parameters. The description adds value by providing an example and explaining the tolerance behavior of provision_ref, which is not present in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves the full verbatim text of a legal provision using document_id and provision_ref, with an example. This distinguishes it from sibling tools like fr_search (search) and fr_list_documents (list).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs alternatives (fr_search, fr_list_documents, fr_validate_citation). The description implies usage for retrieving a specific article but does not explicitly state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fr_validate_citationA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true and idempotentHint=true. The description adds significant context: it explains the fail-closed mechanism, lists all return fields (document_exists, provision_exists, in_force, citation), and mentions error codes ('missing_arg', 'corpus_error'). This goes beyond annotations to inform the agent about expected outputs and failure modes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact with three sentences: purpose, fail-closed behavior, and return fields/errors. Each sentence conveys distinct information with no redundancy. The use of Polish with some English terms is slightly informal but still efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 required params, no output schema), the description covers the essential aspects: what it does, its behavior on failure, and what it returns (including error codes). It does not explain the sibling tools but that is partially covered by their names. The absence of output schema is compensated by the explicit listing of return fields.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters (document_id, provision_ref) are fully described in the input schema (100% coverage). The description's mention of them adds no new semantic details; it only restates their role in the citation check. Baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: grounding citations by checking if a document_id and provision_ref exist in the corpus, explicitly calling it anti-hallucination. This distinguishes it from siblings like fr_search or fr_article, which have different primary functions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides fail-closed behavior (if document or provision missing, provision_exists=false) but does not explicitly guide when to use this tool versus alternatives like fr_search or fr_article. The use case of citation validation is implied but not contrasted with siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fr_list_documentsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only, idempotent, non-destructive behavior. The description adds error code 'corpus_error' and clarifies the tool's discovery purpose, providing useful context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with an additional error code note, front-loading the purpose and avoiding any extraneous text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool, the description covers the core functionality and optional filters. It omits mention of pagination or the limit parameter, but these are well-defined in the schema, so the description is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Parameter descriptions in the schema already cover type, limit, and query with examples and ranges. The description adds no new semantic meaning beyond what the schema provides, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists available documents (codes/laws) for discovering document IDs, distinguishing it from sibling tools like fr_article (retrieve specific article) and fr_search (full-text search).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for listing documents to find IDs, but does not explicitly state when not to use it or mention alternatives, leaving the agent to infer from sibling names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fr_searchA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses return format (snippets with markers, structuredContent.citations) and error codes, adding value beyond annotations which already indicate safe, idempotent, non-destructive behavior. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single, concise sentence with key information front-loaded: search type, domain, output specifics, optional filter, and error codes. No superfluous content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately explains return structure (snippets, citations), error codes, and parameter options. Sibling tools provide complementary context, making this complete for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage; description adds a concrete example for the documents parameter and explains error codes, slightly enhancing understanding beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs full-text search (FTS5) over French legal codes and laws, with distinct outputs like snippets and citations. It differentiates from sibling tools like fr_article and fr_list_documents.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for free-text search but lacks explicit guidance on when to use this tool versus alternatives like fr_article for specific articles or fr_list_documents for document listing. No when-not scenarios mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.