Server Quality Checklist

Tool Scores

• saij_searchA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by explaining the search scope, document types, default behaviors, pagination, and return format. It discloses that 'texto' field only works with sumarios and provides practical examples. However, it doesn't mention rate limits, authentication needs, or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, usage, args, returns) and every sentence adds value. It could be slightly more front-loaded by moving the 'Args' section details into the initial description, but overall it's appropriately sized and organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a 5-parameter search tool with no annotations but with output schema, the description is complete enough. It explains the tool's purpose, parameters, usage, and return format. The output schema existence means the description doesn't need to detail return values, and it covers all necessary context for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing detailed parameter explanations beyond the bare schema. It explains what each parameter does, provides examples for 'query', enumerates all 'doc_type' options with descriptions, clarifies field limitations, and specifies valid ranges for 'limit'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches Argentine legal documents in SAIJ, listing specific document types (court decisions, legal summaries, legislation, doctrine, dictámenes). It distinguishes from sibling tools by focusing on search functionality rather than document retrieval (saij_get_document) or specific summary access (saij_get_sumarios).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('Search Argentine legal documents in SAIJ'), but doesn't explicitly mention when NOT to use it or directly compare to sibling tools. It implies usage through the document type examples but lacks explicit exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• saij_get_documentA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by specifying what data is returned (metadata, PDF URL for fallos, summary text for sumarios) and the types of documents supported (court decisions, laws, legal documents). It doesn't mention rate limits, authentication needs, or error conditions, but provides substantial behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with a clear purpose statement, followed by detailed parameter and return value explanations. Every sentence adds value: the first states the action, the second elaborates scope, and the Args/Returns sections provide essential implementation details without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, no annotations, but with an output schema, the description provides excellent completeness. It explains the single parameter thoroughly, details what information is returned for different document types, and mentions the JSON return format. The output schema likely covers structure, so the description focuses appropriately on semantics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage and only one parameter, the description fully compensates by explaining the 'identifier' parameter in detail: it specifies the two possible formats (SAIJ id-infojus examples like 'FA20000057' or full UUID) and what they represent. This adds crucial meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get a full document'), target resource ('from SAIJ by its ID'), and scope ('complete metadata for a court decision, law, or other legal document'). It distinguishes from sibling tools by focusing on retrieval by ID rather than search or sumario-specific operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying it retrieves documents by ID, suggesting it should be used when you have a specific identifier rather than searching broadly. However, it doesn't explicitly state when NOT to use it or name alternatives, though the sibling tool names suggest potential alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• saij_get_sumariosA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well by explaining what the tool returns (JSON structure with specific fields), the automatic prefix handling for fallo_id, and the nature of the data (legal principles with descriptors). It doesn't mention rate limits or authentication needs, but provides solid behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured and front-loaded: purpose first, then what sumarios contain, then Args and Returns sections. Every sentence earns its place with no wasted words, making it easy to scan and understand.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, no annotations, and the presence of an output schema, the description provides complete context. It explains purpose, usage, parameter behavior, and return structure, making the output schema documentation of the return format sufficient without needing duplication in the description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning beyond the input schema, which has 0% description coverage. It explains what fallo_id represents, provides an example format ('FA20000057'), and documents the automatic prefix handling behavior that isn't captured in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verb ('Get') and resource ('all legal summaries linked to a court decision'), and distinguishes it from siblings by focusing on summaries rather than documents or search functionality. It explains what sumarios contain and their utility.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('for understanding the key legal holdings of a case'), but doesn't explicitly state when not to use it or name specific alternatives among the sibling tools (saij_get_document, saij_search).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.