Server Quality Checklist

Tool Scores

• buscar_sumulasA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It only explains the search behavior, not side effects, authentication needs, or rate limits. Minimal disclosure beyond the obvious.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (two main sentences plus structured Args/Returns), front-loads the key purpose, and uses clear formatting. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple search tool with few parameters and an output schema, the description adequately covers functionality, return fields, and parameter details. Could mention pagination but limite param addresses limit.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description significantly adds meaning beyond the schema titles, explaining each parameter's purpose, format, and defaults. For example, 'query' is terms, 'limite' has range and default, 'tribunal' is optional with examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The tool name and description clearly indicate it searches for 'sumulas' (precedents) on JusBrasil, using a specific filter. It is distinct from sibling 'buscar_jurisprudencia' which likely searches general jurisprudence, though not explicitly stated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives, or when not to use it. The description merely states what it does without contextual advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• buscar_jurisprudenciaA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses key behavioral traits: the periodo filter uses relative dates (not exact), the tribunal filter accepts multiple values separated by commas, and the return format is a list of dicts with specific fields. This goes beyond basic functionality, but does not mention rate limits, authentication, or side effects (likely none).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with Args and Returns sections, front-loading the purpose. It is appropriately sized but could be slightly more concise—some parameter descriptions are verbose (e.g., tribunal notes). However, no sentence is wasted, and the format aids readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, need for output description) and presence of an output schema (mentioned as returning list of dicts), the description is thorough. It covers all parameters and return values. Minor gaps: no mention of error cases or pagination beyond limite, but overall adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 0% description coverage (no descriptions in schema properties). The tool description compensates fully by explaining each parameter with examples, default values, allowed values (e.g., tribunal values: 'STF', 'STJ', etc.), and behavior (e.g., periodo relative). This adds significant meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Pesquisa jurisprudencia no JusBrasil, com filtros reais do site,' specifying the action (search jurisprudence), source (JusBrasil), and distinguishing feature (real site filters). It differentiates from sibling tools like buscar_sumulas (search summaries) and ler_decisao (read decision), which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains parameters and acceptable values, providing context on when to use filters. However, it does not explicitly state when to use this tool over alternatives, nor does it mention when not to use it. There is no direct comparison to siblings, leaving the agent to infer from the tool name and description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compilar_dossieA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries the burden. It discloses that including full text requires login, is slower, and not always available. It also mentions return dict structure. Could mention overwrite behavior, but overall transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is well-structured with summary then args, but slightly verbose. Front-loaded with purpose. Could be tightened, but still effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, but description explains return dict (arquivo, n_itens, n_falhas, com_inteiro_teor). Covers slower behavior and login requirements. Complete for a compilation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so description fully compensates. Explains each parameter: urls (list of URLs), incluir_inteiro_teor (boolean with default and caveats), titulo (default), caminho (default). Adds meaning beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool compiles multiple decisions/sumulas into a single Word document, using the specific verb 'compilar' and resource 'UNICO documento Word (.docx)'. It also distinguishes from siblings by referencing 'buscar_jurisprudencia' and 'buscar_sumulas' as prior steps.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use depois de escolher, com buscar_jurisprudencia/buscar_sumulas, quais decisoes quer reunir', providing clear when-to-use guidance and reference to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ler_decisaoA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully carries the burden. It discloses that null fields indicate extraction failure or actual absence, warns against inventing content, and instructs to check _parse_warnings. It also clarifies the non-official nature of JusBrasil, which is critical for the tool's usage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, usage, anti-hallucination, args, returns). It is somewhat lengthy but every sentence serves a purpose. Minor redundancy could be trimmed, but overall it is efficiently organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and the complexity of the tool (extracting multiple fields, handling failures, anti-hallucination), the description is remarkably complete. It lists all return fields, provides the citation format, and includes critical usage warnings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'url' is exhaustively described: it provides an example URL, explains where to obtain it (from buscar_jurisprudencia's 'url' field), and clarifies it should be the full URL. With 0% coverage from the schema, the description fully compensates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it reads an individual decision from JusBrasil and extracts all metadata. It differentiates from siblings by specifying it is used after buscar_jurisprudencia to obtain formal data (relator, orgao julgador, data de julgamento) for citations. The verb 'ler' and resource 'decisao' are precise.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use (after buscar_jurisprudencia) and provides context for building citations in petitions. It also includes anti-hallucination instructions and warnings about JusBrasil not being an official source, guiding appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ler_inteiro_teorA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes login requirement, behavior when no session, anti-hallucination instructions, limitations of JusBrasil, and return format. No annotations present so description carries full burden and does it thoroughly.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections, but slightly verbose with repeated emphasis on anti-hallucination. Every sentence is valuable, though could be tightened.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers all important aspects: purpose, usage distinction, authentication, error handling, return format, and source reliability. No output schema, but description explains return dict fields fully.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema provides no description for url, but the tool's Args section explains that url can be a decision URL or a direct full-text URL with example. This adds essential meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it extracts full text of a decision (report, vote, full judgment) and distinguishes from sibling ler_decisao which only returns metadata and summary.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly contrasts with ler_decisao, provides authentication requirement, anti-hallucination guidance on null texto, and warns that JusBrasil is not official source.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.