Server Quality Checklist

Tool Scores

• fipe_priceC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It lists the returned fields, which is some transparency, but does not state that this is a read-only operation, the currency of the price data, whether multiple model years are returned, or error behavior for invalid codes. The 'Returns...' clause is largely a restatement of expected output rather than deeper behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with the action front-loaded and no filler. Every clause earns its place, though slightly more content (e.g., usage note) could justify a 5.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  A simple single-parameter tool, but with the sibling fipe_brands likely producing the code, the description omits the workflow connection and how the code is obtained. Return fields are enumerated adequately, yet an agent must infer the prerequisite step. Adequate but with clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. 'Vehicle model code' does clarify that fipe_code refers to a vehicle model identifier, adding meaning beyond the schema's bare 'Fipe Code' title. However, it omits any format expectations, how to source the code, or validity rules.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Look up'), an explicit resource (FIPE price table), and identifies the input as a 'vehicle model code.' The listed return fields (value, brand, model, model year, fuel) reinforce the tool's purpose. It is domain-distinct from siblings like cep/cnpj, though it doesn't explicitly contrast itself with the closely related fipe_brands.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus its sibling fipe_brands, nor on how a valid fipe_code is obtained (likely from fipe_brands). There are no prerequisites, exclusions, or workflow hints, leaving the agent to infer the lookup sequence on its own.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• holidaysA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does state that it returns an array of holidays with date, name, and type, which is a useful behavioral detail. However, it does not disclose potential error cases (e.g., invalid year ranges), the date format (e.g., ISO), or any limitations on historical/future years. For a simple read-only lookup, this is acceptable but not thorough, hence a 3.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two concise sentences: the first states the core purpose, the second outlines the return format. There is no redundant verbiage, and the most critical information (what the tool does) is front-loaded. It is an appropriately sized and well-structured description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one integer input, outputs an array of objects), but with no output schema, the description must explain the return values more thoroughly. It only mentions 'date, name and type' without specifying the date format or the possible types (e.g., 'national', 'observance'). It also lacks details on edge cases like leap years or invalid years. While adequate for a basic lookup, it is not fully complete for an agent to predict all responses.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description merely restates that the year is an input ('for a given year') without adding any meaning beyond the schema's property name. The schema has 0% description coverage, so the description must compensate by specifying format, range, or constraints, but it does not. The sole parameter 'year' is left underdocumented, making it hard for an agent to know valid inputs. This is a clear gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List', the resource 'Brazilian national holidays', and the scope 'for a given year'. It unambiguously distinguishes this tool from siblings like 'cep' or 'banks' which handle different Brazilian data domains. The return structure is also declared, leaving no doubt about the tool's function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance on when to use this tool versus alternatives, but since none of the siblings overlap in purpose, it is implicitly the only tool for holiday queries. However, the description does not state any exclusions or conditions (e.g., only national holidays, not state-level), which would help an agent decide when to rely on it. The usage context is clear but not explicitly framed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cepA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must disclose behavioral traits itself. It does state the return format ('Returns the address: state, city, neighborhood and street'), which is useful. However, it does not mention error handling, rate limits, or whether the operation is read-only (though that is implied for a lookup). The description adds some value beyond the schema but leaves room for more disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, consisting of two short sentences that place the core purpose first and provide necessary format examples. Every sentence earns its place, with no filler or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple lookup tool with one parameter and no output schema, the description is largely complete. It states the input format and the output fields (state, city, neighborhood, street). It could mention potential error cases or the read-only nature, but given the simplicity of the tool, this is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no parameter description coverage, and the description compensates by clearly defining accepted formats: 'Accepts formats like "01001-000" or "01001000".' This goes beyond the bare schema field type and gives the agent actionable formatting guidance, which is highly valuable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Look up' and the resource 'Brazilian postal code (CEP)', making the tool's function unambiguous. It does not explicitly distinguish itself from siblings like cnpj or ddd, but those are obviously different data types, so the absence of a named alternative is not a major drawback. It is specific and concise.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (a CEP lookup for Brazilian addresses) and provides format guidance, but it does not explicitly state when to use this tool versus alternatives or when not to use it. There is no mention of other tools or exclusions, so the guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• banksA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It states the return format (array with fields ispb, name, code, full name), which is helpful, but it does not mention whether the operation is read-only, if there are any rate limits, authentication requirements, or potential errors. Since this is a simple listing tool, the behavior is fairly predictable, but without annotations or explicit safety hints, the description only partially covers transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two short sentences that immediately state the action and the return structure. No fluff, no repetition. It front-loads the core purpose and packs the essential output details into the second sentence. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters, no output schema, and no annotations, the description provides the essential information: what it returns and the fields. It does not mention use cases or related tools, but given the simplicity of the operation, the description is nearly complete. A minor gap is the lack of any note about pagination or data size, but that may be out of scope for this basic list tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema already reflects that (properties empty, coverage 100%). The description adds no parameter details, but none are needed. Per the baseline for 0-parameter tools, a score of 4 is appropriate because there is nothing more to explain about parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the function: 'List every bank registered at the Brazilian Central Bank.' It specifies the resource (banks) and the action (list), and the scope (every bank). This distinguishes it from sibling tools like 'bank' which likely fetches a single bank, so the purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives such as 'bank', 'cnpj', or others. The description does not mention any conditions, prerequisites, or exclusions. An agent has to infer when this list tool is the right choice, which is a gap for a tool with sibling tools that could overlap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• dddA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only states what is returned and does not mention that this is a read-only operation, whether authentication is required, what happens on invalid input, or any other behavioral traits. This is a significant gap for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with no filler. The main purpose is stated first, and the outcome is described immediately. Every word contributes to understanding the tool's function. This is a model of concise, front-loaded documentation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter lookup tool, the description is nearly complete: it identifies the input, the operation, and the output. The absence of an output schema is partially mitigated by the explicit statement of what is returned. However, it does not mention error behavior or any edge cases (e.g., invalid DDD), which an agent might encounter. Still, the core information needed to invoke it correctly is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no description for the parameter 'ddd' (0% coverage), so the description must compensate. It adds that the parameter is a two-digit area code, which gives meaningful semantic context beyond the bare schema property. However, it does not specify format constraints (e.g., exactly two digits, numeric only) or error handling for malformed input, so compensation is partial.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (look up), the resource (Brazilian phone area code / DDD), and the expected output (state and list of cities). It is specific enough to differentiate from sibling tools like cep, cnpj, banks, etc., which operate on entirely different data domains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys its use case implicitly: querying a Brazilian DDD to get state and cities. It does not explicitly mention alternatives or exclusions, but the context of sibling tools makes it clear that this is the dedicated DDD resolver. There are no pitfalls or prerequisites to warn about, so the guidance is adequate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fipe_brandsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action (listing brands) but does not disclose any additional behavioral traits such as response format, ordering, rate limits, or side effects. For a read-only listing operation, this is minimal but not contradictory; still, it leaves the agent uninformed about what the returned data will look like beyond the implied list of brand strings.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences with no fluff. The purpose is front-loaded in the first sentence, and the constraint is placed second. Every word earns its place, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single parameter, no output schema, and a straightforward list operation, the description is largely complete. It specifies the allowed parameter values and the core purpose. It does not describe the return structure, but that is not critical for a simple listing tool. It could optionally mention that the response contains brand names, but the lack of this is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only defines vehicle_type as a string with no enum or description (coverage 0%). The description compensates by explicitly listing the valid values ('carros, motos, caminhoes'), which adds critical semantic value beyond the schema. It does not explain the meaning of each value, but they are self-explanatory for the FIPE domain, so this is adequate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List'), the resource ('FIPE vehicle brands'), and scopes it by 'for a type.' This is specific and distinguishes it from the sibling fipe_price, which deals with pricing. The allowed vehicle types are explicitly enumerated, leaving no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a hard constraint on the parameter ('vehicle_type must be one of...') which is useful, but it gives no guidance on when to use this tool versus alternatives like fipe_price. It does not mention that this is for listing brands as opposed to other FIPE operations, nor does it advise against using it for other vehicle types. The usage context is implied but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bankA
  4.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility for behavioral disclosure. It only says 'look up', which implies a read operation, but omits error behavior, return format, and any potential side effects. For a read-only tool this is a notable gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, well-structured sentence that front-loads the action and object, then explains the condition. No redundant words; every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter lookup, the description adequately tells the agent what the tool does and how to use it. It does not specify the return structure, but given the simplicity and lack of an output schema, this is a minor omission. The description is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema merely defines 'code' as a string with no description; the description compensates by explaining the format ('numeric code') and providing a concrete example ('001' for Banco do Brasil). This fully bridges the 0% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Look up'), a well-defined resource ('a single Brazilian bank'), and the exact method ('by its numeric code') with a concrete example. This clearly distinguishes it from the sibling 'banks' tool, which presumably lists all banks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The context of use is clear: when you have a numeric bank code and need details for one bank. It does not explicitly mention alternatives or when not to use it, but the instruction is unambiguous enough for an agent to select this tool for single-bank lookups. A slight deduction for not differentiating from 'banks'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cnpjA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the input can be with or without punctuation and lists the output fields, which is useful. However, it does not mention error behavior (e.g., invalid CNPJ, not found) or any rate limits/auth requirements, which could be relevant for an API-based lookup. For a read-only lookup, this is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loads the purpose, and includes only high-value details (input flexibility and returned fields). Every word earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter lookup tool without an output schema, the description covers the essential aspects: what it does, how to format the input, and what data is returned. It does not detail edge cases or exhaustive output, but given the low complexity, it is sufficiently complete for an agent to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides only the type 'string' with no description (0% coverage). The description compensates by explaining that the CNPJ can be provided with or without punctuation, which is a crucial formatting detail. It does not fully describe validation rules or length, but it adds significant meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('look up'), a resource ('Brazilian company'), and an identifier ('CNPJ'), which clearly distinguishes it from sibling tools like cep (postal code) or banks. It also lists the returned fields, leaving no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly identifies the use case (company lookup by CNPJ) and implicitly differentiates it from other lookup tools (CEP, banks, DDD, etc.). It does not explicitly mention when not to use it, but the resource and identifier make the context unambiguous, placing it above 'implied usage' yet below explicit exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.