Server Quality Checklist

Tool Scores

• get_historical_trendB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states the return type (historical series and percentage change) but does not mention whether the operation is read-only, any side effects, authentication requirements, rate limits, or data freshness. The description adds minimal behavioral context beyond the obvious.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: one sentence stating the purpose, followed by a bullet-like listing of arguments. Every word earns its place; there is no filler. The main action is front-loaded, and the argument descriptions are compact.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 2 parameters, no annotations, and an output schema (unseen), the description covers the basic return but lacks details. It does not explain what 'historical series' entails (e.g., frequency, date range, format) or any constraints on 'days'. The description is adequate but leaves room for ambiguity about the exact output structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains 'commodity' as the name of the commodity and 'days' as the window of days with a default of 30. This adds value beyond the schema's type and default, but it does not specify valid commodity identifiers, allowed range for days, or format constraints. The description partially clarifies parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns historical price series and percentage change for a commodity. It uses a specific verb ('retorna') and resource ('série histórica de preços'), distinguishing it from sibling tools like get_commodity_price (current price) and list_commodities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool versus alternatives. It lacks explicit context like when to prefer historical data over current prices or how this tool differs from calculate_margin or generate_report. No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_commodity_priceA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It only states the core behavior (returns current price) but does not disclose what happens on invalid parameters (e.g., unknown commodity, region), whether the price is real-time, or any side effects. Error behavior is unclear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with two lines of purpose and an Args block. Every sentence serves a purpose. It could be slightly more structured (e.g., separate behavior from parameters), but it is not wasteful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (2 params, no nesting, output schema exists), the description covers the key aspects: what it does, parameter meanings, and optionality. It does not mention error handling or data freshness, but the output schema likely covers return format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description adds examples for commodity ('soja', 'milho', 'boi_gordo') and region ('MT', 'PR') and clarifies that empty region means national average. This adds significant meaning beyond the schema's type and default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the current price of an agricultural commodity. The verb 'Retorna' and resource 'preço atual de uma commodity agrícola' are specific. It distinguishes from siblings like list_commodities (listing) and calculate_margin (computation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing a current price, but does not explicitly state when to use it vs alternatives (e.g., for historical trends use get_historical_trend). No when-not-to-use guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calculate_marginA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and the description does not disclose behavioral traits such as whether the tool is read-only or has side effects. As a calculation tool, it likely does not mutate state, but this is not stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, with a clear main sentence followed by parameter explanations. Every sentence adds value, and the structure is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (not shown but indicated), the description adequately covers the tool's purpose and parameters. It is complete for a simple calculation tool, though it could mention return format or edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description includes a docstring explaining each parameter (price per unit, cost per unit, quantity) with examples (e.g., R$/saca), adding meaning beyond the schema's property titles. Since schema description coverage is 0%, this is valuable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it calculates revenue, cost, profit, and margin for production, with a specific verb and resource. It distinguishes itself from sibling tools like list_commodities or get_commodity_price which are data retrieval, not calculation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for margin calculation but provides no explicit guidance on when to use this tool versus alternatives like generate_report. No exclusions or conditions are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_commoditiesA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the output (list of commodities and trading units) but does not detail behavior like idempotency or side effects. However, the read-only nature is obvious.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is front-loaded and contains no unnecessary words. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (no parameters, output schema exists). The description fully captures what the tool does, and given the context, no additional information is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so schema coverage is 100%. The description adds no parameter info, but none is needed. Baseline for 0 parameters is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Lista' = lists) and resource ('commodities suportadas e suas unidades de negociação'), and distinguishes from siblings like get_commodity_price or calculate_margin.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied: use this tool to get a list of supported commodities. No explicit when-to-use or when-not-to-use guidance, but the simplicity of the tool makes it straightforward.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_reportA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It implies a read-only aggregation tool with no side effects. Lacks explicit mention of output format but output schema exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short, front-loaded sentences with no wasted words. First sentence defines purpose, second gives usage guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the number of required parameters and existence of output schema, the description provides sufficient context for the tool's role in a pipeline. Missing parameter descriptions are a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage for 7 parameters. Description does not explain any parameter individually. Parameter names are somewhat self-explanatory but description should compensate given low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it builds a markdown report from collected data, using the verb 'monta' and specifying the resource 'relatório em markdown'. It distinguishes from sibling tools which gather data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use por último, depois de obter preço, margem e tendência', providing clear workflow ordering and when not to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.