Server Quality Checklist

Tool Scores

• compute_sec_cagrB
  3.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true, and the description does not contradict this. The description adds behavioral information: it refuses non-positive or missing endpoints, and it mentions the internal method ('log-walk + Go parity') and HTTP endpoint, offering valuable detail beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short (two sentences) and front-loads the computation verb, but the first sentence is dense with technical jargon that may obscure meaning. The second sentence is clear. It could be more accessible without losing substance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema, so return value explanation is unnecessary. However, the description lacks prerequisites (e.g., need for pre-loaded data) and does not relate to sibling tools like lattice_arith_evaluate or query_sec_*. It is adequate for a simple computation but misses contextual cues for an agent navigating multiple tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema coverage is 100% with descriptions for all parameters. The description does not add any additional meaning or context to the parameters (e.g., format, constraints, relationships) beyond what the schema already provides. Baseline score of 3 is appropriate given the schema's completeness.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states it computes 'CAGR' using 'EDGAR facts', which clearly identifies the output and domain. However, jargon like 'Domain 31 lattice' and 'log-walk + Go parity' may reduce immediate clarity for an AI agent. It distinguishes itself from sibling query tools by mentioning a derived computation over raw data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as query_financial_sec or query_sec_metric_exact. The only constraint mentioned is refusal of non-positive or missing endpoints, which is insufficient for an agent to understand prerequisites or preferred use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lattice_arith_evaluateB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description states the return structure ('arith_method, parity_ok, series[], and cell provenance'), which adds some behavioral context beyond the readOnlyHint annotation. However, it does not disclose potential error states, authentication needs, or behavior when cells are missing. Since the annotation already marks it as read-only, the description provides moderate additional value but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences with no wasted words. The first sentence defines the purpose and operations, the second lists the return values. It is front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 8 parameters, a full schema, and an output schema (not shown but noted), the description is adequate but incomplete. It does not mention prerequisites (e.g., whether EDGAR cells must already be fetched), nor does it explain how this tool relates to siblings. The return outline is helpful but not comprehensive. Overall, it meets basic needs but has gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already describes all parameters. The description lists the return fields but does not add extra meaning or constraints for parameters (e.g., how 'op' affects other fields or valid factor ranges). With full schema coverage, the baseline is 3, and the description does not elevate this.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool performs native arithmetic (cagr, ratio, multiply, divide, yoy_series, product_chain) on verified FINSEC EDGAR cells. It specifies the verb (evaluate arithmetic) and the resource (EDGAR cells). However, it does not explicitly differentiate from the sibling tool 'compute_sec_cagr', which may be a more specific CAGR function, leaving some ambiguity about when to use this versus that.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when or when not to use this tool compared to its siblings. For example, it does not clarify when to prefer this over 'compute_sec_cagr' or 'query_financial_sec'. There are no explicit usage contexts, prerequisites, or exclusions, which is a significant gap given the presence of related tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_sec_metric_exactA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint annotation, the description adds significant behavioral: 'Server-side verified fact with provenance' indicates trustworthiness, 'SAFE_REFUSAL when missing' clarifies behavior on missing data, and 'No LLM math' warns against using it for calculations. This is rich context beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short sentences, each earning its place: first states purpose, second adds behavioral guarantees, third clarifies a key limitation. No redundant or filler content. Front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with an output schema and 4 parameters (3 required), the description covers purpose, behavioral traits, and limitations. It omits mention of the optional 'low_tokens' parameter, which is a minor gap. Otherwise, it is sufficiently complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description mentions the required parameters (company, metric, period) but adds no extra meaning beyond the schema's descriptions. The optional 'low_tokens' parameter is not addressed. No parameter-specific semantics are added.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool does 'Exact EDGAR metric recall for company+metric+period,' which is a specific verb+resource combination. The additional context 'No LLM math' and 'Server-side verified fact' distinguishes it from siblings like compute_sec_cagr and lattice_arith_evaluate by emphasizing pure recall without computation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through 'No LLM math' and 'Server-side verified fact,' suggesting it should be used when a precise, verified metric is needed. However, it does not explicitly state when to use this tool versus alternatives like compute_sec_cagr or query_financial_sec, nor does it provide any when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_sec_sector_peersA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds value beyond the annotations. While annotations declare 'readOnlyHint: true' and 'destructiveHint: false', the description explains the dynamic derivation mechanism (SIC codes, 10-K disclosures) and that results are certified (XBLR). It does not mention rate limits or pagination, but the output schema likely covers structure. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loading the main verb and purpose. Every sentence adds distinct value: the first defines what it does and its data source, the second gives usage guidance. No redundancies or filler. Perfectly sized for a CEO.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 5 parameters (100% schema coverage) and an output schema, so the description doesn't need to explain return values. The description covers purpose, derivation method, and use case. However, with 5 parameters and a dynamic peer group, a hint about how 'limit' or 'period' affect results might be helpful but is not critical. The context is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all 5 parameters are documented in the schema. The description itself mentions no parameters explicitly, but the schema's own descriptions are clear and sufficient (e.g., 'sector_or_industry' includes examples). The description does not add new parameter-level meaning, but with complete schema coverage, baseline 3 applies; the tool benefits from the detailed parameter examples in the schema, justifying a 4 for effective overall understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool returns 'certified SEC EDGAR XBRL metrics across an industry peer group' derived from dynamic sources like SIC codes and business segment disclosures. This clearly distinguishes it from siblings like 'query_sec_metric_exact' (which likely focuses on a single company or metric) and 'query_financial_sec' (possibly a broader query). The verb 'returns' and resource 'peer group metrics' are specific and actionable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly recommends use cases: 'Use this for sector screening or when no specific company ticker is provided.' This gives clear when-to-use guidance. However, it does not mention when NOT to use it or suggest alternatives among the siblings (e.g., 'query_sec_metric_exact' for single-company queries). The exclusion guidance is implied but not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• valuation_inputsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true, so the tool is known as a read operation. The description adds valuable context about the source (SEC/EDGAR certified) and the content structure ('12 packs including dcf_valuation_inputs'), which goes beyond annotations. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with front-loaded purpose and clear exclusions. No superfluous words; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present, return values are handled. The description covers tool scope, source, exclusions, and content structure. A minor gap is that it doesn't explain why one would choose between the 12 packs, but that is managed by the pack parameter's enum values in the schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters well. The description does not add additional parameter-level meaning beyond the schema, but that is acceptable given full coverage. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a 'Certified SEC/EDGAR valuation inputs pack' with specific scope ('12 packs including dcf_valuation_inputs historical FCF'), distinguishing it from market-based alternatives. The verb is implicit but clear from context ('Get'), and resource is precisely specified.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when NOT to use this tool by listing exclusions: 'Not market EV, P/E, trading DCF, or macro gauges (Buffett Indicator / CAPE).' This provides clear boundaries and helps the agent select alternatives among siblings like query_financial_sec or vln_capabilities_overview.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• vln_capabilities_overviewA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint annotation already indicates no destructive effects. The description adds transparency about what is returned (structured overview, compact JSON if low_tokens=true) and the domain context (finsec.caleralabs.com), which is valuable beyond the annotation alone.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the key action and resource, enumerates major content categories, and specifies the platform domain. No wasted words; every element earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no required parameters, a rich output schema (not shown but present), and annotations cover read-only safety, the description is complete enough. It explains what categories are included and the optional parameter effect, though it could briefly note that the output is a single JSON object format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% and there is only one optional parameter. The description adds value by explaining that low_tokens affects structure ('compact JSON schema'), which goes beyond the schema's brief description. It efficiently covers the parameter's impact on output format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs ('Returns') and a clear resource ('structured overview of all VLN capabilities'), listing included categories (EDGAR valuation packs, Domain 31 operations, SEC calculations). It differentiates from siblings by being a single source of truth overview vs. the specific computation tools like compute_sec_cagr or lattice_arith_evaluate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it (to get a complete overview of capabilities) and the scope ('product scope boundaries on finsec.caleralabs.com'). It does not explicitly say when not to use it (e.g., for specific valuations use sibling tools), but the sibling context and tool name make this clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_financial_secA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With annotations already declaring readOnlyHint=true, the description goes beyond by disclosing the SAFE_REFUSAL behavior, making clear the tool will refuse to invent numbers, and explaining how currency conversion works. The only minor gap is that it doesn't explicitly state whether the tool can handle non-monetary queries or what constitutes an unsupported question, but overall it adds substantial behavioral context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, packing critical guidance into just three sentences. The first sentence states purpose, the second covers the key refusal behavior, and the third handles currency defaults. Every sentence adds unique value, and the structure fronts the most important information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (handling return values), 100% schema coverage, and clear annotations, the description provides a strong frame for usage. It covers core behaviors (refusal, currency conversion) and usage tips. However, it could further elaborate on what types of queries are in-scope (e.g., supports only numeric financial facts, not textual descriptions) and whether the low_tokens parameter significantly changes output structure. Slight gap for a tool with 4 parameters and 1 requirement.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so each parameter already has a schema-level description. The tool description adds value by explaining the interplay between parameters (e.g., how target_currency and usd_only relate) and providing usage context like 'in USD' in the query. This enriches the semantics beyond the schema, though not every nuance is covered (e.g., what happens if both target_currency and usd_only are set).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific phrase 'Certified SEC fact query' and clarifies the unique nature of the output: 'Returns verified values with provenance or SAFE_REFUSAL.' This clearly distinguishes it from siblings like compute_sec_cagr or query_sec_metric_exact, and states what the tool does with a specific verb ('query') and resource ('Certified SEC facts').

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to use this tool and what to expect: 'Does not invent numbers. Treat SAFE_REFUSAL as success-of-honesty, not a tool failure.' It also provides guidance on currency handling: 'Monetary answers default to the filer's reporting currency; request FX conversion via target_currency / usd_only or natural-language 'in USD'.' This is comprehensive guidance that helps an agent decide and avoid misuse.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.