Server Quality Checklist

Tool Scores

• treasury_ratesB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It mentions 'No API key required' but fails to disclose other behavioral traits such as rate limits, data freshness, error handling, or response structure. The tool is read-only, but this is not explicitly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the primary action and resource. No unnecessary words. Each sentence adds value: purpose and a key benefit (no API key).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and three optional parameters, the description provides an adequate overview but lacks details on response format, date range limitations, or possible rate_type combinations. It is sufficient for a simple GET tool but could be more informative.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% so baseline is 3. The description adds context by listing example rate types (yield curve, bill rates, long-term rates) that map to the rate_type enum, and notes that no API key is needed, which simplifies usage. However, it does not explain the year and month parameters beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves US Treasury interest rates and specifies the types (yield curve, bill rates, long-term rates). This distinguishes it from sibling tools like economic_overview, which covers broader economic data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. It does not specify prerequisites, exclusions, or compare with siblings such as economic_overview, which might be more appropriate for general economic data.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• company_filingsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations present, so description must cover behaviors. Lacks details on rate limits, pagination, data freshness, or error handling. Only states search capability and accepted identifiers.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no filler. Front-loaded with purpose. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a simple search tool but lacks output format description, error cases, or special constraints. Could improve with typical response fields or date range treatment.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. Description adds minimal extra meaning (e.g., CIK as alternative) but mostly restates schema. No significant new insight.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the verb 'Search' and the resource 'SEC EDGAR company filings' with examples of form types (10-K, 10-Q, 8-K). Distinguishes from siblings by specifying SEC filings retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage when company filings are needed, but provides no explicit guidance on when not to use or alternatives. No mention of prerequisites or context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• labor_statisticsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses 'No API key required' and 'v1 API', which are useful. However, lacks details on data freshness, rate limits, or any potential side effects (though read-only is inferred).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with purpose, then key benefit. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema provided, and description does not mention return format or data structure. With no annotations and moderate parameter count, could be more complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% description coverage for all 4 parameters. The description adds no extra value beyond the schema, so baseline score is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Explicitly states 'Get US labor market and economic statistics from the Bureau of Labor Statistics (BLS)', with specific examples like unemployment rate, CPI, payrolls. Clearly distinguishes from siblings like treasury_rates or economic_overview.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage for BLS data but no explicit comparison to sibling tools or when-not to use. No mention of prerequisites or exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• company_factsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description covers return format (standardized financial facts) and mentions behavior when no concept is provided (returns summary). However, it lacks details on authentication, rate limits, or whether the operation is read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that efficiently conveys the primary action and output. It is front-loaded and concise, though it could benefit from a slightly more structured format (e.g., bullet points for parameters).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers basic functionality and key output examples but omits details like output format (single value vs. time series), pagination, or rate limits. Given no output schema or annotations, the description is adequate but leaves gaps for complex use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters have descriptions in the schema (100% coverage). The tool description adds value by explaining the 'concept' parameter's default behavior (returns summary if omitted), which goes beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves XBRL financial data from SEC EDGAR, specifying examples like Revenue, NetIncome, Assets. It distinguishes from siblings like 'company_filings' which likely deals with filings rather than standardized facts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for obtaining financial facts from SEC filings but does not explicitly state when to use this tool versus alternatives like 'company_filings' or 'economic_overview'. No when-not-to-use guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• economic_overviewA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It describes what indicators are returned but does not disclose behavioral traits such as data freshness, caching, or that it is a read-only operation. The term 'snapshot' implies non-destructiveness, but transparency is basic.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, starts with action verb 'Get', directly lists content. No fluff or redundancy. Perfectly concise for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and no output schema, the description adequately explains what the tool returns by listing key indicators. It is complete enough for an overview tool, though could mention that it returns a combined data object.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters in the input schema (0 params), so the description cannot add meaning beyond schema. Baseline for 0 params is 4. The description does list the returned indicators, which is helpful for understanding output but not parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it provides a snapshot of key US economic indicators, listing specific ones (Treasury yield curve, unemployment rate, CPI, nonfarm payrolls). It distinguishes itself from sibling tools like treasury_rates (which likely focuses only on rates) and labor_statistics (which may be more detailed but narrower).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a broad overview use case ('multi-source overview in one call'), but does not explicitly state when to use this tool versus alternatives like treasury_rates for detailed yield data or labor_statistics for in-depth labor metrics. Some guidance is implied but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ticker_lookupA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description alone must convey behavioral traits. It states that the tool returns 'ticker symbols, names, and CIK numbers,' making the return value clear. It implies a read-only lookup with no side effects, which matches the tool's nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loading the purpose. It contains no redundant information and is appropriately sized for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With one parameter and no output schema, the description is complete. It explains what inputs are accepted and what outputs are returned. No additional context is necessary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully describes the query parameter with a description. The description adds the guidance 'Search by ticker or company name,' which reinforces but does not add significant new meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Look up company ticker symbols, names, and CIK numbers.' The verb 'look up' and resource 'company ticker symbols' are specific. It distinguishes from siblings like company_facts and company_filings which operate on known tickers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use this tool: to search for a company by ticker or name. It provides examples of input. While it doesn't explicitly state when not to use it, the context implies it's for initial lookup, not for retrieving detailed data. No alternatives are mentioned, but sibling names give enough context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.