Server Quality Checklist

Tool Scores

• chi_square_goodness_of_fitC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry full behavioral disclosure. It states the test purpose but fails to mention what the tool returns (e.g., test statistic, p-value, decision), how it handles invalid inputs, or any side effects. The absence of output schema exacerbates this gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, which is concise but excessively minimal. It lacks structure (e.g., bullet points, sections) and does not front-load key distinctions. While it earns its place by stating the basic purpose, it could provide more value in the same space.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given three parameters, no output schema, and a complex statistical context (plus 16 sibling tools), the description is severely incomplete. It omits return format, assumptions, practical usage hints, and differentiation from similar tests, leaving the agent underinf.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description adds no explanation of the three parameters (observed, expected, alpha). The agent receives no semantic help beyond raw schema types and defaults, such as that observed and expected arrays must have equal lengths or that alpha is the significance level.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as a chi-square goodness-of-fit test, stating it checks if observed category counts match an expected distribution. While it names the specific statistical test, it does not differentiate from sibling chi_square_independence, which tests association rather than distribution fit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like one_sample_t_test or chi_square_independence. Assumptions (e.g., expected frequencies ≥5, independent observations) are omitted, leaving the agent to infer usage context from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cramers_vC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must disclose behavioral traits. It only states the formulaic definition and normalization, but does not mention input validation (e.g., negative chi2, non-positive n, rows/cols < 2), the return format, or that it assumes inputs from a chi-square test of independence versus goodness-of-fit. This is insufficient for safe tool invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short sentence, making it concise but not structured to be helpful. It contains no sections, examples, or additional context. Every word serves a purpose but the overall utility is limited.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has 4 required parameters with no schema descriptions, no annotations, and an output schema (whose content is unknown), the description is incomplete. It does not explain how to use the tool, what the output represents, or edge cases. An agent would likely need to infer too much.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must entirely explain the parameters. It does not define what chi2_statistic, n, rows, or cols represent. For example, n could mean total sample size, and rows/cols refer to contingency table dimensions, but this is absent. An agent cannot correctly map real-world values to these parameters without additional context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is an effect size for a chi-squared test of independence, normalized to [0,1]. This specifies the verb (compute effect size), resource (chi-squared test), and a key property (normalization). However, it does not differentiate from sibling effect size tools like cohens_d or cohens_h, though the context of chi-squared independence is implicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use Cramér's V, when not to use it, or how it compares to alternatives such as phi coefficient. There is no mention of prerequisites (e.g., a significant chi-square test) or data assumptions. This leaves the agent without decision-making context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• one_way_anovaC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must cover behavior. It states the test's goal but discloses nothing about the computation, assumptions, output (e.g., F-statistic, p-value), or any side effects. This is minimal transparency for a statistical test.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, which is concise. However, it omits critical information, making it under-specified rather than efficiently concise. For a tool this simple, slightly more detail would not harm conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (statistical hypothesis test with assumptions), zero annotations, no output schema, and two parameters with no descriptions, the description is severely incomplete. An agent would lack the context to invoke the tool correctly or interpret results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain parameters. It does not mention 'groups' (each inner array as a group of observations) or 'alpha' (significance level, default 0.05). The agent has no guidance on how to structure input or what these parameters mean.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: testing whether three or more independent groups have different means. This distinguishes it from siblings like two_sample_t_test (two groups) and chi-square tests (categorical data).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No usage guidelines are provided. The description does not mention when to use ANOVA versus alternatives (e.g., Kruskal-Wallis for non-normal data), assumptions (normality, homogeneity), or post-hoc procedures. Given the many sibling tools, this omission significantly reduces helpfulness.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• power_for_two_proportion_testC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. It only states the statistical purpose but does not explain what the output contains (e.g., whether it returns the power value, effect size, or critical value), nor any side effects or computational limits. The tool is likely a pure calculation without side effects, but the description doesn't confirm this.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise at one sentence, but for a statistical tool with four parameters and an output schema, it is under-specified. A second sentence explaining the output or parameter constraints would be warranted. Still, it is not verbose, so it scores well on brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that this is a moderately complex statistical tool with 4 parameters, no annotations, and no description of the output schema (though one exists), the description is incomplete. It fails to explain the statistical context (e.g., that this is a two-sided test by default?) or how to interpret the return value. The output schema likely provides structure, but the agent needs to know the tool computes power, not sample size, which it does state.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description only mentions n_per_group, p1, and p2 implicitly ('n observations per group', 'difference between two proportions'). The alpha parameter and its default value are not mentioned at all, and no guidance is given on valid ranges for proportions (0-1) or sample size (positive integer). The description adds minimal value over the parameter names, leaving the agent to guess semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states this tool computes statistical power for detecting a difference between two proportions with a given sample size per group. The verb 'power' combined with 'detect a difference between two proportions' makes the purpose specific and distinct from sibling tools like two_proportion_z_test which would compute the test itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly guide when to use this tool versus alternatives. For example, it doesn't mention that this is a pre-experiment planning tool (as opposed to two_proportion_z_test for analyzing collected data) or that the user also has sample_size_for_two_proportion_test for solving the reverse problem. The context is implied but no direct comparison or exclusion criteria are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• paired_t_testB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavioral traits. It only states the tool performs a test, but doesn't mention assumptions (e.g., normality of differences), return values (p-value, test statistic), or potential edge cases. This is insufficient for a statistical test tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 18 words, front-loaded with the core purpose. It is efficient and easy to parse, though it sacrifices some necessary detail. It earns its place but could be slightly expanded without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (statistical test with 3 parameters, no output schema, no annotations), the description is incomplete. It omits parameter explanations, assumptions, return value details, and usage context. The example helps but does not provide enough for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate by explaining what 'a' and 'b' represent (the paired samples). It does not mention any parameters at all, leaving the agent to infer from the vague title 'A' and 'B'. This fails to add meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool tests whether the mean difference between paired observations is zero, using the example 'before/after on the same subjects'. This verb-resource pair is specific and differentiates from sibling tools like 'two_sample_t_test' (independent groups) and 'one_sample_t_test' (single sample).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (paired data) via the example 'before/after on the same subjects', but provides no explicit guidance on when not to use it or how it compares to alternatives like 'two_sample_t_test' or 'one_sample_t_test'. The context is clear but lacks exclusions or direct comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chi_square_independenceB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. The description reveals the test's hypothesis and input structure, but does not mention that it computes a p-value, assumes certain conditions (e.g., sample size), or how it handles invalid input (e.g., zero rows). It adds basic behavioral context but lacks depth for a statistical test tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with a parenthetical example that is concise and front-loaded with the main action. It avoids unnecessary words while still providing a concrete use-case hint. The example adds value without making the description overly long.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of output schema, no annotation coverage, and moderate complexity (a statistical test with assumptions), the description is only partially complete. It states the goal but leaves out output behavior, assumptions, error cases, and parameter details. The tool is still usable with domain knowledge, but an agent might misinterpret the table format or ignore alpha.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate by explaining both the required parameter 'table' and optional 'alpha'. The description only mentions the table conceptually without describing its exact format (e.g., 2D array of numbers). Alpha is completely omitted. There is no guidance on default behavior or additional semantics beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs a chi-square test of independence on a contingency table, with a helpful parenthetical example. The verb 'Test whether...are independent' and resource 'contingency table' are specific, and the example distinguishes this from the sibling chi_square_goodness_of_fit, making the purpose clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is used for testing independence between categorical variables, which differentiates it from chi_square_goodness_of_fit (which tests distribution fit). However, there is no explicit guidance on when not to use it, prerequisites (e.g., expected cell count assumptions), or comparisons with alternative tests like Fisher's exact test.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cohens_hB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations provided (e.g., readOnlyHint, destructiveHint), so the description must fully disclose behavior. It only states the formulaic purpose and transformation but does not mention what the output contains (e.g., the Cohen's h value), whether it accepts invalid inputs (e.g., proportions outside 0–1), or any side effects. For a statistical effect size tool, this lacks crucial transparency about return format and error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short (one sentence) and front-loaded with the key purpose. It avoids unnecessary words and communicates the essential computation. While it could benefit from additional brief details on output or usage, the conciseness is appropriate for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With two required parameters and an output schema present (implied from context signals), the description need not detail return values if the output schema is descriptive. However, given the lack of annotations and low schema description coverage, the description should provide more context about input constraints and interpretation. It is complete in stating the core formula but incomplete for an agent to correctly set up inputs without further assumptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides parameter names (p1, p2) and types (number) with no descriptions or ranges, and schema description coverage is 0%. The description adds minimal meaning by stating that p1 and p2 are proportions, but it does not clarify constraints (e.g., valid range 0–1, order sensitivity, or expected array vs. single values). Given baseline 3 for two parameters with no schema descriptions, the description adds some context but is insufficient for precise parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that this tool computes Cohen's h, which is an effect size measure for the difference between two proportions using an arcsine transformation. It provides a specific verb ('Effect size for') and resource ('difference between two proportions'), which distinguishes it from other statistical tools in the sibling list that focus on tests, other effect sizes, or sample size calculations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for comparing two proportions via the arcsine transform, which is a standard usage of Cohen's h. However, it does not specify when to choose Cohen's h over effect sizes like Cohen's d (for means) or Cramér's V (for categorical associations). No guidance is given on assumptions, such as requiring independent proportions or appropriate sample sizes, or alternative tools for related tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• one_proportion_z_testB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full responsibility for behavioral disclosure. It does not mention what the tool returns (e.g., test statistic, p-value, confidence interval), nor does it state any assumptions (e.g., normal approximation). For a statistical test, this is a critical omission that leaves the agent uncertain about the output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is direct and information-dense. Every word contributes to the core purpose. There is no redundancy or filler, making it highly efficient for an AI agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a statistical hypothesis test with 4 parameters, no output schema, and many sibling tools, the description is too sparse. It fails to explain the return value, assumptions, or interpretation. An agent would need additional context to use this tool correctly, especially to distinguish it from related tests like chi-square goodness-of-fit.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains that 'successes out of n' defines the observed proportion and 'p0' is the hypothesized proportion, giving meaning to the three required parameters. However, it does not explain the optional 'alpha' parameter (significance level) or its default value, and it omits constraints (e.g., successes must be between 0 and n). The description adds partial value but is not fully comprehensive.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool tests whether an observed proportion (successes out of n) differs from a hypothesized proportion p0. The verb 'test' and the resource 'proportion' are specific, and it inherently distinguishes from siblings like two_proportion_z_test (which compares two proportions) and one_sample_t_test (for means).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs. alternatives. The description implies it is for a single proportion, but it does not mention assumptions (e.g., large sample size, independence) or when not to use (e.g., small n, where an exact binomial test might be more appropriate). Given many sibling tools, this lack of differentiation is a significant gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• power_for_two_sample_t_testB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. The description only states the computation's purpose—nothing about side effects (it's a read-only calculation, presumably safe), assumptions (equal n, two-tailed? default alpha already specified in schema but not mentioned as one- or two-tailed), or constraints (e.g., effect_size_d must be positive? n_per_group must be integer?). Lacks disclosure of any behavioral traits beyond the core math.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, 15 words—efficient and front-loaded with the key concept. It includes the essential elements (power, Cohen's d, n, two-sample t-test) with no filler. Could be marginally improved by adding a verb like 'Calculate' at the start for clarity, but as-is it's concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, 3 parameters (0% schema coverage), and an output schema present (but hidden from this context), the description covers the input semantics adequately but misses return-value context. It doesn't describe what the output represents (probability of detection? range 0-1?), which the output schema might handle, but with no annotations, the description should hint at typical output. Also no mention of one-tailed vs. two-tailed defaults, or whether it returns power or other diagnostics. Adequate but incomplete for a statistical tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% (the JSON schema has zero descriptions for parameters), but the description names all three parameters and binds them: effect_size_d (Cohen's d), n_per_group (per group), alpha (significance level). This provides meaning beyond the raw schema. However, alpha defaults to 0.05 but is not explicitly described as optional, and effect_size_d constraints aren't specified. Given the baseline of 4 for 0% coverage and 3 params, this scores a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Statistical power to detect a given Cohen's d with n observations per group, using a two-sample t-test.' This identifies the verb ('calculate power'), resource ('two-sample t-test'), and key inputs (Cohen's d, n per group). It distinguishes from siblings like power_for_two_proportion_test and sample_size_for_two_sample_t_test. However, it lacks a concise verb phrase like 'Calculate' or 'Compute' at the start, and doesn't explicitly say it is for planning an experiment, which slightly reduces clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when planning a two-sample t-test: given effect size and sample size, estimate power. No explicit guidance on when to use this vs. alternatives (e.g., sample_size_for_two_sample_t_test for reverse calculation, power_for_two_proportion_test for proportions, or why this is appropriate for balanced designs). No mention of prerequisites (e.g., assuming equal variance or normality). This is adequate but not instructive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cohens_dB
  3.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full behavioral disclosure. It explains the return format (value as float or null, with warnings array) and explicitly describes the null case (both samples zero variance with unequal means, indicating an infinite effect size) and the recommended action ('use the raw mean difference instead'). This addresses a critical edge case. However, it does not mention other potential issues like missing data, assumptions, or computational limits, but the key behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each serving a distinct purpose: purpose definition, interpretation guidance, return format with edge case. It is front-loaded, no redundant or filler content. Every sentence adds value, and the structure is logical. This is an exemplar of conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (two numeric arrays, no output schema, no annotations), the description covers the core aspects: what it computes, how to interpret results, what the return structure looks like, and a critical edge case. It does not mention assumptions (e.g., independence, normality) or handling of missing data, but for a basic effect size function, the provided information is sufficient for an agent to use it correctly in most scenarios. The lack of explicit parameter descriptions slightly reduces completeness, but the overall picture is clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, meaning parameters a and b have no textual descriptions in the schema. The tool description only refers to 'two samples' implicitly, without naming the parameters or explaining they are arrays of numbers representing the two groups. Since the schema already defines them as number arrays, the description adds minimal clarity beyond the schema. For a tool with zero schema coverage, the description should explicitly describe each parameter's role and constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Standardized mean difference between two samples (pooled SD)', which clearly identifies the tool as computing Cohen's d for two independent groups. It distinguishes from sibling effect sizes like cohens_h (for proportions) and cramers_v (for association) by specifying the measure as a mean difference. However, it does not explicitly state 'independent samples', which could cause confusion with paired designs, though the pooled SD implies independence.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides rough interpretation guidelines (0.2 small, 0.5 medium, 0.8 large) but offers no guidance on when to use Cohen's d versus alternative effect sizes (e.g., cohens_h, cramers_v) or related tests (e.g., two_sample_t_test). It does not mention prerequisites, assumptions, or when not to use this tool. Agents would need external knowledge to decide when to invoke this tool over siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sample_size_for_two_proportion_testA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries full burden. It states the tool calculates required sample size but does not disclose the underlying statistical method (e.g., continuity correction, two-sided test) or any constraints. For a simple statistical calculator, this is adequate but not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single concise sentence of 20 words that front-loads key information. Every word contributes to the purpose without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and 4 parameters (0% described), the description is minimal. It explains what the tool returns (sample size per group) but not the format (integer? rounded up?) or any edge cases (e.g., unequal group sizes). Adequate for a simple tool but not rich enough to fully prepare an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It clarifies that p1 and p2 are proportions to detect a difference between, but it does not explain alpha or target_power beyond the implicit reference in 'at the target power.' Since these are standard terms, the description adds modest value but does not fully compensate for missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it calculates 'how many observations per group are needed to detect a difference between two proportions,' with a concrete example (conversion rates). This distinguishes it from sibling tools like power_for_two_proportion_test (which calculates power) and sample_size_for_two_sample_t_test (for continuous outcomes).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when planning a study with two proportions and a target power, but does not explicitly state when to prefer this tool over siblings (e.g., one_proportion_z_test) or mention assumptions (e.g., independent groups, normal approximation). No exclusion criteria or alternatives are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• benjamini_hochberg_correctionA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool controls FDR (less conservative than Bonferroni) and operates on a batch of p-values. It does not mention assumptions (e.g., independence) or other behavioral traits like side effects or auth needs, which is acceptable for a statistical function.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at two sentences, front-loading the main action and key differentiators immediately. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, and the description covers purpose and comparison. However, with no output schema and missing parameter descriptions, an AI agent may need more detail on how to provide inputs (e.g., format of p-values). It is minimally adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% and the description does not explicitly describe the parameters. 'Batch of p-values' implies the `p_values` parameter, but `alpha` is not mentioned at all. Given the lack of schema descriptions, the description should compensate but only partially does.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: adjusting a batch of p-values for multiple comparisons while controlling the false discovery rate. It distinguishes itself from siblings by noting it is 'less conservative than Bonferroni' and the 'standard choice when testing many hypotheses at once.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly compares to Bonferroni and recommends this tool for scenarios with many hypotheses. However, it does not explicitly state when not to use it (e.g., when strict family-wise error rate control is needed). The comparison provides clear context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• two_proportion_z_testA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It accurately describes the tool as a hypothesis test (inferring typical test behavior) but does not disclose details like the assumptions (e.g., normality via sample size, independence), what the test computes (z-statistic, p-value), or how it handles edge cases (e.g., zeros). It adds context about 'standard test behind comparing conversion rates' which is helpful but incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence that efficiently conveys the purpose and a key use case. It is front-loaded with the core purpose. Loses a point because it could be slightly more structured (e.g., separating purpose from example) but overall is appropriately sized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a hypothesis test with 5 parameters, all required except alpha, and no output schema), the description is adequate for specifying what the test does. However, it does not mention what the tool returns (p-value? z-statistic?), nor does it cover assumptions or preconditions (e.g., independence of groups, sufficient sample size). There is room for improvement to help an agent use the test correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It does not explain the meaning of specific parameters like successes1, n1, successes2, n2, or alpha beyond what their names and types imply. The names are somewhat self-explanatory, but the description misses the opportunity to clarify that successes1/n1 are for group 1 and successes2/n2 for group 2, or that alpha is the significance level. Baseline 3 is appropriate as the description adds limited value over the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is for testing whether two independent proportions differ, which is a specific statistical test (two-proportion z-test). It explicitly distinguishes it from siblings like the one-proportion z-test or chi-square tests by focusing on 'two independent proportions' and providing a concrete use case (A/B test conversion rates).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says this is the standard test for comparing conversion rates between two groups (A/B test), giving users clear context for when to use it. However, it lacks explicit guidance on when NOT to use it or mention of alternatives like chi-square test of independence, which could also apply for proportions in certain situations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sample_size_for_two_sample_t_testA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It correctly discloses that the function requires an effect size and returns two values. However, it omits details like whether it uses a specific approximation method (e.g., Lehr's formula, exact calculation), assumptions (e.g., equal group sizes), or behavior for invalid inputs (e.g., negative effect size). A score of 3 reflects a passable but not thorough disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loads the query purpose, and provides essential information without any wasted words. Every sentence serves a clear function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's straightforward nature (3 parameters, no output schema), the description is largely complete. It explains the tool's purpose, return values, and main parameter (effect size). The only minor gap is the lack of explanation for alpha, which could confuse some users, but overall the description equips the agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning the description must compensate for explaining parameter meaning. It mentions 'Cohen's d' and 'target power,' which map to two of the three parameters (effect_size_d and target_power), but does not explain alpha. The default values for alpha and target_power are provided in the schema, but the description adds value by stating the context around Cohen's d. The slight gap for alpha prevents a perfect score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool does: calculates required sample size per group for a two-sample t-test given Cohen's d and target power, and specifies that both continuous and rounded integer values are returned. This distinguishes it from sibling tools like power_for_two_sample_t_test (which computes power, not sample size) and sample_size_for_two_proportion_test (which is for proportions).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (for sample size planning in a two-sample t-test) but does not explicitly state when not to use it or recommend alternatives. It would benefit from mentioning that it's for independent groups, or pointing to sample_size_for_two_proportion_test for proportion-based comparisons, but it's minimally adequate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• one_sample_t_testA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that the tool returns the t-statistic, degrees of freedom, two-tailed p-value, a confidence interval for the mean, and any assumption warnings. This provides useful behavioral context beyond the parameter schema, especially since no annotations are provided. It covers key output aspects without needing to mention destructive or read-only hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loading the core purpose in the first sentence and listing return values in the second. Every part is essential and clearly phrased.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low parameter count (3), no annotations, and no output schema, the description covers the tool's purpose and outputs well. It lacks explicit assumption warnings in the description (though it mentions 'assumption warnings' as part of output), but the statistical context is clear enough for an agent familiar with t-tests.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate. It does so by explaining 'mu0' as a hypothesized value and implicitly linking 'data' to the sample. It does not detail 'alpha', but its default value (0.05) is common knowledge for significance level. Overall, the description adds meaningful meaning beyond the bare parameter list.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool tests whether a sample's mean differs from a hypothesized value mu0, which is a specific verb+resource pair. It distinguishes this from sibling tools like two_sample_t_test and paired_t_test by explicitly mentioning a single sample and a hypothesized value.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly indicates when to use this tool (for testing a single sample mean against a hypothesized value) but provides no explicit guidance on when not to use it or alternatives. It does not mention prerequisites such as normality or independence assumptions, which are common in t-tests.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• bonferroni_correctionA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden. It correctly labels the tool as conservative. However, it does not disclose other behavioral traits like whether it handles an empty array, returns adjusted p-values, or requires specific input validation beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, short sentence with no wasted words. It states the action, the purpose, and the trade-off (conservative) in a front-loaded manner.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters, no output schema, no nested objects), the description is largely complete for its purpose. It lacks information on return format or error handling, but for a statistical adjustment function, the core use case is well covered. The sibling context helps fill gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does not describe the parameters individually, but the parameter names ('p_values', 'alpha') are self-explanatory for a user familiar with the topic. The default value of alpha (0.05) is in the schema. The description adds context about the adjustment purpose but not parameter details, warranting a 4 given the straightforward parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it adjusts p-values for multiple comparisons to control family-wise error rate. It specifies the tool's conservative nature and distinguishes it from the Benjamini-Hochberg correction sibling by stating the use case (any false positive is costly).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: 'use when any false positive among the batch is costly'. It also implies when not to use it by contrasting with less conservative methods. The sibling tool name 'benjamini_hochberg_correction' provides a clear alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• two_sample_t_testA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses key default behavior (Welch's test, no equal variance assumption) and the alternative variant (equal_var=true). It could further describe return type or hypothesis test details, but given the common statistical test, this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no extraneous words. The first sentence states the core purpose, and the second provides critical default behavior and a conditional use case. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple function (t-test), no output schema is expected. The description covers purpose, default behavior, and a key option. It lacks explicit mention of return value structure (e.g., statistic, p-value) but for a well-known test, this is acceptable. Completeness is high for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for all four parameters. The description explains 'equal_var' (default false means Welch's test) and 'alpha' is implied by the test context, though not explicitly described. The required arrays 'a' and 'b' are clear from the test definition. Some minor improvement could explicitly list all parameters with their roles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Test') and resource ('two independent samples') with the clear goal of determining if means differ. It also distinguishes itself from siblings like 'paired_t_test' and 'one_sample_t_test' by specifying 'independent' samples.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool (comparing means of two independent samples) and when to choose the classic pooled-variance test (set equal_var=true). It does not explicitly exclude alternatives like 'one_way_anova' for more than two groups, but the sibling context makes this implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.