Server Quality Checklist

Tool Scores

• search_variableA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses that search spans all design records and uses partial matching, but lacks details on return format, pagination, or performance implications.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence that is direct and front-loaded with key action and scope. No redundant words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description sufficiently covers the operation and parameter usage. No gaps given low complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema only defines 'name' as string with 0% coverage. Description adds meaning: the parameter can be a partial name or description, clarifying how to use the parameter beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'Search', resource 'variable', scope 'across all design records', and method 'by partial name or description'. Distinguishes from sibling tools which are not search-oriented.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description implies usage context (searching by partial name or description) but does not explicitly state when to use this tool versus alternatives like get_design_record or get_classifiers.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_classifiersA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses read-only nature, filter behavior (case-insensitive substring), and contents. However, it omits details on output format, error handling, or dependency on eph_setup (e.g., what happens if not called).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Five concise sentences, each adding unique value: purpose, prerequisite, use case, contents, filter behavior. No unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, the description adequately covers return contents and filter behavior. It includes a prerequisite but lacks details on error conditions or the scope of the filter substring match.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'filter' is well-explained: 'Pass filter to get only matching lines (case-insensitive substring match).' This adds semantic value beyond the schema's empty title, though default value is not mentioned.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Returns the complete CNO 2001 classification' with a specific verb and resource. It distinguishes the tool from siblings like search_variable by focusing on occupation classification retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit prerequisite ('Call eph_setup() first') and use cases ('correctly interpret or recode occupation variables'). Lacks explicit 'when not to use' but gives clear context compared to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_design_recordA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It clearly indicates this is a retrieval operation (returns a dictionary) and states a prerequisite (eph_setup). However, it could more explicitly state that the tool is read-only and has no side effects, though the action of 'returning' implies no mutation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is logically structured: purpose first, then usage instructions, then parameter explanation, then conditional guidance. It is slightly lengthy but each sentence adds value, earning a high score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the existence of an output schema and sibling tools, the description adequately covers its role and prerequisites. It lacks error handling details (e.g., invalid tipo), but the overall guidance is sufficient for effective tool selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% (parameter tipo has no description), but the description compensates fully by listing and explaining all valid values for tipo, including their temporal contexts. This adds critical meaning beyond the schema's mere string type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the official variable dictionary for EPH microdata files, identifying its specific purpose and resource. It distinguishes from siblings by emphasizing its role as a dictionary to be used before referencing variable names.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-call instructions: call eph_setup() first if needed, call before writing code referencing variables, and advises not to use variable names from memory. It also explains the two main scenarios for tipo values and instructs to ask the user if the period is unspecified.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• eph_setupA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries full burden. It discloses that the tool returns workflow, rules, and topics, implying a read-only setup operation. While it does not detail exact return format or side effects, the context suggests no destructive actions. The transparency is good but could be enhanced by stating it is non-destructive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: three short sentences plus a bullet list. It uses formatting (bold, dashes) to highlight key points. Every sentence earns its place by conveying essential instructions, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has zero parameters and no output schema, the description fully explains its role as a mandatory first step, what it returns, and how to proceed. It covers the entire context needed for an agent to use this tool correctly within the workflow.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the input schema is fully covered trivially (100%). The description confirms this implicitly by not referencing any parameters. Since no parameter info is needed, the description adds value by clarifying the no-argument nature implicitly through its focus on the single call.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool's purpose: 'Returns the mandatory workflow, critical rules, and available topics.' It uses strong language like 'CALL THIS FIRST' and 'step 1 of every EPH task,' making the role unmistakable and differentiating it from siblings that are listed as subsequent steps.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance: 'CALL THIS FIRST — before writing any code or referencing any variable. No exceptions.' It also tells what to do after calling this tool, listing specific sibling tools with their purposes, effectively guiding the agent on the correct workflow order.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_methodologyA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but description implies a read-only operation returning documentation. It does not detail error behavior or rate limits, but the context is sufficiently clear for a simple retrieval tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise two-sentence purpose/guidance followed by a well-formatted list of options. No wasted words; every line adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given one parameter, an output schema exists, and the description covers prerequisite, ordered usage, and all topic values. No gaps for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Parameter 'tema' has 0% schema coverage, but the description compensates fully by listing all valid string values with detailed explanations, adding essential meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb 'returns' and specific resource 'official EPH methodological documentation for a specific topic'. The description then enumerates valid topic values, distinguishing it from sibling tools like get_classifiers or get_package_docs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states prerequisite 'Call eph_setup() first if you have not already' and ordering 'Call this BEFORE computing rates, applying weights, or building indicators', providing strong guidance on when to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_package_docsA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but the description fully discloses the tool's behavior: it returns documentation with language-specific details. It also implies a read-only operation without side effects, meeting transparency needs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Five concise sentences covering purpose, prerequisite, usage order, warnings, and parameter values. No redundancy; each sentence adds essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple parameter set and existing output schema, the description fully covers what agents need: return type (documentation), when to use, and parameter mapping.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description adds complete meaning for the single 'lang' parameter by listing valid values ('r', 'python') and their associated packages and classes/functions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns the OFFICIAL API reference for the EPH data package, with a specific verb and resource. It distinguishes from sibling tools like eph_setup and get_classifiers by emphasizing documentation retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to call eph_setup() first and to use this tool before writing any code, providing clear context and preventing misuse. Warns against relying on memory for function names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.