Server Quality Checklist

Tool Scores

• get_dataC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states 'Fetch', which implies a read-only operation, but it does not mention any limits, mutual exclusions, or response structure. The output schema covers return format to some extent, but behavioral traits are otherwise undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short sentence, which is concise and front-loaded. It avoids unnecessary details, though it lacks structural elements like usage examples. Every word contributes to the core meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has five parameters with mutual exclusions and a set of related sibling tools. The schema is detailed and an output schema exists, so return values are covered. However, the description does not provide overarching context or guidance on how to combine with other tools, making it minimally adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with descriptions for all parameters, so the description adds little beyond the schema. It mentions 'one or more SINIM variables' which aligns with the 'codes' parameter, but does not clarify other parameters or their relationships.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Fetch' and identifies the resource as 'municipal data for one or more SINIM variables', which clearly states the tool's function. It doesn't explicitly differentiate from sibling tools like preview_data or export_data, but the name 'get_data' and the description imply core data retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives such as search_variables or preview_data. The description provides no context about appropriate use cases, exclusions, or selection criteria among sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_municipiosB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden for behavioral disclosure, but it only states the basic action and filter. It omits details about result volume (~345 municipalities), pagination, or any potential side effects, leaving the agent without important behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One short sentence ('List municipalities, optionally filtered by region.') with no filler or redundancy. It is appropriately front-loaded, and every word earns its place for a tool of this simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, one optional parameter, and an existing output schema, the description is adequate. However, it lacks any mention of related tools or output expectations, leaving minor contextual gaps that could confuse an agent choosing between list_municipios and search_municipalities.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema fully documents the 'region' parameter (coverage 100%), so the baseline is 3. The description merely reiterates that filtering is optional, adding no new semantic meaning beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('List') and resource ('municipalities') with an optional filter ('by region'), clearly conveying the tool's function. It doesn't explicitly differentiate from sibling 'search_municipalities' but the list vs search distinction is implicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving municipality lists, optionally scoped by region, but provides no explicit guidance on when to choose this over search_municipalities or list_areas, nor any exclusion criteria. It gives a clear context but no alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• export_dataA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that data goes to disk and mentions a 'generous safety ceiling' for long-running queries, which adds useful context. However, it does not explain behaviors like file format, destination path, failure modes, or whether the operation is destructive, leaving significant gaps for a write tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, with a clear main action in the first sentence, a practical prerequisite in the second, and a reassurance about safety. Every sentence contributes value with no wasteful filler, making it well-structured and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 6 parameters, no annotations, and only a basic output schema indicator, the description is incomplete. It omits parameter semantics, output location/form, and clear differentiation from get_data. The brief safety note and preview advice are helpful but do not fully cover the tool's operational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description never mentions any of the six parameters, including required ones like codes and years. The description provides no additional meaning beyond the schema, failing to compensate for the lack of parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: 'Write a full SINIM panel to disk instead of the model's context.' It distinguishes from siblings like get_data and preview_data by emphasizing output to disk rather than context, and recommends preview_data as a precursor. This is a clear, specific purpose with sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description tells the user to call preview_data first for sanity-checking, which is a clear usage guideline. It implies this tool is for large exports that exceed context limits, contrasting with in-context retrieval. However, it does not explicitly enumerate alternatives or state when not to use it, so it lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_variable_infoA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must convey behavioral traits. It describes a read-only operation ('Get'), but does not disclose potential side effects, authentication requirements, or the exact scope of 'full metadata'. The absence of contradiction but limited detail yields a mid score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single efficient sentence, but it could be more informative without sacrificing conciseness (e.g., mentioning that output includes full metadata fields). Still, it is not verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, output schema present), the description is mostly adequate. However, it lacks details on what constitutes 'full metadata' and does not reference the output schema for completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a single parameter 'code' already described. The description adds no additional meaning or usage context beyond what the schema provides, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and the specific resource ('full metadata for a single SINIM variable code'), distinguishing it from sibling tools like 'get_data' (retrieves data values) and 'search_variables' (finds variables by criteria).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing metadata for a specific variable code, but it does not explicitly state when to prefer this tool over alternatives like 'search_variables' or 'get_data', nor does it provide context on prerequisites or limitations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• preview_dataA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the full burden. It does indicate the tool is a bounded preview and contrasts it with export_data which writes to disk, implying it does not write. However, it doesn't mention auth requirements, rate limits, or any side effects beyond the read-only nature, so full transparency is not achieved.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with two sentences and a line break. It front-loads the core purpose and provides actionable guidance without waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The presence of an output schema helps with return values, but the tool has six parameters with no schema descriptions and no param explanations in the description. While the purpose and usage are clear, the missing parameter semantics leave a significant gap for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description provides no parameter-level explanations. It only hints at the limit parameter indirectly with 'bounded preview,' but doesn't map any other parameters or provide usage examples. The description does not compensate for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: to inspect a query's shape and coverage before full export. It distinguishes itself from the sibling export_data by explicitly recommending export_data for writing the complete panel to disk.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says to use this before calling export_data, and provides a clear context: bounded preview for checking columns, coverage, and sample values. It also implies not to use it when you need the full result, which is handled by export_data.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_areasA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Description discloses that results are distinct area names and sorted, which is sufficient behavioral detail for a read-only list tool. No annotations present, so description carries full burden.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words. Clearly states purpose and return format.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters, an output schema, and a simple list operation, the description is fully complete. It specifies what is returned (distinct area names) and how (sorted).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so schema coverage is 100%. Description does not need to add parameter details, and it appropriately omits them.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it lists all SINIM subject areas with examples (finance, education, health), distinguishing it from siblings like list_municipios or list_years which list other entities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this vs alternatives. The simplicity and lack of parameters make it straightforward, but mentioning when to prefer this over other list tools would improve it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_yearsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It explains the dynamic discovery and return format (ascending years). Though it doesn't explicitly state read-only safety, it's clear from context. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences: purpose, dynamic discovery detail, return format. No unnecessary words, front-loaded with the main action. Highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with an output schema, the description covers the essential aspects: what it returns (years ascending), how it works (dynamically discovered), and example range. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so baseline is 4. The description adds value by explaining the output (available years ascending) and the dynamic nature, which is helpful beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists the years with data available in SINIM. It uses a specific verb-resource pair ('list years') and distinguishes itself from sibling tools like get_data, list_areas, etc., which focus on other aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions dynamic discovery and self-updating, which implies usage context, but does not provide explicit when-to-use or when-not-to-use guidance relative to sibling tools. No alternatives are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_variablesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses fuzzy, accent-insensitive, and case-insensitive matching. It doesn't mention side effects or read-only nature, but for a search tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: first states purpose, second adds key behavioral detail. No fluff, front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given output schema exists and moderate complexity, description covers search intent and fuzzy behavior. Could mention pagination or sorting, but limit is described.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, baseline 3. The description adds value by specifying matching behavior (fuzzy, accent-insensitive) beyond the schema, and explains area filter as substring match.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the verb 'fuzzy-search' and the resource 'SINIM variables', with optional area filter, clearly distinguishing from sibling tools like get_data or get_variable_info.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that matching is accent- and case-insensitive, providing useful context for when to use this tool. However, it does not explicitly contrast with alternatives or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_municipalitiesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of disclosing behavior. It explicitly states that matching is accent- and case-insensitive, with a concrete example ('nunoa' finds 'ÑUÑOA'). It also uses the term 'fuzzy-search', implying approximate matching. This goes beyond a simple tautology and provides meaningful behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, front-loaded with the primary purpose. The second sentence adds a key behavioral detail. No filler or redundant information; every word contributes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple search tool with an output schema, this description is complete. It states what the tool does, how matching behaves, and the optional region filter. The output schema covers return values, and uncertainty about ordering/pagination is acceptable given the presence of the limit parameter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers 100% of the parameters with descriptions, but the description adds value by explaining the fuzzy/accent-insensitive matching behavior for the query parameter. It also reinforces that region is an optional filter. This enriches the schema's bare definitions without redundancy.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Fuzzy-search Chilean municipalities by name, optionally filtered by region.' The verb 'fuzzy-search' is specific and distinguishes this from sibling tools like list_municipios, which likely provides a full list rather than a name-based search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the primary use case: when you need to find a municipality by name with fuzzy matching, optionally narrowing by region. It does not explicitly name alternatives or exclusions, but the context is clear enough that an agent would know to use this tool for name-based search rather than listing all municipalities.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.