Server Quality Checklist

Tool Scores

• list_datasetsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of disclosing behavior. It explains the return fields (name, versions count, period, schema count) but does not explicitly state safety traits like read-only or mention potential side effects, rate limits, or permissions. For a simple listing, this is adequate but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main action, and every sentence adds value. It avoids redundancy and is appropriately sized for a simple no-parameter listing tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, the description is complete: no parameters, an output schema exists, and the description explains the return semantics. It covers the essential information an agent needs to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema fully covers the input. Per the rubric, 0 params yields a baseline of 4. The description adds no parameter details but none are needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists open data sets of FGIS CS with history depth, using a specific verb and resource. It distinguishes from siblings like dataset_versions (which likely focuses on a specific dataset) by covering all datasets, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as dataset_versions or salary_history. The only context is the purpose itself; there are no explicit exclusions or cross-references to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• dataset_versionsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries the full burden of behavioral disclosure. It notably reveals that the portal itself hides unchanged versions, while this tool marks them—critical context for interpreting results. It does not mention any side effects or read-only status, but for a version-history tool this is implied and not a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: it leads with the core purpose, highlights an important nuance, and ends with an argument reference. It avoids unnecessary details and is easy to scan, though the 'Args' section slightly overlaps with the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a single parameter and the existence of an output schema, the description covers the essential context: what the tool does, how to call it, and a key interpretation detail about unchanged content. It does not explain return value structure, but that is presumably handled by the output schema, so overall it is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no description for the `dataset` parameter, so the tool description must compensate. It does so by giving a concrete example (7707082071-ksrms) and pointing to list_datasets for obtaining valid identifiers. This is sufficient for basic usage, though it could elaborate on format or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly indicates the tool provides version history for a single dataset, including publication timing and content-change status. It distinguishes itself from sibling tools by focusing on version-level details rather than dataset listings or salary metrics. However, it lacks an explicit action verb like 'returns' or 'shows,' making it slightly less directive.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly guides usage by requiring a dataset identifier and referencing list_datasets as the source for valid IDs. It also explains a unique scenario where the tool adds value (detecting versions with unchanged content), which helps the agent decide when to use it. It does not explicitly name alternatives, but the sibling tools are clearly distinct in purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• salary_historyA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool returns rates for each data version since 2020 and notes that the source only has the latest snapshot, which is a useful behavioral trait. However, it does not mention output format, ordering, error behavior, or limitations such as potential missing versions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: a one-sentence summary, a clarification about the data source, and a clearly formatted Args section. Every sentence adds value without redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with an output schema, the description covers the essential aspects: what the tool does, what parameter to provide, and a notable caveat about the source data. It does not explain the output structure, but the output schema presumably covers that. The only minor gap is the absence of guidance on handling missing subjects or unclear matches.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no descriptions (0% coverage), so the parameter explanation in the description is essential. It specifies 'subject' as a full or partial name of a Russian region with concrete examples ('Иркутская' or 'Москва'), clarifying the matching behavior. This adequately compensates for the missing schema metadata, though it could also mention case sensitivity or formatting nuances.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool shows the dynamics of average monthly wages for a 1st-category construction worker by region, with a specific verb 'Показывает' (shows) and a defined scope: rate per data version since 2020. This differentiates it from siblings like salary_growth_ranking and dataset_versions by focusing on historical version-over-version values.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that the source only contains the latest upload and that dynamics are not present there, indicating this tool is needed to see historical version data. However, it does not explicitly compare against alternatives like salary_growth_ranking or provide clear conditions for when to prefer one tool over another.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• salary_growth_rankingA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full transparency burden. It discloses a key behavioral trait—using 'entire available history'—and explains the effect of the ascending parameter on sorting. However, it does not state that the operation is read-only or describe the return format, though the output schema covers that.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two lines: a purpose statement followed by a concise Args section. Every element earns its place, and the parameter explanations are directly mapped. There is no verbose or redundant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core purpose and both parameters, and the output schema is available to document the return structure. It is complete for a simple ranking tool, though it could mention potential edge cases like missing data. Overall, it leaves no major gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no descriptions for the parameters, but this description fully compensates by explaining 'limit' as the number of regions to show and 'ascending' as whether to start from smallest or largest growth. This adds clear semantic meaning beyond the bare type declarations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool ranks regions by salary growth in construction over all available history. This is specific and distinct from siblings like salary_history, which provides historical data for individual entities. The purpose is unambiguous despite using a noun phrase rather than a verb.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a ranking of regions by salary growth is needed, but it does not explicitly contrast this tool with sibling tools like salary_history or list_datasets. There are no exclusions or alternative recommendations, so usage guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.