Server Quality Checklist

Tool Scores

• list_record_setsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the transparency burden. It unambiguously describes a read-only listing operation and enumerates exactly what is returned, including field data types and source chains. There is no misleading mutation or side-effect language; the non-destructive behavior is implicit but clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose first, a helpful definition of RecordSet, and an explicit Usage note. There is some redundancy between the earlier sentence enumerating returned fields and the final Returns list, but overall the structure is logical and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's purpose, output, and relationship to `get_records_preview`, but it does not explain how to provide the input: whether one of `jsonld_url`, `jsonld_path`, or `jsonld_content` is required and whether exactly one should be supplied. Since the tool has three optional-looking inputs, this is a notable gap for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input parameters are already fully documented. The description adds no extra detail about how to choose between `jsonld_url`, `jsonld_path`, and `jsonld_content`, making this a baseline 3 rather than a higher score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists RecordSets in a Croissant/GeoCroissant document and defines what a RecordSet is. It also distinguishes this tool by noting that it returns high-level dataset structure rather than the actual record data, and explicitly points to `get_records_preview` as the next step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage guidance: call this tool to discover which RecordSet names to pass to `get_records_preview`. It does not contrast against every sibling tool, but it establishes a clear entry point in the workflow and implies this is the discovery/listing step.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_geocroissant_spec_referenceA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the behavioral burden. It clearly states the tool returns Markdown-formatted reference documentation and implies read-only behavior with 'Returns' and 'Read this once'. It does not explicitly state that it has no side effects, but nothing suggests mutation or external changes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is detailed but not bloated; the opening sentence states the primary purpose, followed by a useful content breakdown and a 'Usage' section. The RETURN section is a little redundant with the first line, but every part contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter, read-only documentation tool, the description provides enough detail: what to expect, what is included, when to use it, and what follows. An output schema is present, and the description already covers the return format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single 'topic' parameter is fully documented in the schema with all of the valid options described. The description adds no information beyond the schema, so it meets but does not exceed the baseline for a fully covered parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Returns the GeoCroissant specification reference documentation'. It then details exactly what the documentation contains (properties, domains, cardinalities, JSON-LD context, sample, Python snippets), which fully distinguishes this from authoring, validation, and inspection siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit context: 'Read this ONCE before authoring or editing GeoCroissant documents' and then directs the agent to create_geocroissant_scaffold and validate_croissant. It does not strongly contrast with inspecting or searching tools, but it clearly places the reference tool in a workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_structure_graphA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full disclosure burden. It clearly explains what graph is built, what nodes and edges represent, and that it's the internal mlcroissant representation. This is sufficient for a read-only analysis tool, though it does not discuss failure modes or source-selection behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, then detail, usage context, and return information. The Returns section is slightly redundant with the existing output schema, and the text is a bit long for what it conveys; but it earns its place by explaining the graph's internal semantics.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no required parameters and an output schema, plus the description fully conveys the returned graph structure and usage context. The main gap is that three possible input sources are not explained as alternatives or prioritized, which is important for a tool that can take a URL, a path, or raw content.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes all three parameters (jsonld_url, jsonld_path, jsonld_content) with 100% coverage. The description adds no parameter-specific guidance beyond the schema. It could have clarified that the three inputs are presumably alternative sources and whether exactly one is required, but it is not required at baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Extracts the internal structure graph of a Croissant document.' It further details the node and edge types, which clearly distinguishes this static-analysis/lineage tool from siblings like validate_croissant or get_records_preview.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use the tool: reason about dataset lineage and dependencies, explain architecture before writing loading code, and answers questions like 'which files feed this field?' It provides clear context but does not name alternatives or exclusion conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_distribution_urlsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility. It thoroughly explains what is extracted, the output structure, and the count semantics. Potential error conditions or network behavior are not mentioned, but for a pure extraction tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with a one-sentence purpose, followed by a compact extraction details list, a usage note, and a structured return section. There is slight redundancy between 'downloadable URLs' and 'contentUrl', but every block adds useful context without excessive length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given three optional parameters and a defined output schema, the description provides all necessary context: the purpose of starting extraction, a compressed return value structure, and number of items. An agent can correctly invoke the tool with any of the three compatible input sources and interpret the output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All three input parameters (jsonld_url, jsonld_path, jsonld_content) already have descriptions in the schema, so schema coverage is 100%. The description does not add parameter-specific guidance; however, since the schema handles that, the baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb-object pair ('Extracts downloadable URLs from a Croissant document's distribution') and enumerates the exact data captured: contentUrl per FileObject, encoding formats, sizes, checksums, FileSet includes patterns. This fully differentiates it from sibling tools like inspect_geocroissant or get_structure_graph, which handle different aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'Usage' sentence clearly states when to invoke this tool: to obtain concrete download links (e.g., GeoTIFF/COG/ZIP) for ingestion code without parsing the JSON manually. It does not explicitly list exclusions or alternative tools, but the primary use case is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_geocroissant_from_stacA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden. It openly discloses that this runs a real STAC search, derives specific metadata properties, validates through mlcroissant, returns a structured dictionary, and optionally writes to a file when output_filename is provided. This is far more than generic 'creates metadata' and gives the agent a reliable model of side effects and outcomes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than typical, but it deserves the length: a high-level summary, numbered pipeline steps, usage guidance, and a structured returns breakdown. It is scannable and avoids filler. The only minor issue is some near-duplication between the numbered pipeline and the Returns section, but overall the structure earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity with 11 parameters, no annotations, and a rich transformation pipeline, the description covers the key decision-relevant context: input search criteria, output validation, the return dictionary shape, and follow-up tool recommendations. Nothing critical is missing for an agent to decide to call it and interpret the result correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already covers every parameter with 100% description coverage, including defaults, ranges, and types. The tool description contributes little parameter-specific semantics beyond grouping bbox, collections, datetime, and cloud cover as search criteria, and mentioning output_filename in the return path. This meets the baseline but does not add substantial new parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Searches live EO data and generates VALIDATED GeoCroissant metadata from it.' It distinguishes itself from sibling tools by framing itself as the flagship end-to-end pipeline and naming follow-up inspection tools rather than other generators. This gives an agent a clear, unambiguous understanding of what this tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly states this is the tool for turning discovered EO data into GeoCroissant and suggests follow-up sibling tools to use after generation. It does not explicitly contrast with create_geocroissant_scaffold or list search-only siblings, so no exclusionary guidance is given, but the context of when to invoke it is sufficiently clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_geocroissant_scaffoldA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries full responsibility, and it delivers: it explains the scaffold passes mlcroissant validation, what valid=True means, that errors/warnings may be returned when not fully clean, and that an output file path is provided when output_filename is used. This is more than what annotations alone would have provided.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose, detail section, usage instruction, and Returns list. It is somewhat long, and the Returns section overlaps with the existing output schema, so a small amount of redundancy prevents a 5.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a highly parameterized tool with 25 parameters, the description is complete: it explains the generation workflow, validation guarantees, how to continue after creation, what the result contains, and the optional file-writing side effect. An agent has enough context to call and use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 25 parameters have schema descriptions, so the input schema already carries most semantic weight. The tool description adds context about how parameters map into the generated JSON-LD (CRS, resolutions, bands, RecordSet wiring), but it does not explain individual parameters beyond the schema. This matches the baseline for a schema with complete coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb-resource pair: 'Generates a validated GeoCroissant JSON-LD scaffold from parameters.' It clearly differentiates from the STAC-based sibling by emphasizing 'from parameters' and stating the output is modeled on the official GeoCroissant example and validated by mlcroissant.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear workflow: call this first when creating new dataset metadata, then edit the scaffold and re-check with validate_croissant, and review with inspect_geocroissant. However, it does not explicitly mention the main alternative sibling create_geocroissant_from_stac or state when someone should use that instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_records_previewA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and handles it well. It discloses that the tool downloads or resolves FileObjects/FileSets, applies extracts and transforms, may download data for remote distributions, and that inline 'cr:data' rows are returned directly without downloads. It also explains the 'truncated' flag behavior, which is genuinely non-obvious execution context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core function. The Returns section duplicates some information likely provided by the documented output schema, but each section earns its place, and the page's important warning is clearly separated.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for this tool's complexity: it names the necessary input record_set, describes the materialization pipeline, covers download side effects, directs users to 'list_record_sets' for IDs, and describes the outputs. The presence of an output schema reduces the burden of restating return types, and what the description adds — pipelining, download warnings, truncation — is exactly the contextual content an agent needs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 6 parameters are already documented in the input schema with 100% schema description coverage, so the description does not need to re-explain them. It adds practical parameter guidance for 'limit' ('keep `limit` small on large datasets'), and it indirectly reinforces the 'record_set' parameter via the return description, but it does not add substantial new parameter-level meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: it "Materializes the first records of a RecordSet by executing the data pipeline." It also clarifies that this yields actual data, equivalent to 'Dataset.records(record_set)', and is not just metadata inspection — distinguishing it from sibling tools like list_record_sets.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives concrete use cases: 'preview/sample a dataset's actual data before writing training code' and 'sanity-check that a generated Croissant description produces the expected columns and values.' It does not explicitly name alternative tools to avoid, but it clearly implies this is the data-content tool rather than structure/validation/list tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_croissantA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the full behavioral burden and it succeeds: it discloses delegation to the official MLCommons validator, the specific checks performed, support for the GeoCroissant extension, and the exact shape of the returned dictionary including errors vs. warnings. No behavior is hidden.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The structure is logical and front-loaded: purpose, validation steps, usage timing, supported variants, then return values as a bulleted list. It is thorough but not padded; the only minor redundancy is the validation-steps list being slightly over-explained, which prevents a 5.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a validation tool with three well-documented inputs and a documented output dictionary, the description is complete: it covers when to call, what is validated, which document variants are supported, and what will be returned. The provided output schema further reduces the need for the description to explain return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage with clear per-parameter explanations for jsonld_url, jsonld_path, and jsonld_content. The tool description adds no additional parameter-level meaning, such as interaction between them or whether exactly one must be provided, so it stays at the high-coverage baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with the specific verb+resource combination 'Validates a Croissant or GeoCroissant JSON-LD document' and enumerates the full validation pipeline (JSON syntax, JSON-LD expansion, structure-graph construction, schema conformance). This clearly distinguishes the tool from the sibling creation, search, and inspection tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage triggers: call it whenever the dataset description is created or edited, BEFORE publishing, and after any modification of an existing file. It does not name when-not-to-use cases or explicitly route to an alternative tool, which keeps it short of a full usage-and-exclusion statement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inspect_geocroissantA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses an important behavioral trait beyond the annotations: it will reject invalid documents via strict syntax/schema check, which is critical for an agent to know. It also details what content will be returned (core metadata, geo extension properties, distributions, record sets, fields, etc.). Since annotations are absent, the description carries the burden, and it does an above-baseline job by describing the validation side effect and return shape, though it could be even richer about error handling and conflicts among the input parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well structured with clear sections: the core purpose in the first line, the implementation detail in the next, an explicit 'Usage' block, and a 'Returns' section with a bulleted list. Every sentence earns its place and covers the key behavioral aspects. Front-loading is good: an agent can quickly answer 'What does this do?' and 'When do I call it?'.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete given the tool complexity: an inspection tool with three mutually-exclusive inputs and a rich output that is fully described. It even goes slightly beyond the call by describing the return dictionary structure, which combines well with the output schema already present. The only limitation is param precedence, but that is a minor gap and the overall context is fully sufficient for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (all three parameters have descriptions: 'URL', 'Path', 'Raw JSON string'), so the baseline is 3. The description itself does not add much insight about parameter precedence or how to choose among the three parameters when multiple are provided, which is a meaningful gap an agent would face. It relies on the schema, which is adequate but not enhanced.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource ('Inspects a Croissant/GeoCroissant document') and distinguishes it from siblings like validate_croissant, create_geocroissant_from_stac, and get_records_preview, which have different purposes. The mention of mlcroissant and 'strict syntax/schema check' and the structured return digest make the scope unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly names when to use this tool ('to READ and UNDERSTAND a dataset description before consuming it, comparing datasets, or planning how to load records'). It doesn't list exclusions, but the ' 'Usage' sentence plus the sibling context provides clear enough guidance for an agent to select it over the alternative creation/validation/search tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_eo_scenesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It transparently states it executes a real STAC item search via pystac-client against the Earth Search endpoint and lists the filtering by spatial, temporal, and cloud-cover criteria. It also summarizes the return payload. It does not fully detail error cases or rate limits, but for a read-only search tool these are minor gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and efficiently front-loads the purpose, then usage context, and then return format. Each sentence adds value, with no repeated schema details or fluff. The Usage and Returns sections make it easy for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given an output schema exists and all six parameters are documented in the input schema, this description provides sufficient high-level context. It explains what the tool does, the external service it reads, the intended workflow position, and the fields in the result. Nothing critical for selection or invocation is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds limited value for parameter semantics but does map the filter concepts (spatial extent, time range, cloud cover) to the overall API behavior and mentions the curated default collections via 'modality'. It doesn't enrich individual parameter fields beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the operation ('Searches satellite SCENES inside a bounding box'), identifies the resource ('satellite scenes' vs datasets), and clarifies context by distinguishing them as individual acquisitions. It separates itself from sibling search_eo_datasets and routes to the create_geocroissant_from_stac.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly places the tool in a workflow: 'Use after search_eo_datasets ... to check actual data availability' and 'Feed promising results into create_geocroissant_from_stac'. This tells the agent when to use it and points to the related data availability tool, giving clear when/when-not context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_eo_catalogsA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It explains that the registry is data-driven via config/catalogs.yaml, names the current catalog and contents, and specifies the returned dictionary structure. For a read-only, no-argument list operation, this is strong transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The first sentence states the purpose immediately, and the 'Usage' and 'Returns' sections provide practical, scannable guidance. The listing of catalog contents is slightly redundant with the Returns section but not excessive.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple zero-argument tool, and the description covers its trigger, output, and relationship to following search steps. There are no missing details that would prevent an agent from selecting or invoking it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description has no parameter semantics to add. The baseline for zero-parameter tools is 4, and the description appropriately focuses on what the output represents instead.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Lists the Earth observation STAC catalogs registered on this server.' It also distinguishes itself from sibling search tools by stating this is the discovery step before using search_eo_datasets / search_eo_scenes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Call once to see where EO data can be discovered from before using search_eo_datasets / search_eo_scenes.' This gives clear sequencing and routing, making it obvious when this tool should be selected over its siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_eo_datasetsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full transparency burden. It discloses the two-stage behavior: 'Queries hit the topics map first ... then fall back to keyword matching against live collection metadata'. It also states that 'every hit is classified by sensor modality' and lists the returned dictionary content, giving the agent a clear moidal picture of what will happen after invoking the tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with the core behavior, then organized into usage guidance and a return block with no repeated information. Each section earns its place, and code-styled references keep the text scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search-only tool with two optional parameters, an output schema, and clear sibling routing, the description is complete. It states scope, how the query is interpreted, return fields, and recommended follow-up siblings. Missing details such as pagination or error handling are not needed for correct selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers the parameters at 100%, so the baseline is 3. The description adds meaningful semantic context beyond the schema: it explains the query flows through a topics map and then keyword matching, and that every result is modality-annotated. This complements the schema's free-text and modality descriptions rather than merely repacing them.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb and resource: 'Searches Earth observation DATASETS (STAC collections) by keyword' and clarifies it is a 'collection-level search' on the Earth Search STAC API. It also names the sibling operation `search_eo_scenes` and distinguishes dataset-level discovery from scene-level discovery, making this tool's scope unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells the agent when to start here: 'Start here for dataet-level discovery'. It then guides next actions: 'Then use the returned collection ids with search_eo_scenes, or jump staraight to create_geocroissant_from_stac'. This gives clear context for when to use this tool versus its siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.