Server Quality Checklist

Tool Scores

• fetch_dataA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden and does so well: it explains that CSV is parsed into columns and sample rows, JSON is summarized as structure plus excerpt, other text is shown as-is, and save_path streams content to a file. It also discloses the public http/https restriction, which is important operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured: first the core action, then the content-format behaviors, then the save_path streaming guidance and URL restriction. Each sentence adds distinct value with no repetition or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and no annotations, the description explains the main behavior, the supported content types, how to avoid previewing large/binary files, and URL restrictions. Minor gaps remain, such as possible file overwrite behavior with save_path and handling of download errors, but the essential usage context is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3, but the description adds meaningful semantics beyond the schema: it explains the purpose of save_path for large/binary files and adds the restriction that only public http/https URLs are allowed. This helps the agent choose and set parameters more correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool downloads a distribution URL (downloadURL or accessURL) and inspects its content, with distinct handling for CSV, JSON, and other text. It is specific about the resource and action, though it does not explicitly differentiate from siblings like get_resource in the prose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives practical usage guidance for save_path, recommending it for large or binary files and subsequent analysis with normal tools. However, it does not explicitly say when to choose this tool over sibling tools like get_resource, search, or ask, leaving the comparison mostly implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden and does well: it discloses that an empty query with filters returns all matching resources, that the response includes aggregations with counts, and that aggregations reveal valid filter values. It doesn't cover response structure beyond aggregations, but for a search tool that is meaningful behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, each adding a distinct piece of information: core function, use cases, aggregations behavior, and empty-query semantics. Nothing feels redundant or padded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex 8-parameter tool without an output schema, the description covers the essential behaviors: what is searched, how filtering and pagination work at a high level, and what aggregations are for. It doesn't give full response shape or sorting details, but the schema and the description together are enough to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 88%, so the schema already documents most parameters. The description still adds value by explaining the purpose of aggregations (which filter values exist and their counts) and the empty-query-with-filters behavior, beyond what the schema's includeAggregations description states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific statement: full-text search across title, description, and keywords with filters and pagination. It also frames when this tool is the right choice ('presisjon og kontroll'), which partly distinguishes it from siblings, though it never names an alternative search tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  'Bruk dette når du vil ha presisjon og kontroll' gives an explicit use case, and it adds counting resources and discovering valid filter values via aggregations. It does not, however, explicitly say when not to use it or point to a sibling alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sparqlA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure, and it delivers: it states the query scope, lists graph resource types and common prefixes, and explains return formats for SELECT/ASK vs CONSTRUCT/DESCRIBE. It also warns that the graph is large and advises LIMIT and COUNT usage. The claim that ASK returns rows is inaccurate (ASK returns a boolean), which slightly undermines transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than a minimal one, but every section earns its place: purpose, when to use, graph contents, prefixes, return types, and a size warning. It is front-loaded with the core purpose and then progressively adds supporting detail, with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a SPARQL tool with no output schema and no annotations, this description covers the essential context: target graph, supported query forms, return serializations, and querying pitfalls. It could mention timeouts, read-only guarantees, or clarify ASK's boolean return, but overall an agent has enough information to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline applies; the description adds little about the two parameters themselves. It does provide context that helps construct the query (prefixes, resource types), but it does not elaborate on maxRows behavior beyond what the schema already states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: "Kjør en SPARQL 1.1-spørring mot hele den høstede DCAT-grafen." It clearly defines the tool's scope and distinguishes it from the search sibling by stating when SPARQL is needed (counts, grouping, cross-references, unindexed properties). The resource types and prefixes further cement what the tool operates on.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to use this tool when "søket ikke rekker til" (search is not enough) and gives concrete use cases: counts, grouping by publisher, cross-references, and unindexed properties. It does not mention alternatives such as ask or get_resource, so exclusions are not exhaustive, but the primary routing guidance is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• askA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It discloses expected latency ('Bruker rundt 4 sekunder'), default scope ('Søker i datasett med mindre du setter type'), query constraints, and the ranked-with-justification return style. This is substantial context beyond raw functionality.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four tight sentences, each earning its place: function, use case, constraints, latency, and default scope. Front-loaded with the core action and outcome. No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter tool with no output schema, the description covers inputs, defaults, behavior, latency, and intended scenario. It could mention result-count or error behavior, but nothing critical is missing for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents both parameters well. The description's mention of '3–255 tegn' and 'helst norsk' largely restates schema constraints and query description. It adds the default-scope clarification, but baseline 3 is appropriate because the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Still et spørsmål i naturlig språk') and a concrete outcome ('få rangerte ressurser tilbake, hver med en begrunnelse'), clearly distinguishing it from a generic search. It also offers an explicit use case ('Start her når du ikke vet hva datasettet heter') that separates it from sibling tools like get_resource or search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description tells the agent when to use this tool ('når du ikke vet hva datasettet heter') and why it fits over technical alternatives ('Krever ingen fagtermer'). It does not enumerate alternatives or say 'don't use when X', but the guidance is concrete enough to route a call correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_organizationA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral burden. It discloses the bidirectional lookup behavior and explains the hierarchical meaning of orgPath with concrete examples. It does not describe result multiplicity or limit behavior, but the core lookup behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three compact, front-loaded sentences: core purpose first, then a concrete orgPath example, then the reverse lookup case. Every sentence earns its place with useful, non-redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description still covers the essential inputs, the returned orgPath concept, and both lookup directions with an example. It lacks explicit response shape and limit details, but it is actionable enough for an agent to invoke correctly in most cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema describes `name` but not `limit`, so schema coverage is 50%. The description adds useful meaning to `name` by showing it can be a full/partial name or an organization number and by linking it to orgPath, but it does not explain the `limit` parameter or behavior when multiple organizations match.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Finn organisasjonsnummer og orgPath for en utgiver'. It clearly distinguishes this from generic siblings like search or get_resource by focusing on publisher identifier lookup and the orgPath filter value.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit contexts: use it to obtain the orgPath needed for publisher-filtered searches, and use it in reverse to resolve organization numbers found in aggregations. It does not name alternatives or explicitly state when not to use it, so it falls just short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_resourceA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of explaining behavior. It discloses that the tool returns catalog metadata, that datasets expose distributions with downloadURL/accessURL, and that setting format to turtle/jsonld/rdfxml changes the response to an RDF graph. This goes beyond the schema by revealing output structure and interpretation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three focused sentences, each earning its place: the first defines scope and lookup keys, the second explains the most important output field and the downstream action, and the third describes the format switch. The most important operational detail is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with five parameters, no output schema, and no annotations, the description provides a strong practical context: where the id comes from, what the key dataset field contains, how to continue to fetch_data, and how to request alternate formats. The main gap is that the `full` flag and the type-required constraint are left for the schema rather than integrated into the usage narrative.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is high (80%), so the baseline is 3, but the description adds meaningful semantics beyond property names: it explains the id/uri distinction, highlights distributions as the key dataset field, and explains the functional effect of the format parameter. It does not cover the `full` parameter explicitly, but that is already described in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific retrieval verb ('Hent') and names the exact resource scope: everything the catalog knows about one resource, addressed by FDK-id or RDF-URI. It clearly differentiates from sibling tools by positioning itself as the metadata lookup step between search and fetch_data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives concrete guidance on when to use the tool: after a search hit, with an FDK-id, or directly with an RDF-URI. It also explains the downstream workflow—extract downloadURL/accessURL from distributions and pass them to fetch_data—and when to switch format for an RDF graph. It does not explicitly list when not to use it relative to find_organization or sparql, but the intended flow is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.