Skip to main content
Glama

Validate an ai-catalog.json

validate_catalog

Check an ARD catalog document for conformance. Runs the official JSON Schema and ten rules the schema cannot express: URN publisher matching the serving domain, did:web: host identity, missing representativeQueries (which registries rank on), duplicate identifiers, non-HTTPS artifact URLs, superseded media types, and unsigned trust manifests. Returns a grade and every finding with its JSON path.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
domainYesDomain the catalog is served from. Needed to check that entry URNs name the right publisher, which is how copied examples get caught.
catalogYesThe catalog document as a JSON string.

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden and does well: it discloses the ten rules, the schema validation, and the return format (grade + findings with JSON paths). It doesn't cover failure modes or side effects, but for a validator this is sufficient context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but front-loaded with the core purpose. The list of rules in one sentence is long yet each item is meaningful. It could be slightly more scannable, but no content is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description explicitly states return values (grade and findings with JSON paths). It covers the tool's complexity by listing all ten rules and provides domain context, making it complete for an agent to select and invoke the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already fully documents both parameters. The description adds no extra parameter-level meaning beyond what's in the schema, giving the baseline score of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool validates an ARD catalog document for conformance, enumerates specific rules, and mentions the return of a grade and findings. This distinguishes it from siblings like generate_catalog (creates) and audit_domain (domain-focused).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies its use for conformance checking and gives context on why certain rules matter (e.g., registries rank on representativeQueries). However, it does not explicitly name alternatives or state when not to use it, so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

A4.5/5.0
Disambiguation5/5

Each tool has a clearly distinct responsibility: audit_domain checks a live domain's discoverability and trust across ARD paths, while validate_catalog checks a static document against schema rules. search_registry and get_capability follow the classic query-versus-fetch-by-ID pattern. No two tools would plausibly compete for the same call.

Naming Consistency5/5

Every tool follows a strict lowercase snake_case verb_noun pattern: audit_domain, generate_catalog, get_capability, search_registry, validate_catalog. The two catalog-related tools use distinct leading verbs (generate vs. validate) that reinforce their different purposes rather than confusing them.

Tool Count5/5

At 5 tools, the server is lean and well-scoped, sitting squarely in the ideal 3-15 range. Each tool maps to a distinct stage of the ARD lifecycle—search, fetch, generate, validate, audit—with no redundancy or bloat.

Completeness5/5

The toolset covers the full agent workflow end-to-end: search and get for discovery, generate and validate for authoring, and audit for trust assessment. There are no dead ends or fabricated outputs, as generate_catalog explicitly returns underivable fields as a to-do list, and the domain doesn't obviously require update/delete or registration tools.

Resources