ModelSpec

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so: bearer API key requirement, cost of 2 credits with per-credit pricing, the upstream POST endpoint it proxies, and a 16 KB response cap. It also discloses output behavior (caller order preserved, unknown values stay null), which is exactly the kind of behavioral context annotations would otherwise supply.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded and every sentence carries operational facts (auth, price, endpoint, size cap). Slight waste in 'It doesn't choose a model and does not rank or choose a winner', where 'choose' is repeated twice, but the overall density is appropriate for the information delivered.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description must describe returns — and it does, listing the returned data shape (snapshot facts, offerings, evidence, sources), ordering guarantees, null handling and the 16 KB truncation limit. Combined with cost and auth disclosure, an agent has everything needed to call and interpret this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single `models` parameter is fully documented with pattern, min/max items and an example id. The description's '2 to 4 named models' and 'in caller order' restate the schema rather than extend it, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb and resource ('Which facts, offerings and evidence do 2 to 4 named models have?') and declares the boundary that separates it from the ranking/decision siblings ('does not rank or choose a winner'). An agent can distinguish it from `rank`, `decide` and `model_info` without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The negative case is explicit — it is not for choosing or ranking a winner — which steers agents away from misuse. However it never names the sibling tools (rank, decide) that should be used instead, so the routing is implied rather than given. The scoping to 2–4 named models is clear context for when this applies.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and discharges it: it discloses the auth requirement ('Needs an API key (Authorization: Bearer)'), per-call credit cost and dollar pricing, the concrete endpoint, the 16 KB response bound, decision_id being 'a citation, not a stored lookup,' that 'task free text is rejected,' and that nulls mean unresearched. It also spells out failure/trim behavior and links to recovery docs. This is unusually rich beyond any structured field.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded, but the body is an undifferentiated ~700-word wall that interleaves output rules, epistemological cautions, pricing, HTTP details, and a worked example with no headings or grouping. Much content is valuable, yet the density and repetition (repeated 'never claim/never show/never add' rules) make it hard to scan, and much of it is report-style guidance rather than tool-selection information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a complex tool with no annotations and no output schema, and the description compensates thoroughly: it enumerates the answer components, output size limits, default and expanded field sets, error/trim handling, cost model, auth, and snapshot pinning, plus a concrete right/wrong reporting example. An agent has essentially everything it needs to call the tool and interpret the response.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 100%, yet the description still adds real semantic meaning: 'Put Musts in where: these gates exclude. Put Prefers in optimize.weights: weights rank and never exclude. Unknown values go to may_qualify,' plus 'explicit fields, including null, is sent as given' and the explain=none defaults (limit=10, fields model_rank, cost_per_task, estimates, p_best). It clarifies the gating-vs-ranking distinction that the raw schema does not convey, well above the baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The opening names the job concretely — 'Which models fit this job?' with 'Returns answer, ranked results, may_qualify, reading and recovery hints, decision_id and snapshot' — so the agent knows it is a spec-driven model-selection/decision tool. It also draws a behavioral boundary against generic routing ('Not a router: call it once per job or role, not per request'). It never contrasts itself with the adjacent siblings rank, compare_models, or policy_check, so the purpose is clear but not fully sibling-differentiated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage context is explicit: 'Call decide early with a template-based Spec; refine from reading and recovery hints,' 'call it once per job or role, not per request,' 'Read `next` before another call,' and 'One vocabulary lookup per unknown facet, then decide.' It also says how to route requirements into where vs optimize.weights vs may_qualify. What it lacks is explicit when-not-to-use guidance relative to siblings such as rank, compare_models, or policy_check.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden well: it discloses no key/auth header is forwarded, that the call is free, the privacy constraint on note content, and the possible response statuses including that storage is currently off. It doesn't detail retry/rate-limit behavior, but coverage is strong for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the purpose, then constraints, then the endpoint and privacy caveats. Every sentence adds an operational fact with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite no output schema, the description explains the return status ('recorded' or 'not_recorded') and its caveat, and covers auth, privacy and usage context. Nothing an agent needs to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real meaning by instructing that the answer's decision_id must be included (the schema marks only rating as required) and reinforcing the note privacy constraint.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource: submitting a reliability rating ('reliable, unreliable, trustworthy, untrustworthy or confusing') about a ModelSpec answer, and names the exact endpoint it proxies. An agent can distinguish this from siblings like decide or policy_check immediately.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly scopes when to use it ('After you act on an answer') and when not to ('Not for questions or support'). It also tells the agent to include the answer's decision_id, which is the trigger condition.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does reasonably well: it discloses the authentication requirement ('Needs an API key'), the exact resource returned, and clarifies this is informational rather than a decision. It is implicitly a read-only listing. It omits rate limits, caching, or error behavior, which keeps it below a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences: the question the tool answers, the payload it returns, then the prerequisite and the alternative. Zero filler, and the routing hint ('use decide') is placed last where it belongs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter, read-only metadata tool with no output schema and no annotations, the description covers what is returned, what auth is needed, and how it differs from the decision tool. The unresolved name-vs-content mismatch ('use_cases' vs ranking profiles) is the only thing that keeps it from being fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters and the input schema is empty, so there is no parameter semantics to document. Per the rubric, 0 params establishes a baseline of 4; nothing in the description conflicts with this.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a concrete question and states exactly what is returned: 'the profiles, the evidence floors and the neutrality commitment' from a named JSON resource. It distinguishes itself from the sibling 'decide'. The one gap is that the tool name 'list_use_cases' does not match the described content (ranking profiles/policy), which leaves a small ambiguity about scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit exclusion and an alternative: 'Not a decision: use decide for that', routing the agent away from the decision-making sibling. It also states the prerequisite 'Needs an API key'. It does not address the other siblings (rank, policy_check, model_info), but the key routing decision is covered.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does well: it discloses the remote URL fetched, the API-key requirement, and an important return-value caveat about data splitting reducing output to just the display name. It stops short of describing error or rate-limit behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three compact sentences, front-loaded with the core purpose and followed by the caveat and the alternative. Slight density/jargon in the data-splitting sentence keeps it from a 5.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter read with no output schema, the description covers source, auth, output caveat, and scope boundaries, which is nearly everything an agent needs. Nothing critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 100%, so the baseline is 3, but the description adds real value by explaining that model_id is a provider/slug and showing how it maps onto the request URL (modelspec.dev/api/models/<model_id>.json), which the schema alone does not convey.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource – reading a model's ModelSpec card – and even gives the exact endpoint template with example id format. It also explicitly distinguishes itself from the sibling 'decide' by denying ranking/evidence semantics.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear context: needs an API key, is not current evidence, is not a ranking, and routes ranking needs to 'decide'. It names one alternative but does not address other plausible siblings like rank or compare_models.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full load and does so well: it discloses auth requirements (Authorization: Bearer), cost (5 credits per call, with per-credit price range and a pricing URL), the endpoint and proxy behavior, and the crucial result semantics that 'undetermined is not a pass.' It does not cover pagination behavior or error handling, which is why it stops short of 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the framing question and then layered with the information an agent needs (semantics, cost, alternative, transport). Sentences are dense but each carries a distinct fact; the pricing parenthetical is the only slightly expendable detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description correctly takes on the burden of describing results (pass/fail/undetermined per row) and adds the non-obvious rule that undetermined is not a pass. It remains silent on pagination for limit/offset and on the required fields of the policy object, leaving modest gaps for a 7-parameter tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% across 7 parameters, so the description must compensate. It partly does: it explains the shape of `policy` (licence, origin, data handling) and ties the pass/fail/undetermined vocabulary to the `verdicts` parameter, but `limit`, `offset`, `models`/`platforms` usage, and `require_no_undetermined` receive no explanation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a precise framing question ('Which models, on which platforms, does my policy allow?') and then states the verb+resource+scope: it checks a policy document per model and per platform. It explicitly distinguishes itself from the sibling `decide` ('It doesn't choose a model: use decide'), so an agent can route without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Names the alternative tool (`decide`) and the condition that selects it (choose among passing models), which is strong routing guidance. It does not state an explicit when-not use case beyond model selection or any prerequisites for invoking it, so it falls just short of the full when/when-not/alternative triad.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and delivers: it is deprecated, requires an API key (Authorization: Bearer), costs 1 credit per call with credit pricing, and returns shortlist data that is 'not a quality verdict.' It also specifies handling of return content (present summary_for_user unchanged, keep must_mention, offer next_move), which is rich behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The front-loaded 'Legacy v1; use decide. Deprecated:' immediately routes the agent, and each subsequent sentence adds operational value (cost, auth, output handling, endpoint). It is dense and somewhat run-on, but little is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a deprecated tool with no output schema and nested objects, the description covers behavior well: deprecation, auth, cost, endpoint, and return-field handling. The main gap is the semantic meaning of the constraints and environment parameters, which is left unaddressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 25% (only use_case is documented in the schema), so the description should compensate but largely does not. It alludes to a 'use-case profile' but gives no meaning for limit, the constraints object (open_weights, include_rehosts, price_sensitivity, max_cost_per_million_input_tokens), or the environment object, leaving half the nested parameters undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: ranking models for 'one fixed use-case profile' and returning a shortlist. It also explicitly contrasts itself with the sibling 'decide' ('use decide', 'Legacy v1'), so an agent can distinguish it from alternatives without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit when-not-to-use guidance: 'Don't call it as a fallback when decide names no single leader: report decide's tie,' and directs the agent to 'use decide' generally. The condition selecting this tool (a fixed use-case ranking) versus the alternative is spelled out.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and mostly succeeds: it discloses auth requirement ('Needs an API key'), default output shape ('compact by default, with 20 rows per page and no counts'), what split-mode detail excludes, and the termination rule ('an empty page ends the list'). It stops short of error/rate-limit behavior and does not fully characterize what a compact row contains.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The strongest fact (what the tool returns) is front-loaded, but the remainder is a run-on sequence of clipped fragments with no grouping by concern (auth, paging, search, starter mode all interleaved). Most sentences earn their place content-wise; the organization does not.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter tool with no output schema and no annotations, the description covers output shape, paging, search ranking, miss behavior, and the starter shortcut, which is close to what an agent needs. Gaps are secondary: exact compact vs. full field sets and failure modes beyond 'nearest ids'.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description earns above that by explaining cross-parameter interaction, e.g. that 'id/ids' details intersect with the search section and that a non-starter section scopes rather than replaces search. It adds paging semantics ('including full details') that the schema only hints at.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The opening line gives a specific verb+resource plus the exact contents: 'Returns the decision vocabulary: valid facet ids, benchmarks, domains, providers and task types.' It also fences itself off from siblings with 'not to compare models,' pointing at compare_models. It loses a point only because the purpose statement is buried under a pile of operational detail.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states the trigger ('use it for missing ids'), the exclusion ('not to compare models'), and the workflow cadence ('one vocabulary lookup per unknown facet, then decide'). It even sequences the follow-up ('Read `next` before another call') and names the condition that selects the starter section.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.