Model Finder

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so the safety profile is known; the description still adds real value by naming excluded inputs (secret-like files, dependency folders), the fact that no line content is returned, and the result ordering. The 'no network calls, no files written' line reinforces rather than contradicts annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the payload contents, then usage routing, then behavioral caveats and pagination – a logical order with almost no filler. It is dense in a single paragraph, but each clause carries distinct information rather than restating the name or title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return shape need not be spelled out, and the description still covers scope, exclusions, ordering and pagination for a six-parameter read tool with all-optional arguments. Nothing an agent needs in order to call this correctly and interpret the results is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so path, limit, offset, hard_coded_only, provider_family and response_format are all documented in the schema; baseline is 3. The description adds only the ordering ('Sorted by usage') and the pagination pattern ('paginate with limit/offset'), which are useful but do not explain any individual parameter beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('List the AI model IDs ... and AI SDKs a repository uses') and enumerates the returned fields: file:line locations, provider family, route hints, usage counts, hard-coded vs env-var origin. That is far more than the title's generic 'Audit AI models' and makes it distinguishable from siblings like estimate_cost or generate_routing.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit sequencing rule ('Use first, to see which models a project depends on before comparing or replacing them') and names a concrete alternative for a different need ('use modelfinder_draft_eval_examples to find test inputs instead'). Both when-to-use and when-to-use-something-else are covered.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, but the description adds cost context ('before spending money on a run') and states 'no network calls, no files written', which sharpens the local/no-side-effect profile. It also names the response fields (passed, failures, checks_run), though that partly duplicates the output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences, front-loaded with the action, then the usage routing, then the side-effect and return summary. No filler or restated title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema, full parameter coverage and rich annotations, the description only needs to establish purpose, routing and safety, all of which it does. Nothing an agent needs to call this correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the schema already enumerates the check types, per-type fields (value, pattern, keys, ignore_case) and the 1-50 item bound. The description's list of check types only echoes the enum, adding no syntax or format detail beyond the schema, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb and resource: 'Run deterministic checks ... on one model reply and report which failed', and enumerates the check types. It also distinguishes itself from modelfinder_rank_eval_results, so an agent can pick it without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

States the use case ('design and debug eval checks or routing guards against a sample reply before spending money on a run') and names the alternative for the other case ('to score a whole eval run use modelfinder_rank_eval_results'). The when-to-use and when-to-use-something-else conditions are both explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so the safety profile is covered; the description still adds real behavior beyond them by disclosing that examples are screened for personal data and secrets and by stating exactly what comes back (config, valid flag, API call count, next step). The 'no network calls, no files written' line largely restates the annotations rather than adding new information.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Five compact sentences, front-loaded with the action and the alternative, then behavior, then return values, then the key-handling caveat. Nothing is padding and there is no repetition of the title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex tool with 7 parameters, nested objects and enums, the description covers what is built, what side effects do not occur, what is returned and what to do next. With an output schema present it does not need to enumerate return fields, and the screening and file-writing behavior are exactly the non-obvious facts an agent needs.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so every parameter is already documented in the schema, including the env-var-only key convention and the checks type list. The description restates the top-level shape (candidates, examples with checks, settings) and reinforces the env-var rule, but adds no syntax, constraints, or examples the schema does not already carry; baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and composite resource: 'Build an eval config from candidate models, examples with checks, and settings,' then adds the validation and personal-data/secret screening steps. It also names the sibling it is not (modelfinder_validate_eval_config), so an agent can separate the two without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states the trigger condition ('Use once examples and checks are agreed') and explicitly routes the alternate case to modelfinder_validate_eval_config for configs the user already has. That is a concrete when/when-not pair with a named alternative, not implied usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only, closed-world, idempotent, non-destructive, so the description is free to add genuinely new behavior: no network calls, no files written, filtering of personal data/secrets with only categories and counts reported, and the important caveat that detection is pattern-based so names and addresses can still leak. It also warns that draft text is untrusted repository content and notes pagination with stable ids.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose and source locations are front-loaded, with usage ordering and safety caveats following in a logical sequence; nearly every sentence carries a distinct fact. It is dense and slightly long, but no sentence is redundant with the schema or annotations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only drafting tool with a full schema and an output schema, the description covers what is produced (empty-check drafts), what is deliberately excluded (personal data/secrets, with the detection-limit caveat), and the next step. Nothing an agent needs to call it correctly or interpret its output is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so path, limit, offset and response_format are already fully documented, which sets the baseline at 3. The description only adds that results are paginated with ids stable across pages, which supplements limit/offset but says nothing about path resolution or response_format beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Draft eval examples from the repository's prompt files and fixtures") and then enumerates the exact source locations (prompts/, evals/, fixtures/, *.prompt.md, JSON/JSONL with input/prompt/chat messages). This lets an agent distinguish it from siblings like modelfinder_create_eval_config without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit ordering rule ("Use after modelfinder_audit_models when the user has no examples yet") and an explicit hand-off (agree checks with the user, then pass them to modelfinder_create_eval_config). Both the trigger condition and the downstream alternative are named.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes beyond the annotations by disclosing the key caveat that prices are not looked up and must be supplied from the provider's pricing page, and by explaining what the pass rate does (cost per accepted result, counting retries). The 'read-only and local: no network calls, no files written' sentence largely restates readOnlyHint/openWorldHint, so it earns credit for the price caveat rather than that line.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the calculation and its inputs, then the use case, then the safety caveat, then the return shape. Dense but every sentence carries information; the only mild redundancy is the read-only/local line that duplicates annotation hints.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a nine-parameter calculator with an output schema, the description covers inputs, the planning use case, the critical external dependency (prices passed in, not fetched), and what is returned. Nothing an agent needs to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 100%, so the baseline is 3; the description adds real meaning on top by defining what pass_rate actually computes (cost per accepted result, with failures retried) and framing cached-input share as an optional modifier. It does not spell out response_format's effect beyond what the schema enum already says.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Estimate monthly token cost') and enumerates the exact inputs that drive it (requests per month, average tokens per request, USD-per-million prices). It is clearly distinguishable from the sibling modelfinder_estimate_memory, which covers a different resource.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Use for planning or to compare models before running an eval' gives a concrete decision context tied to the eval-oriented siblings. It does not name an explicit alternative or a when-not-to-use condition, so it stops short of full routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive and openWorldHint=false, so the description only needs to add beyond that — and it does, calling out 'no network calls, no files written' and the important limitation of 'standard MHA/GQA attention only'. Listing the return fields is somewhat redundant given an output schema exists, so it falls short of the top score.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Four dense sentences that are front-loaded with the core operation and follow with the scoping rule, safety statement and return values. Nothing is filler, though the trailing list of return fields duplicates information already available in the output schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 11-parameter tool with full schema coverage, annotations and an output schema, the description supplies everything left open: the KV-cache parameter coupling, the attention-mechanism limitation, the local/offline behavior, and the shape of the result. An agent can select and invoke this confidently.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline would be 3, but the description adds genuine cross-parameter meaning: layers, kv_heads, head_dim and context must all be supplied together to include the KV cache, otherwise only weights are counted. That dependency is not expressed in the schema and directly affects how an agent should fill in optional parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb and resource — estimating memory for a local open-weight model — and enumerates the components counted (weights at a precision plus KV cache for a context length and concurrency). This is clearly distinct from the sibling modelfinder_estimate_cost, which is about cost rather than hardware memory.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states 'Use when choosing a local model or GPU', giving a concrete decision context. It does not explicitly name or exclude an alternative sibling, but the use case is unambiguous enough that an agent would not confuse it with the eval or cost tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive/closed-world, and the description adds real context on top: no network calls, no files written, nothing deployed, and the user's active model is left untouched. It also discloses a genuine limitation of the generated artifact — the LiteLLM fallbacks fire on API errors only and do not enforce the quality guards or personal-data policy — which is the kind of caveat no annotation field could convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences, each carrying distinct load: the transformation, the prerequisite and trigger, the side-effect profile, and the return shape. The primary output is front-loaded and there is no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a six-parameter tool with nested guard objects and an output schema, the description covers what is produced, what is not touched, and the key semantic gap between the routing file and the proxy config. Return values are summarized rather than duplicated from the output schema, which is the right division of labor.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so every parameter (guards, pii_policy, response_format, include_unknown_cost, chain) is already documented in the schema. The description adds only conceptual framing — that ordering is cheapest-first and that guards/policy exist — without per-parameter syntax or interaction detail. Baseline 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific transformation with named artifacts: a finished eval run (results.json) becomes a cheapest-first routing.json plus a LiteLLM proxy config. It also positions itself against the sibling modelfinder_rank_eval_results, so an agent can distinguish it without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit trigger: 'Use after modelfinder_rank_eval_results when the user wants a fallback chain.' That is clear sequencing plus a selection condition. It does not name what to do instead when the user wants something other than a fallback chain, so no explicit exclusion is present.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, closed-world and non-destructive, and the description partly restates that ('no network calls, no files written'). It earns credit for behavior the annotations cannot express: the exact outputs returned (recommendation, ranked list, cost/reliability cascade, dropped models) and a precise definition of the domination rule, which an agent cannot infer from any structured field.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded in the first clause, followed by routing guidance and then output semantics; every sentence carries distinct information. The parenthetical defining domination is long but it replaces ambiguity with an exact rule, so the density is justified rather than padded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers inputs, the decision logic being applied, the returned artifacts, side-effect profile and sibling routing. With an output schema present, the description need not detail return shapes, yet it still names the key components, so an agent has everything needed to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and both parameters are documented there, including the enum for response_format. The description adds only the provenance of the results object ('written by eval_models.py'), which the schema already states, so this sits at the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource: 'Rank the models in a finished eval run' and identifies the exact input artifact (results.json from eval_models.py). It also names two sibling tools and the distinct job each does, so an agent can separate this from modelfinder_check_output and modelfinder_generate_routing without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit timing ('Use after an eval run') plus two named alternatives with the condition that selects each: single reply -> check_output, fallback chain -> generate_routing. This is the when/when-not/alternatives pattern at full strength.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover readOnly/idempotent/non-destructive/closed-world, but the description adds materially new behavior: no network calls, no files written, and that matches are never echoed back — only categories, counts and a clean flag. That data-handling guarantee is exactly what an agent needs before passing sensitive text through a scanning tool and is not derivable from the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the detection scope, then usage, then safety/return semantics. No filler or restatement of the title, and the parenthetical qualifiers are load-bearing rather than decorative.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter, non-destructive scanner with an output schema and full parameter documentation, the description covers everything an agent needs: what is detected, when to call it, what side effects do not occur, and what shape of answer to expect. Nothing material is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both `text` (size limit) and `response_format` (enum meanings) fully documented in the schema. The description adds no syntax or format guidance beyond that, so the baseline 3 applies; nothing is lost, but nothing is added either.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ("Detect") plus the exact resource classes: Aadhaar, PAN, payment cards, emails, UPI IDs, Indian mobiles, API/private keys. Validation methods (Verhoeff, Luhn) further pin down what counts as a match, so an agent can distinguish this from check_output or validate_eval_config without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Use before sending prompts or eval examples to a provider" gives an explicit trigger, and "modelfinder_create_eval_config already runs this on its examples" names the sibling that duplicates the behavior, telling the agent when this call is redundant. Both when-to-use and when-not-to-use are present.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description still adds real value by stating 'no network calls, no files written' and by enumerating the specific validation checks and the guard behavior, which the annotations cannot express. It loses a point only because the return-value sentence overlaps with the existing output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Single front-loaded paragraph that leads with the action and follows with the sibling routing, then behavior. It is dense but not padded; the only weak element is the trailing return-value list, which partly restates the output schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter, read-only validation tool with an output schema, the description covers purpose, routing, side-effect profile, and the categories of problems it surfaces, including the personal-data guard and unpriced models. An agent has everything needed to decide to call it and to interpret the outcome.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and both parameters are documented in the schema, so the baseline is 3. The description adds only that the config is the one 'the user wrote or edited' and says nothing about response_format or the expected config shape beyond what the schema already states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Check') and resource ('eval config') and enumerates the actual validations performed: schema, key leakage, HTTPS endpoints, template placeholders, and checks. It explicitly names the sibling it is not (modelfinder_create_eval_config), so the agent can distinguish it without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit when-to-use ('on a config the user wrote or edited') and a when-not with the alternative ('to build a new one use modelfinder_create_eval_config'). The 'before a paid run' framing also tells the agent the sequencing context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.