Model Finder

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint and non-destructive, so the safety profile is covered. The description adds genuine behavioral context beyond that: it skips secret-like files and dependency folders, never returns line content, and results are sorted by usage and paginated. It stops short of detailing output shape or rate behavior, but the added filtering/redaction context is meaningful.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences that front-load the returned content and then the behavioral constraints. Dense but well-ordered with no filler, though the second sentence bundles several distinct traits into one clause chain.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only audit tool with a full output schema and 100%-covered input schema, the description supplies the remaining needed context: what is scanned, what is excluded for safety, sorting and pagination. An agent has enough to invoke it correctly without opening anything else.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so every parameter is already documented in the schema, including enums, defaults and limits. The description restates output concepts (provider family, route hints) but adds no syntax or semantics beyond what the schema provides, making the baseline 3 appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (List) and resource (AI model IDs and AI SDKs) and enumerates the returned fields: locations, provider family, route hints, usage counts, hard-coded vs env-var. This clearly distinguishes it from sibling tools like modelfinder_scan_sensitive_data or modelfinder_estimate_cost.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description establishes context (an audit of the repo) but never states when to choose this over siblings, nor any prerequisites or exclusions. Usage is implied rather than guided, so it sits at the minimum-viable level.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds useful behavioral context ('deterministic', cheap pre-run testing) but does not disclose how failures are reported or that all checks must pass, which lives only in the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, zero redundancy, with the action front-loaded and the motivation immediately following. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a full output schema covering return values, rich annotations, and 100% schema description coverage, the description supplies what an agent needs to select and call it. Minor gaps: no statement about check limits or failure granularity, though these are implied by the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so types, enum values, defaults and per-field meanings are already fully documented in the schema. The description adds no parameter-level detail beyond implying a single reply, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific action ('run deterministic checks') on a specific resource ('one model reply'), which is concrete enough to distinguish from config-oriented siblings like validate_eval_config. It does not, however, explicitly contrast itself with any named sibling, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a clear use context and a motivating condition: design eval checks and routing guards before spending money on a real run. It omits when-not-to-use and any named alternative, so it is clear guidance without exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds genuinely useful context beyond that: it validates the config and screens examples for personal data and secrets, and it returns (rather than persists) the config, plus the security constraint that keys are referenced by environment variable name only.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short clauses, front-loaded with the core action (assemble), followed by the processing guarantee (validate/screen) and the output/security note. No filler or repetition; each sentence earns a place, though the phrasing is densely packed.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists so return shape needn't be explained, yet the description still says the config is returned for saving as JSON. Together with the validation/screening disclosure and annotation coverage of safety, an agent has what it needs to call this correctly; only explicit sibling routing is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents candidates, examples, checks, settings, thresholds and enums thoroughly. The description restates the top-level triple (candidates, examples, settings) and reinforces the env-var-only convention for api_key_env, which the schema also states, so it adds little parameter meaning beyond the structured data.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a concrete verb and resource: assemble an eval config from candidates, examples with checks, and settings, then validate and screen it. It also names what the output is for (eval_models.py), but it doesn't explicitly distinguish itself from validate_eval_config or draft_eval_examples, and the 'screen for personal data and secrets' step overlaps with the scan_sensitive_data sibling.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied rather than stated: it's the assembly step, and the output is meant to be saved as JSON for eval_models.py. There is no explicit when-to-use/when-not, and no routing to validate_eval_config or draft_eval_examples, even though the description mentions validation as if it were an internal step.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations cover the read-only/idempotent/non-destructive profile, and the description adds substantial behavior beyond them: candidates with personal data or secrets are dropped, only categories and counts are reported, drafts have empty checks, and pagination yields stable ids across pages. This is rich, non-obvious disclosure an agent could not infer.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded, followed by data-handling behavior, the downstream-tool caveat, and pagination notes. Four dense clauses all carry information, though the parenthetical source list is somewhat heavy for a single sentence.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers source scope, sensitive-data handling, output shape (categories and counts), pagination stability, and the required user-agreement step before downstream creation. With an output schema present, return values need no explanation, so nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so path, limit, offset, and response_format are already fully documented. The description's 'Paginated; ids are stable across pages' lightly reinforces the paging parameters but adds no syntax or format detail beyond the schema, making baseline 3 appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb 'Draft' plus resource 'eval examples,' scoped to concrete sources (prompt files, fixtures, *.prompt.md, JSON/JSONL with input/prompt/chat messages). It is clearly distinguishable from siblings like modelfinder_create_eval_config, which it explicitly names as the downstream step.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description routes the agent forward: drafts have empty checks, and the user must agree on checks before calling modelfinder_create_eval_config. That establishes workflow context, though it does not state explicit when-not-to-use conditions or compare against alternative drafting paths.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare a safe, idempotent, read-only, closed-world computation, so the safety bar is met. The description adds real context beyond that: the pass-rate model ('cost per accepted result, counting retries') and the sourcing requirement for prices. It doesn't discuss rounding, currency, or accuracy caveats, which keeps it short of a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Effectively one sentence, front-loaded with the core purpose. It is dense with parentheticals but every clause carries information; there is no filler. Not perfectly scannable, so short of a 5.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return-format explanation is unnecessary, and annotations cover the safety profile. Parameters are fully documented in the schema. The remaining gap is the retry/pass-rate computation mechanics, which are gestured at but not fully specified.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so every parameter is already documented, establishing a baseline of 3. The description restates the cached-input share and pass-rate parameters and clarifies that retries count toward cost, which adds marginal meaning but largely duplicates the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb (estimate) and resource (monthly token cost) and enumerates the driving inputs. It is clearly distinguishable from the sibling modelfinder_estimate_memory by resource (cost vs. memory). An agent knows exactly what this computes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Planning estimate' signals it is for pre-hoc budgeting rather than measured spend, and it states that prices must come from the provider's official pricing page. However, it never names an alternative tool or an explicit when-not condition. Usage is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish the safe, read-only, idempotent profile, so the description's added value is the computational model and its limits: which components are summed, the headroom concept against available memory, and the explicit MHA/GQA-only restriction. That limitation disclosure is real behavioral context not present in the annotations. It does not discuss precision/rounding or how headroom is applied, leaving a small gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences: the first front-loads the computation and scope, the second carries the actionable parameter rule. No filler, and the parenthetical attention restriction is placed exactly where it matters.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a full output schema, rich annotations, and 100% parameter descriptions, the description only needs to explain the model and its scope, which it does. The one remaining ambiguity is behavior when only some KV inputs are supplied (it implies KV cache is omitted, but does not say whether that is an error or a graceful degradation).

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description goes beyond the field-by-field docs by explaining an inter-parameter dependency: the four KV-related fields must all be supplied together to trigger KV-cache inclusion. That relational semantics is not encoded anywhere in the schema and is the kind of guidance an agent needs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (estimate) and resource (memory to run an open-weight model locally), and breaks the computation into its components: weights at a precision plus KV cache for context length and concurrency. It is clearly distinguishable in domain terms from the sibling estimate_cost, though it never names that sibling to sharpen the contrast.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a concrete conditional usage rule: 'Give all four of layers, kv_heads, head_dim and context to include the KV cache,' which tells the agent how to steer the computation. It also bounds applicability with 'standard MHA/GQA attention only.' It stops short of stating when to prefer this over the cost estimator or what to do for unsupported attention variants.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the bar is lower; the description still adds real value by stating the outputs are returned and 'nothing is written or deployed,' which closes the loop on side effects. It does not cover failure modes for invalid guard specs or oversized input.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence that front-loads the core transformation, then names both artifacts and the no-side-effect guarantee. No filler, nothing repeated from the name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return values need not be explained, and the description confirms both artifacts are returned alongside a clear side-effect statement. The only gap is workflow context: when in the eval-to-routing pipeline it should be invoked relative to rank_eval_results or modelfinder_validate_eval_config.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and every parameter, including guards, pii_policy, and response_format enums, is documented in the schema itself. The description only alludes to guards and the personal-data policy, so it adds almost nothing beyond structured data; baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb-and-artifact transformation: results.json becomes a cheapest-first routing.json plus a LiteLLM proxy config. That is clearly distinct from sibling tools like rank_eval_results or create_eval_config, though it never names those alternatives to sharpen the distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The mention of results.json 'written by eval_models.py' implies the tool is used downstream of evaluation, but there is no explicit when-to-use statement, no prerequisites, and no routing to a sibling when the caller only wants ranking or config validation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish readOnly, idempotent, non-destructive, closed-world behavior, lowering the bar. The description usefully discloses that it computes rankings, a cost/reliability cascade, and dropped dominated models, but says nothing about handling of malformed input, missing threshold, or failure modes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense sentence that front-loads the primary output (cheapest passing model) and then appends the secondary artifacts. Efficient, though the clause stacking makes it slightly heavy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and full annotation coverage, the description need not restate return structure, yet it still characterizes the result meaningfully. It is complete enough to call correctly, with only usage-context guidance missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both 'results' and the 'response_format' enum are already fully documented in the schema; the description merely echoes that it reads a results.json object. Baseline 3 is appropriate since the schema carries the parameter meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

States a precise verb ('Read') and resource ('results.json object') and spells out the exact return: the cheapest model meeting the pass threshold, ranked by cost per passing call, plus the cascade and dominated models. An agent can distinguish this ranking/analysis tool from siblings like estimate_cost or generate_routing without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies its place in the workflow (consume a results.json written by eval_models.py), but it never states when to reach for this versus alternatives, nor any prerequisites or exclusions. Usage is only inferred from the input/output description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, non-destructive and closed-world, so the safety profile is covered. The description adds genuinely useful behavior beyond that: the detection algorithms used (Verhoeff for Aadhaar, Luhn for cards) and, importantly, the privacy guarantee that it 'Returns categories and counts only, never the values' – a non-obvious disclosure an agent could not derive from annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, both load-bearing: the first defines scope and rigor of detection, the second defines the return contract. Nothing is repeated from the title or annotations and the most decision-relevant information (what is detected) is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only, single-argument scanner with a full output schema and complete annotation coverage, the description supplies everything an agent needs: target entity types, validation strictness, and the guarantee that raw values are never returned. Input limit and output format are handled by the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both 'text' (with its 1,000,000-character limit) and 'response_format' are fully documented in the schema. The description adds nothing about either parameter, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('Detect') and resource (text) and enumerates exactly what it looks for: Aadhaar, PAN, payment cards, email, UPI IDs, Indian mobile numbers, API/private keys. That level of specificity leaves no ambiguity about what the tool does, and none of the sibling tools (eval config, cost/memory estimation, routing) overlap with it.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the verb 'Detect ... in text' – an agent can infer it is for scanning arbitrary text for sensitive content. However, there is no explicit when-to-use guidance, no stated prerequisites or exclusions, and no named alternative, so it stays at the implied-usage level.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds genuinely new behavioral detail: "Makes no network calls" reinforces the closed-world claim, and it lists what the report contains (API call count, required env var names, unpriced models, personal-data guard outcome).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero filler. The purpose and check list are front-loaded, and the behavioral caveat about network calls is placed last as a supporting note.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need no elaboration, yet the description still summarizes the report contents. Combined with the enumerated checks and the no-network-call guarantee, an agent has everything needed to call this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with only two parameters, so the schema already documents both the config object and the markdown/json response_format. The description adds nothing about parameter syntax or the config's expected shape, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource ("Check an eval config before running") and enumerates the exact checks performed (schema, embedded keys, HTTPS endpoints, template placeholders, valid checks), which cleanly separates it from siblings like modelfinder_create_eval_config or modelfinder_check_output.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase "before running" gives a clear precondition for when to call this tool. It stops short of naming an alternative (e.g. create_eval_config) or stating when not to use it, so it's clear context rather than full routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.