Model Finder

Overview Schema Source Related Servers Discussions Admin

Draft eval examples from the repo

modelfinder_draft_eval_examples

Read-onlyIdempotent

Draft eval examples from prompts and fixtures, remove personal data or secrets, and report safe categories and counts for review before creating an eval config.

Instructions

Draft eval examples from prompt files and fixtures (prompts/, evals/, fixtures/, *.prompt.md, and JSON/JSONL with input, prompt or chat messages). Drops any candidate with detected personal data or secrets and reports only categories and counts. Drafts have empty checks: agree checks with the user before modelfinder_create_eval_config. Paginated; ids are stable across pages.

Input Schema

TableJSON Schema

Name	Required	Description	Default
`path`	No	Directory inside the workspace, relative to the project root (e.g. "services/api"). Defaults to the project root. If a call fails because the project is unknown, pass the absolute path of the project (your current working directory).
`limit`	No	Items per page (1-200, default 30).
`offset`	No	Items to skip; use next_offset from the previous page.
`response_format`	No	markdown (default) for a readable summary, json for the full machine-readable result.	markdown

Output Schema

TableJSON Schema

Name	Required	Description	Default
`drafts`	Yes
`excluded`	Yes
`pagination`	Yes

Schema Changelog

Changes observed during successful MCP inspections.

First observedv1.3.5Oct 18, 2026

TDQS

A4.4/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations cover the read-only/idempotent/non-destructive profile, and the description adds substantial behavior beyond them: candidates with personal data or secrets are dropped, only categories and counts are reported, drafts have empty checks, and pagination yields stable ids across pages. This is rich, non-obvious disclosure an agent could not infer.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded, followed by data-handling behavior, the downstream-tool caveat, and pagination notes. Four dense clauses all carry information, though the parenthetical source list is somewhat heavy for a single sentence.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers source scope, sensitive-data handling, output shape (categories and counts), pagination stability, and the required user-agreement step before downstream creation. With an output schema present, return values need no explanation, so nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so path, limit, offset, and response_format are already fully documented. The description's 'Paginated; ids are stable across pages' lightly reinforces the paging parameters but adds no syntax or format detail beyond the schema, making baseline 3 appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb 'Draft' plus resource 'eval examples,' scoped to concrete sources (prompt files, fixtures, *.prompt.md, JSON/JSONL with input/prompt/chat messages). It is clearly distinguishable from siblings like modelfinder_create_eval_config, which it explicitly names as the downstream step.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description routes the agent forward: drafts have empty checks, and the user must agree on checks before calling modelfinder_create_eval_config. That establishes workflow context, though it does not state explicit when-not-to-use conditions or compare against alternative drafting paths.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server