PoisonDash MCP Server
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@PoisonDash MCP ServerCompare my finished experiments and export a sanitized dataset."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
PoisonDash MCP Server
A Model Context Protocol server that lets an LLM agent (Claude, or any MCP client) operate PoisonDash, a research platform for detecting data-poisoning and backdoor attacks in image-classification training sets.
Through 28 typed tools, an agent can generate poisoned datasets with published attacks (Narcissus, Label-Consistent, Sleeper Agent, Hidden Trigger Backdoor), configure and launch the two-phase detection pipeline, follow training logs, compare runs, inspect flagged samples as images, and export a sanitized dataset with the poisoned samples removed.

PoisonDash itself (the FastAPI backend, ML pipeline, and web dashboard) is private research-lab software. This repository holds only the MCP server, which I designed and built, plus a mock backend so you can run the whole thing without access to the lab's infrastructure.
flowchart LR
A["MCP client<br/>(Claude Code, Claude Desktop, ...)"] -- "JSON-RPC over stdio" --> B["poisondash-mcp<br/>(this repo)"]
B -- "REST + bearer JWT<br/>/api/*" --> C{{"PoisonDash backend<br/>(private)"}}
B -. "same contract" .-> D["mock-backend/<br/>(this repo)"]The server never touches a database or runs ML code itself. It is a thin, careful client of the backend's REST API, and the backend keeps authorization, validation, and the GPU work.
Try it in two minutes (mock backend)
Requires Node 20 or newer.
npm ci
npm run build
npm run mock # terminal 1: in-memory PoisonDash API on http://127.0.0.1:8321Then register the server with your MCP client. For example, put this in a project-local
.mcp.json for Claude Code, or in claude_desktop_config.json:
{
"mcpServers": {
"poisondash": {
"command": "node",
"args": ["/absolute/path/to/poisondash-mcp/dist/index.js"],
"env": {
"POISONDASH_API_URL": "http://127.0.0.1:8321",
"POISONDASH_EMAIL": "admin@demo.local",
"POISONDASH_PASSWORD": "demo-admin",
"POISONDASH_ADMIN": "true"
}
}
}
}Ask the agent something like "Which experiments have finished? Compare them, show me the highest-scoring sample from the latest one, and export a sanitized copy of its dataset."
To see the tools work without a client, run the scripted walkthrough:
npm run test:walkthrough # terminal 2: drives the server over stdio and prints each tool's output== results ==
| Experiment #100 (status: completed)
| Scored suspect samples: 500 (clean reference set: 400)
| Predicted poisoned (score >= threshold): 26 / 500 (detection rate 5.2%)
| Avg poisoning score: 0.159
| Score distribution (0->1, 10 bins): [203, 175, 85, 11, 1, 0, 8, 10, 4, 3]
| Threshold method: kmeans | Threshold: 0.4382
| Important features: 14 identified
| Experiment #100: 500 suspect samples scored
| Mean: 0.1589 Median: 0.1283 Std: 0.1600
| >= 0.4382 (stored threshold): 26 (5.2%)
== inspect and export ==
ok - get_sample_image returned a PNG
| Saved sanitized dataset to .../experiment_100_sanitized.npz (1460370 bytes)
| Original samples: 500
| Removed (score >= threshold): 26
| Kept: 474The mock seeds three demo accounts (admin, registered, and guest; it prints them on startup), poisoned and clean datasets, a model checkpoint, completed and failed experiments, and a pending upload for the moderation queue. Generation and detection jobs run on timers, stream log lines, record provenance, and can be cancelled. The numbers are synthetic but consistent with each other: every summary is computed from the per-sample scores.
Related MCP server: TrustHarness
Engineering highlights
These are the parts I'd point a reviewer at:
Credentials never pass through the model. There is deliberately no
authenticatetool. Tool arguments reach model context, conversation history, and client telemetry, so credentials come only from the server's environment. A preset token and a login credential are mutually exclusive, so an expiring token can never silently hand the session to a different account (src/auth.ts).Lazy, deduplicated auth with silent recovery.
initializeandtools/listnever wait on the backend. The first tool call logs in if the startup warm-up hasn't finished, a burst of concurrent calls shares one in-flight login, and a 401 triggers exactly one re-login and retry, with no loops.Timeouts that know what they are guarding. JSON responses are timed through the body read, so a peer that sends headers and then stalls can't hang a call. The sanitized-dataset download is timed only until its headers arrive, because a multi-gigabyte export is legitimately slow (src/client.ts).
A file-writing tool that can't be steered outside its sandbox.
get_sanitized_datasetstreams to disk without buffering. It resolves the destination against a configured output directory usingrealpath, so symlinks and Windows junctions are handled too. It reserves the destination withO_EXCL, writes to a randomizedO_EXCLtemp file (which closes the symlink write-through hole), and renames the file into place. An interrupted download never replaces a good file (src/tools/analysis.ts).Honest tool metadata. Every tool publishes MCP annotations. The protocol defaults (
readOnlyHint: false,destructiveHint: true) would otherwise advertise a plain listing as a destructive write, and clients would gate it behind a confirmation prompt.Bounded fan-out. Tools that look up related rows, such as uploader names or multi-experiment comparisons, go through a small concurrency limiter, so
limit=500doesn't open 500 sockets against a single-process backend (src/concurrency.ts).Defensive parsing of a loosely-typed API. Several backend fields are JSON-in-a-string, and failed or cancelled runs persist only a stub. Every parse tolerates missing, partial, or stub shapes, and the output omits values the backend is known to report wrongly rather than showing a misleading zero (see the caveats below).
Tested at three levels. There are 146 unit tests with a mocked
fetch, and they run portably on Windows and POSIX without shell globs. A smoke test drives the real stdio server through every tool, its error paths, and teardown. A walkthrough takes a run to completion. CI runs all three on Ubuntu and Windows with Node 20 and 22.
Tool inventory
Area | Tools |
Dashboard |
|
Datasets |
|
Attacks |
|
Generation |
|
Configuration |
|
Experiments |
|
Analysis |
|
Admin (gated) |
|
Configuration (environment variables)
Variable | Purpose | Default |
| Base URL of the backend. A path prefix (e.g. |
|
| Login credential, also used for silent 401 recovery | — |
| Pre-set JWT. Mutually exclusive with the credentials above | — |
| Cosmetic visibility gate for the admin tools in |
|
| The only directory | user's Downloads folder, else a temp dir |
| Per-request timeout. Covers the JSON body as well as the headers. The streamed |
|
Read tools need an account with role guest or higher. Mutating tools
(create, run, generate, configure, cancel) need registered or higher, and admin tools
need admin, which the backend enforces.
If both POISONDASH_TOKEN and the email/password pair are set, the token wins and the
credentials are ignored with a warning on stderr. If neither is set, the server still
starts and tools/list still works. Auth-requiring tools then return a clear error until
the server is restarted with credentials.
Keep the client entry that carries these values in a local, git-ignored settings file, never in a shared, version-controlled one.
While developing, run npm run dev (via tsx, straight from src/), or point the client
at npx tsx /path/to/src/index.ts so you don't need a rebuild after every change.
Testing
npm test # 146 unit tests, mocked fetch, no backend needed
npm run build # type-check and compile
npm run mock # then, in another terminal:
npm run test:smoke # every tool, error paths, and teardown against a live backend
npm run test:walkthrough # one detection run end to endBoth integration scripts accept --dist to test the compiled output. They read
SMOKE_API_URL, SMOKE_EMAIL, and SMOKE_PASSWORD, which default to the mock's admin, and
SMOKE_VERBOSE=1 echoes the server's stderr. They provision everything they need through
the API and delete exactly what they created. Against the real backend, point them at a dev
instance, never at a database you care about.
Mock backend knobs: MOCK_PORT (8321), MOCK_JOB_MS (simulated run length, 8000), and
MOCK_TOKEN_TTL_S (3600). Set the TTL to a few seconds to watch the server re-authenticate
silently mid-session.
Backend caveats this server works around
results_json,config_json, and provenancedata_json/important_featuresare opaque strings. Every parse goes throughsafeJsonParseand tolerates missing, partial, or stub shapes. Failed or cancelled runs may carry onlysummary.statusplusmetadata.log_tail.metadata.batch_level_epochsandmetadata.sample_level_epochsin experiment results are always 0, because the config snapshot taken at experiment creation doesn't populate them.get_experiment_resultsomits them rather than surfacing a misleading zero. Useget_configurationfor the real epoch counts./api/dashboard/summaryreports storage installation-wide, so the dashboard tool also calls/api/datasets/storage/infoand reports per-user and installation-wide storage separately.list_datasets,list_configurations, andlist_experimentsare always scoped to the caller, even for admins. Those endpoints have no server-side "show everyone's" flag; that is whatget_system_statsandlist_pending_datasetsare for.Attack type isn't present in experiment results or config, so
compare_experimentsomits it rather than doing a fragile name-matched dataset lookup.
Project layout
src/
index.ts stdio server: tools/list + tools/call dispatch
auth.ts, client.ts token lifecycle, typed fetch client, error shaping
config.ts environment parsing
concurrency.ts bounded mapLimit
tools/ one module per area; registry.ts holds the registration + visibility gate
types.ts mirrors of the backend's response schemas
mock-backend/ zero-dependency in-memory stand-in for the private backend
test/unit/ node:test suites (run via test/run-unit.mjs)
test/integration/ stdio smoke test and walkthroughCredits
PoisonDash, including the detection pipeline, backend, and web app, is developed by a UofM-DSP Lab. This MCP server was my contribution to the project. It is published here, and the attack implementations named above belong to their respective published papers.
License
Available Tools
24 toolsanalyze_score_distributionARead-onlyIdempotent
Compute summary statistics over an experiment's per-sample poisoning scores: mean, median, std, min, max, quartiles, and counts above fixed cutoffs (0.3, 0.5, 0.7) plus a chosen threshold. per_sample_scores covers only the suspect set with no clean/suspect ground-truth label, so these statistics describe the suspect score population only -- a clean-vs-suspect separation isn't computable from this data. Useful for tuning the detection threshold.
| Name | Required | Description | Default |
|---|---|---|---|
| threshold | No | Threshold to report the above/below split for; defaults to the experiment's stored threshold | |
| experiment_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, so safety is covered; the description goes further by disclosing the data scope limitation — per_sample_scores covers only the suspect set, carries no ground-truth labels, so separation metrics cannot be derived. This is exactly the kind of caveat an agent needs before interpreting results, though it stops short of noting edge cases like an empty suspect set.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences: what is computed, what the data does not support, and what it is for. The statistic enumeration is dense but front-loaded and every clause carries decision-relevant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description correctly takes on the burden of enumerating return values, and it adds a critical interpretation caveat. It is nearly complete for a two-parameter read tool; only empty-data behavior and the exact shape of the cutoff counts are left implicit.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 50%: threshold is documented in-schema as defaulting to the experiment's stored value, and the description reinforces its role ('a chosen threshold', the above/below split). experiment_id has no description in either place, leaving the description to compensate only partially for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb ('Compute summary statistics') over a specific resource ('an experiment's per-sample poisoning scores') and enumerates exactly which statistics are produced (mean, median, std, min, max, quartiles, counts above 0.3/0.5/0.7 plus a chosen threshold). That level of enumeration makes it clearly distinct from raw-result siblings like get_experiment_results or compare_experiments.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It states a use case ('Useful for tuning the detection threshold') and a strong usage constraint ('a clean-vs-suspect separation isn't computable from this data'), but it never names an alternative sibling or says when to prefer get_experiment_results / compare_experiments instead. Guidance is implied rather than explicit routing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
cancel_experimentADestructiveIdempotent
Cancel a running or pending experiment. If a background thread is actively running, this only requests cancellation -- status reads 'cancelling' until it finishes unwinding.
| Name | Required | Description | Default |
|---|---|---|---|
| experiment_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations, the description clearly discloses important asynchronous behavior: cancellation may only be requested while a background thread is running, and status remains 'cancelling' until unwinding finishes. It does not contradict the destructive or idempotent hints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two efficient sentences with no wasted text. The core action is front-loaded, followed immediately by the important asynchronous caveat.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter cancellation tool with annotations covering safety and idempotency, the description provides the key behavioral nuance an agent needs. It does not explain return values or permission requirements, but no output schema exists and those gaps are relatively minor here.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has zero description coverage, and the description does not mention the sole parameter or its expected format. However, with only one required integer named experiment_id, the parameter meaning is largely self-evident from the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: cancel an experiment. It also narrows the eligible states to 'running or pending,' which lets an agent distinguish this from related experiment tools such as get_experiment_status or run_experiment.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when cancellation applies by saying 'running or pending experiment,' but it does not explicitly state when not to use it or name alternatives. An agent must infer that already-completed experiments are out of scope.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
cancel_generationADestructiveIdempotent
Cancel an in-progress (or pending) dataset generation job. If a background thread is actively running, this only requests cancellation -- the status will read 'cancelling' until it finishes unwinding.
| Name | Required | Description | Default |
|---|---|---|---|
| dataset_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
This description provides excellent behavioral context by explaining that if a thread is running, the tool only 'requests cancellation.' It crucially informs the agent that the status will be 'cancelling' until the job finishes unwinding, which is vital information beyond what annotations provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The two-sentence structure is highly efficient, front-loading the primary purpose and immediately following up with the critical caveat about background threads. There is no superfluous language.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that this is a destructive action (indicated by the annotation) without an output schema, the detailed explanation of the asynchronous cancellation process makes the definition highly complete for an agent deciding whether or not to invoke the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and the input only requires a single integer ID. The description does not elaborate on the ID format or any constraints beyond its implied use in identifying the job, maintaining the baseline '3' score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the function: to cancel a 'dataset generation job.' It is specific to the generation domain, distinguishing it from other potential cancel functions in the sibling list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly mentions the scope of the action—canceling an 'in-progress (or pending) job.' While it does not name sibling tools to explicitly compare against, the context strongly implies it is the dedicated tool for cancelling generation tasks.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
compare_experimentsARead-onlyIdempotent
Compare multiple experiments side by side: dataset name, threshold method, threshold, detection rate, average score, and runtime. Fetches each experiment's name and results in parallel. Attack type is not available from these endpoints and is omitted (it can only be approximated via a best-effort dataset-name lookup).
| Name | Required | Description | Default |
|---|---|---|---|
| experiment_ids | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already declare readOnlyHint=true and idempotentHint=true, so the safety profile is covered. The description adds useful behavioral context beyond annotations: it fetches each experiment's name and results in parallel, and it explicitly discloses that attack type is unavailable and only approximated via a best-effort lookup.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, front-loaded with the core comparison purpose and returned fields, followed by useful execution and limitation notes. Every sentence contributes information without excess.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description appropriately lists the fields returned: dataset name, threshold method, threshold, detection rate, average score, and runtime. It also explains the attack-type omission and parallel fetch behavior. It could be slightly more complete with ordering or error behavior, but it is adequate for this simple comparison tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single required parameter, experiment_ids, is self-explanatory from its name and array-of-integers schema, but schema description coverage is 0%. The description only says 'multiple experiments' and does not explain where to obtain the IDs, how invalid IDs are handled, or reinforce the min/max constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and resource: 'Compare multiple experiments side by side', and then enumerates the exact comparison fields. This distinguishes it from single-experiment siblings such as get_experiment_results or list_experiments without ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by 'Compare multiple experiments side by side', but the description does not explicitly state when to choose this tool over alternatives such as get_experiment_results or list_experiments. It also does not mention prerequisites or exclusions beyond the attack-type limitation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_configurationA
Create a new pipeline configuration. Any field you omit takes the backend's own default (see get_default_configuration). A runnable configuration needs BOTH dataset_id (the suspect dataset) and clean_dataset_id (the clean reference set) -- running a pipeline without the clean dataset raises FileNotFoundError. dataset_id/clean_dataset_id/model_id must reference datasets or models the caller owns (admins can reference anyone's), and a rejected dataset cannot be used.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | ||
| seed | No | ||
| device | No | ||
| cv_model | No | ||
| lr_gamma | No | ||
| model_id | No | ||
| momentum | No | ||
| n_groups | No | ||
| nesterov | No | ||
| optimizer | No | ||
| scoring_k | No | ||
| batch_size | No | ||
| dataset_id | No | Suspect dataset ID | |
| lr_schedule | No | Comma-separated epoch milestones | |
| num_workers | No | ||
| target_class | No | ||
| total_epochs | No | ||
| weight_decay | No | ||
| learning_rate | No | ||
| phase_i_zscore | No | ||
| clean_dataset_id | No | Clean reference dataset ID | |
| threshold_method | No | ||
| training_setting | No | Free-text training-setting label | |
| batch_level_epochs | No | ||
| model_architecture | No | Free-text architecture label. Overwritten by the linked model's name when model_id is set. | |
| poisoning_threshold | No | ||
| sample_level_epochs | No | ||
| confidence_threshold | No | ||
| batch_level_start_epoch | No | ||
| sample_level_start_epoch | No | ||
| phase_ii_feature_importance | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false, destructiveHint=false, and idempotentHint=false. The description adds meaningful behavioral context beyond annotations: omitted fields inherit backend defaults, both dataset IDs are required for a runnable configuration, missing clean_dataset_id raises FileNotFoundError, ownership restrictions apply, and rejected datasets cannot be used.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core action and then covers the non-obvious constraints efficiently. Every sentence adds operational value: default inheritance, required dataset pairing, ownership checks, and rejected dataset exclusion.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 31-parameter mutation tool with no output schema, the description covers the most important non-obvious requirements: defaults, required dataset pairing, ownership validation, and rejected dataset restrictions. The schema supplies enums and numeric bounds for individual parameters, so the remaining omission of return behavior is minor.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 16% across 31 parameters, so the description must carry significant semantic weight. It does explain the critical dataset_id/clean_dataset_id pairing, model_id ownership, default behavior for omitted fields, and rejected dataset restrictions, but it leaves most hyperparameters and configuration fields unexplained beyond their schema constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Create a new pipeline configuration.' It clearly distinguishes creation from sibling configuration tools such as get_configuration, list_configurations, update_configuration, and get_default_configuration.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explains when to use this tool for creating a new configuration and explicitly points to get_default_configuration for omitted field defaults. It does not explicitly name update_configuration as the alternative for modifying existing configurations, so it falls short of full when/when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
create_experimentA
Create a new experiment bound to an existing configuration (the caller must own the configuration, or be admin). Does not start the run -- call run_experiment with the returned id next.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | ||
| description | No | ||
| configuration_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the mutation profile (readOnlyHint=false, idempotentHint=false, destructiveHint=false), and the description goes beyond that by disclosing the authorization requirement and the non-starting lifecycle behavior. That is exactly the kind of context annotations cannot express.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no filler, with the prerequisite and the sequencing cue front-loaded. Every clause carries information an agent needs.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a non-destructive mutation with no output schema, it covers the critical gaps: permissions, lifecycle sequencing, and the fact that an id is returned for the next call. The remaining hole is parameter-level meaning, which the schema also leaves undocumented.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and there are 3 parameters, so the description must compensate. It only hints at configuration_id's meaning via 'bound to an existing configuration'; 'name' and 'description' get no guidance, and format/uniqueness constraints are unstated.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Create a new experiment') and adds the key constraint that it is bound to an existing configuration. It is clearly distinguishable from siblings like run_experiment, create_configuration, and list_experiments.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states the precondition (caller must own the configuration or be admin) and the follow-up action ('Does not start the run -- call run_experiment with the returned id next'). This is a textbook when-to-use plus named-alternative pairing with run_experiment.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_configurationARead-onlyIdempotent
Get a single saved pipeline configuration by ID, including the batch-level and sample-level epoch counts that experiment results do not carry. This is the source of truth for an experiment's real hyperparameters: take configuration_id from list_experiments and look it up here.
| Name | Required | Description | Default |
|---|---|---|---|
| config_id | Yes | Configuration ID (0 returns the built-in defaults) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, so safety is covered; the description adds genuinely non-structured context by explaining what the payload uniquely contains (epoch counts absent from experiment results) and where the ID must come from. It stops short of describing the full returned shape, which leaves a small gap for a tool with no output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no filler, and the distinguishing capability (epoch counts) is front-loaded before the lookup instructions. Every clause carries either differentiation or an actionable instruction.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter read with no output schema, the description covers the essential unknowns: what makes the returned data special and how to obtain the ID. A brief note on other returned fields (pipeline steps, dataset binding) would make it fully self-sufficient, but nothing an agent needs in order to call it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the schema itself documents config_id, including the 0-means-defaults case, so the baseline applies. The description still earns credit by giving the parameter provenance — the value comes from list_experiments — which is a usage fact the schema does not state.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Names a specific verb and resource ('Get a single saved pipeline configuration by ID') and immediately scopes it with what makes it distinct: the batch-level and sample-level epoch counts that experiment results do not carry. That clause separates it from get_experiment_results, list_configurations, and get_default_configuration without the agent needing to open any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly routes the agent: 'take configuration_id from list_experiments and look it up here,' and establishes the precedence rule that this is the source of truth for real hyperparameters rather than experiment results. It does not mention get_default_configuration as the alternative for built-in defaults, but the schema's config_id=0 note covers that gap.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_dashboard_summaryARead-onlyIdempotent
Get the calling user's dashboard: counts of their datasets/models/experiments, their 5 most recent datasets and experiments, storage usage, and the latest analysis from their most recent completed/running experiment. Counts and recent items are scoped to the logged-in user even for admins; storage is reported both per-user (against the caller's quota) and installation-wide (against the system cap). For system-wide state across all users, use get_system_stats (admin only).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, and the description adds genuinely non-obvious behavior: counts and recent items stay scoped to the caller even for admins, and storage is reported twice (per-user against quota, installation-wide against the system cap). It does not mention cost, caching, or rate limits, but for a zero-argument read this is solid added context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each doing distinct work: first enumerates the payload, second clarifies scoping and the dual storage report, third routes to the alternative. Front-loaded with the return contents and free of filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no parameters and no output schema, the description must carry the return-value burden, and it does so by naming every section of the response including the quota-vs-cap distinction. Nothing an agent needs in order to call or interpret this tool is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes no parameters, so the baseline is 4 and there is nothing for the description to disambiguate. It does not need to explain argument semantics at all.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ("Get the calling user's dashboard") and then enumerates exactly what it contains: counts of datasets/models/experiments, five most recent datasets and experiments, storage usage, and latest analysis from the most recent completed/running experiment. An agent can distinguish it from list_datasets, list_experiments, and get_system_stats without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly routes system-wide needs to get_system_stats and notes it is admin only, which is a clear alternative-plus-condition. It does not, however, say when to prefer the dashboard summary over the sibling list_* tools for per-user enumeration, so one comparison is left implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_datasetARead-onlyIdempotent
Get full detail on a single dataset or model by ID: generation status/progress, attack type, base dataset, poison ratio, and moderation (approval) state. Note: the generation parameters used to create the dataset and its on-disk file path are intentionally not exposed by the API -- no endpoint returns them.
| Name | Required | Description | Default |
|---|---|---|---|
| dataset_id | Yes | The dataset/model ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide safety hints (readOnly/idempotent). The description adds vital behavioral context by disclosing API limitations—which data is intentionally omitted (generation parameters/file path). This prevents an agent from searching for non-existent fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The structure is excellent, front-loading the core functionality and immediate value proposition (what it gets), followed by a crucial cautionary note. It is dense but highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of an output schema, the description is highly complete. By detailing exactly which information is included and which is intentionally excluded, it removes major ambiguities for the AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the schema adequately defines the `dataset_id`. The description merely reinforces that the ID is the key to fetching comprehensive details, which is typical for a single ID lookup function.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description is highly specific, using the verb 'Get' and clearly defining the resource (a single dataset or model). It enumerates the specific types of details returned, making the purpose unambiguous to an agent.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
While it does not explicitly name alternatives, it provides critical constraints on usage: specifically that generation parameters and on-disk file paths are unavailable. This prevents the agent from making an invalid assumption about the tool's capabilities.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_default_configurationARead-onlyIdempotent
Get the built-in default pipeline configuration (id=0, a virtual template, not a saved row). Use this to see PoisonDash's default hyperparameters before creating a real configuration.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, so the description's added value is the key trait that this returns a virtual template with no backing row, meaning no persistence and stable output. It could go slightly further on the return shape, but the essential behavioral distinction is disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences: identity/mechanics first (id=0, virtual template, not a saved row), then the purpose (see defaults before creating a real configuration). No filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no parameters, no output schema, and annotations covering the safety profile, the description supplies what an agent needs to decide to call it and what to expect conceptually (default hyperparameters). A brief note on the response contents would make it fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so there is nothing for the description to document and the baseline of 4 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('get the built-in default pipeline configuration') and disambiguates from the sibling get_configuration/list_configurations by explaining it is id=0, a virtual template rather than a saved row.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says when to use it: to inspect PoisonDash's default hyperparameters before creating a real configuration, which implicitly steers the agent to create_configuration for persistence. It does not spell out a 'when not to use' case, but the contrast with saved configurations is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_experiment_logsARead-onlyIdempotent
Fetch new log lines for a running or finished experiment. Poll with since (the cursor from the previous call's next_cursor) to get only new lines. Pass tail_lines instead to fetch everything from the start and return just the last N lines (a 'tail' view) -- this overrides since. After the in-memory log buffer is evicted (5 minutes after the run finishes) this falls back to the persisted log tail in the experiment's results, if any.
| Name | Required | Description | Default |
|---|---|---|---|
| since | No | Cursor from a previous call's next_cursor | |
| tail_lines | No | Return only the last N lines | |
| experiment_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, and the description goes well beyond that by disclosing the 5-minute in-memory buffer eviction window and the fallback to a persisted log tail in results (if any). That is exactly the kind of lifecycle/retention behavior an agent cannot infer from annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with the core action, then polling mechanics, then the fallback caveat — no filler. The final sentence is dense but each clause (eviction timing, fallback source, conditional 'if any') carries distinct information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 3-parameter read tool with no output schema, the description covers the polling model, cursor semantics, override behavior, and data-retention fallback. It could mention return shape (lines plus next_cursor) more explicitly, but next_cursor is referenced sufficiently for an agent to close the loop.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 67% schema coverage, the description adds real value: it defines `since` as the previous call's next_cursor (creating a polling loop) and explains the `tail_lines`-overrides-`since` precedence, which the schema does not state. `experiment_id` remains undocumented in both schema and description, a minor gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: 'Fetch new log lines for a running or finished experiment,' which immediately scopes what is returned. It does not name a sibling tool it competes with (e.g., get_experiment_results), though the mention of the persisted log tail in results hints at the relationship.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives concrete when-to-use guidance: poll with `since` for incremental lines, or pass `tail_lines` for a full-from-start tail view, including the precedence rule that `tail_lines` overrides `since`. It lacks guidance on choosing this tool over sibling tools like get_experiment_results or get_experiment_status, so it stops short of full routing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_experiment_resultsARead-onlyIdempotent
Get the detection results for a completed experiment: summary counts, average poisoning score, score distribution, threshold, important features, and runtime. Set include_per_sample_scores=true to also include the raw per-sample scores (large -- displayed truncated for very large sets). Failed/cancelled runs may return only a status stub with a log tail.
| Name | Required | Description | Default |
|---|---|---|---|
| experiment_id | Yes | ||
| include_per_sample_scores | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare readOnly and idempotent; the description adds genuinely new behavior: the exact result payload, that per-sample scores are large and displayed truncated for very large sets, and that failed/cancelled runs may return a status stub with a log tail. That is real disclosure an agent cannot get from the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, zero filler, with the payload contents front-loaded, the optional-parameter cost second, and the edge case last. Every clause earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, yet the description lists the returned fields and covers the degraded-response case, which is exactly the compensation required. An agent can call this correctly and interpret the response without further documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must carry parameter meaning. It explains include_per_sample_scores (default off, adds raw scores, large payload, truncated display) and its trade-off, but says nothing about experiment_id's provenance or format, leaving half the parameters implicit.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Get the detection results for a completed experiment') and enumerates exactly what comes back: summary counts, average poisoning score, score distribution, threshold, important features, runtime. The word 'completed' separates it from get_experiment_status and get_experiment_logs without ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The scope 'for a completed experiment' gives clear context for when this tool applies, and the note about failed/cancelled runs returning a status stub prepares the agent for that case. It stops short of naming alternatives (e.g., use get_experiment_status while running), so no explicit exclusions are offered.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_experiment_statusBRead-onlyIdempotent
Poll an experiment's run status and progress percentage.
| Name | Required | Description | Default |
|---|---|---|---|
| experiment_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, so the safety profile is covered externally; the description's 'Poll' wording is consistent with repeated safe reads. It adds the return content (status plus progress percentage) but says nothing about terminal states, error behavior, or whether status can be stale — no contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One short sentence with zero filler, front-loading the action and the resource. Nothing is wasted and nothing is buried.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description does carry the burden of conveying return values, and it does mention status and progress percentage. However, for a polling tool it omits the most decision-relevant details: the set of possible statuses, what progress percentage means (and whether it is null when not running), and how to proceed to results once complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% for the single required parameter experiment_id, so the schema names it but explains nothing. The description does not compensate — it never says what an experiment_id is, where to obtain one (e.g., from list_experiments or create_experiment), or whether it must reference a running experiment.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Poll') and resource ('an experiment's run status and progress percentage'), so the agent knows exactly what it returns. It does not distinguish itself from close siblings such as get_experiment_results or get_experiment_logs, which also read per-experiment state.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The word 'Poll' implies repeated calls while an experiment is running, which gives implicit usage context. There is no explicit when-to-use/when-not guidance, no routing to get_experiment_results, get_experiment_logs, or list_experiments, and no note about polling cadence or when to stop.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_generation_statusBRead-onlyIdempotent
Poll the progress of a dataset generation job started with start_generation.
| Name | Required | Description | Default |
|---|---|---|---|
| dataset_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, so safety is covered. The description adds that this is a polling operation against a previously started job, but says nothing about what progress data comes back, whether terminal states are reached, or rate/interval expectations for polling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, front-loaded with the action and resource, with zero filler. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description should describe what progress information is returned (state enum, percentage, completion flag) so the agent can interpret results. Combined with the undocumented dataset_id, this leaves a real gap despite adequate safety coverage from annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter has 0% schema description coverage, so the description carries the full burden. It implies dataset_id identifies the job, but it is genuinely ambiguous whether it is the dataset identifier, a job identifier returned by start_generation, or the source dataset — a meaningful risk of wrong invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (poll) and resource (dataset generation job) and ties it to the originating sibling, start_generation. That distinguishes it from get_experiment_status and get_dataset, though the description never explicitly contrasts those siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The word "poll" implies repeated invocation after start_generation, which gives implicit usage context. However, it doesn't say when to stop polling, what to do instead (e.g., cancel_generation, or get_dataset once complete), or whether polling is required at all.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_provenance_recordsBRead-onlyIdempotent
Get the pipeline-stage provenance trail for an experiment: batch-level feature importance (phase I), sample-level dataset composition (phase II setup), and scoring thresholds (phase II results). Summarizes the three pipeline stages.
| Name | Required | Description | Default |
|---|---|---|---|
| experiment_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, and the description adds the content breakdown by phase. It does not describe output shape, size, pagination, or access requirements, so the added behavioral context is moderate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the action and then the three phases. The final 'Summarizes the three pipeline stages' is mildly redundant but the description remains efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only tool with one parameter and no output schema, the description covers the three return categories an agent needs to know. It still omits parameter detail and output format, but the core completeness is adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the sole parameter experiment_id has no description in the schema. The description only implies 'for an experiment' and does not specify the identifier's format or meaning, so it fails to compensate for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource ('Get the pipeline-stage provenance trail for an experiment') and enumerates the three returned components, so the tool's purpose is clear. It does not name sibling tools such as get_experiment_results or get_experiment_logs to differentiate its role, so 4 rather than 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It establishes the experiment context but gives no explicit when-to-use or when-not-to-use guidance versus get_experiment_results, get_experiment_logs, or get_dataset. Only implied usage is present.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_sample_imageARead-onlyIdempotent
Fetch a single sample from an experiment's suspect dataset as a PNG image, for visually inspecting a flagged sample.
| Name | Required | Description | Default |
|---|---|---|---|
| sample_index | Yes | ||
| experiment_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already establish that the operation is read-only and idempotent. The description adds meaningful context by disclosing the output format (PNG image) and the scope (a single sample from the suspect dataset), though it does not cover error behavior, image resolution, or access requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. It places the verb and returned resource first, followed by the intended use case.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only image fetch, the description plus annotations cover the basic purpose and output type. However, the lack of parameter semantics and edge-case behavior leaves gaps in fully equipping an agent to invoke it correctly in all cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate for the two undocumented parameters. It implies that experiment_id selects an experiment and sample_index selects a sample, but it does not explicitly define either parameter, explain the zero-based indexing, or clarify that sample_index indexes the suspect dataset.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb, 'Fetch', and clearly identifies the resource: a single sample from an experiment's suspect dataset, returned as a PNG image. It is clear what the tool does, but it does not explicitly distinguish itself from nearby dataset-oriented siblings such as get_dataset or get_sanitized_dataset.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'for visually inspecting a flagged sample' gives useful implied usage context. However, it does not state when to use this tool versus alternatives, nor does it provide exclusions or preconditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_sanitized_datasetADestructive
Download the experiment's suspect dataset with flagged poisoned samples removed, as a .npz file suitable for re-training. Requires the experiment to be completed with per-sample scores present. The saved file always lands inside POISONDASH_OUTPUT_DIR (the user's Downloads folder by default) -- output_path is resolved relative to that directory and any attempt to escape it, including via a symlink, is refused. Won't overwrite an existing file unless overwrite=true.
| Name | Required | Description | Default |
|---|---|---|---|
| overwrite | No | ||
| threshold | No | Override the poisoning-score threshold; defaults to the experiment's stored threshold | |
| output_path | No | Where to save the file, relative to POISONDASH_OUTPUT_DIR (or an absolute path inside it) | |
| experiment_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the annotations: it discloses the output-directory sandbox (POISONDASH_OUTPUT_DIR, Downloads default), that relative paths resolve there, that path escape attempts including symlinks are refused, and the overwrite guard. These are exactly the non-obvious side effects an agent needs before invoking a destructiveHint=true tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each carrying distinct load (what it produces, the precondition, the filesystem safety rules). Purpose is front-loaded and the sandbox constraint follows immediately.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive write tool with no output schema, this covers prerequisites, path safety, and overwrite behavior. Minor omission: no mention of the resulting file's naming, size, or how errors (e.g., failed precondition) are surfaced, though these are largely inferable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 50% schema coverage, the description compensates well: it explains output_path resolution and its containment rule, and clarifies overwrite semantics ('won't overwrite unless overwrite=true'). It doesn't clarify what 'threshold' controls beyond the schema's own text or note experiment_id's role, leaving a small gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Download') and a precisely scoped resource ('the experiment's suspect dataset with flagged poisoned samples removed, as a .npz file'), which distinguishes it cleanly from the sibling get_dataset (the unfiltered dataset). Format and re-training intent are both named.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives an explicit precondition ('Requires the experiment to be completed with per-sample scores present'), which tells the agent when this will fail. It does not name the alternative get_dataset or explain when the raw version is preferable, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_available_attacksARead-onlyIdempotent
List the poisoning attacks PoisonDash can generate (lc, narcissus, sa, htb), with which base datasets each supports and which extra parameters each requires. Optionally filter to attacks that support a specific base_dataset. Parameter descriptions are client-supplied documentation, not part of the API response.
| Name | Required | Description | Default |
|---|---|---|---|
| base_dataset | No | Only list attacks that support this base dataset |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Read-only and idempotent are already covered by annotations, yet the description adds what the response contains (attacks, their dataset support matrix, and required extra parameters) plus the caveat that parameter descriptions are client-supplied documentation rather than API output. That caveat prevents a real misinterpretation; nothing is said about size of the result, but for a static enumeration this is minor.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, the capability and its returned metadata front-loaded, the optional filter second, and the disambiguation caveat last. Every sentence carries information, though the closing caveat reads slightly like metadata bookkeeping rather than tool behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the burden of describing returns and does so: attacks, per-attack dataset support, and per-attack extra parameters. Only the exact shape (list of objects, field names) is left implicit, which is a small gap for a one-parameter discovery tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the enum is fully enumerated, so the baseline is 3. The description still adds meaning by clarifying the filter's semantics — it selects attacks that *support* the dataset, not attacks that produce it — which the terse schema wording leaves ambiguous.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('List the poisoning attacks PoisonDash can generate') and enumerates the concrete values (lc, narcissus, sa, htb). It also declares the two pieces of metadata returned per attack (supported base datasets, required extra parameters), which clearly separates it from dataset/experiment siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Optionally filter to attacks that support a specific base_dataset' states the one conditional usage path and marks it optional. There is no sibling that is close enough to need an explicit exclusion, so no alternative is named, but the discovery-then-configure context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_configurationsBRead-onlyIdempotent
List the caller's own saved pipeline configurations.
| Name | Required | Description | Default |
|---|---|---|---|
| skip | No | ||
| limit | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, so the safety profile is covered. The description contributes the ownership-scoping behavior, which is useful, but says nothing about ordering, whether full config bodies or summaries are returned, or how pagination behaves.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler. However, its brevity is also the source of the missing pagination and return-shape context, so it's efficient but arguably undersized for the job.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description should indicate what a listed configuration contains (summary vs. full body) and how results are truncated. With zero parameter documentation as well, the definition leaves real gaps for an agent that must paginate through results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%: skip and limit have types/defaults/bounds in the schema but no textual explanation, and the description adds nothing about pagination. With 0% coverage the description was expected to compensate and does not.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (List) and resource (saved pipeline configurations) with an ownership scope ('the caller's own'). This clearly distinguishes it from get_configuration (singular fetch) and list_datasets, though it doesn't explicitly name those siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'The caller's own' implies a scope restriction, suggesting it won't return shared or team-wide configurations, but no when-to-use guidance or named alternatives (e.g. get_configuration for a single item) is given. Usage is inferable rather than stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_datasetsARead-onlyIdempotent
List the caller's own datasets and/or models (uploaded or generated). Per-user: only shows items the caller owns, with no server-side status filter -- the owner sees all of their own items regardless of approval state. Use type to filter to just 'dataset' or just 'model'.
| Name | Required | Description | Default |
|---|---|---|---|
| skip | No | ||
| type | No | Filter to datasets or models only | |
| limit | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover read-only/idempotent safety, but the description adds real behavioral context: scoping is per-user, and there is deliberately no server-side status filter, so the owner sees items in every approval state. That is exactly the kind of non-obvious behavior an agent needs before interpreting empty or full results.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, with the scope and ownership constraint front-loaded and the filter tip last. The 'Use type to filter' sentence partially duplicates the schema's own enum description, a minor redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and no annotations about result shape, the description does not say what a returned item looks like or whether pagination metadata accompanies results. The ownership/status nuance is covered well, but the return contract and paging behavior are left implicit.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 33%: only 'type' is documented in the schema, and the description merely restates that enum. The pagination parameters 'skip' and 'limit' (defaults 0/20, max 500) are never explained, so the description does not compensate for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('List the caller's own datasets and/or models') and disambiguates the resource with 'uploaded or generated'. It is clear an agent can distinguish it from get_dataset, though it never names the sibling explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Use type to filter to just dataset or just model' gives concrete invocation guidance, and the per-user/ownership note tells the agent when this listing will and won't surface items. It stops short of naming alternatives such as get_dataset for individual retrieval.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_experimentsARead-onlyIdempotent
List the caller's own experiments. There is no server-side status filter -- fetch and filter by status client-side if needed (pending, running, cancelling, completed, failed, cancelled).
| Name | Required | Description | Default |
|---|---|---|---|
| skip | No | ||
| limit | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover readOnly and idempotent, so the bar is lower, and the description adds a genuinely useful behavioral constraint: there is no server-side status filtering and results must be filtered client-side. It omits result ordering, pagination behavior, and what a page of results contains, keeping it short of a 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences with zero filler; the purpose comes first and the client-side filtering caveat follows immediately with the enumerated status values inline.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter read-only list tool with annotations covering safety and no output schema, the description supplies the one non-obvious fact (no server-side status filter) and the valid status vocabulary. Only minor gaps remain around pagination expectations and result ordering.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description carries the burden, and it does implicitly document the absence of a status parameter while enumerating the valid status values. It says nothing about skip/limit semantics, though the schema exposes defaults, bounds, and types for both, so an agent can still call correctly.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ("List ... experiments") and adds a meaningful scope qualifier ("the caller's own"), so an agent knows this is owner-scoped listing rather than a global search. It does not distinguish itself from siblings like compare_experiments or get_experiment_status, leaving that differentiation to be inferred.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives one concrete usage rule: filter by status client-side because no server-side filter exists. However, it names no alternatives and gives no guidance on when to reach for this versus get_experiment_status, get_experiment_results, or compare_experiments, so usage is only partially implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
run_experimentADestructive
Start (or restart) an experiment's detection pipeline in the background. Fails with 409 if already running, or 400 if the linked configuration/dataset is missing or was rejected by an admin. Poll get_experiment_status or get_experiment_logs afterward -- this call returns as soon as the run is queued.
| Name | Required | Description | Default |
|---|---|---|---|
| experiment_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
While annotations cover the destructive nature, the description adds critical behavioral traits not found in structured data: the execution is in the background, it is asynchronous (returns when queued), and it provides specific HTTP failure codes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The definition is dense but highly efficient; every clause conveys vital operational information regarding execution, failure, and lifecycle management.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is highly complete for an asynchronous, stateful operation, covering exactly what the agent needs to know to manage the experiment run, including failure conditions and how to check status.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema is very simple and fully covered by the schema description (1 parameter), so the description provides minimal additional semantic meaning beyond what is necessary.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly defines the tool's purpose as starting or restarting an experiment's detection pipeline. It uses specific operational language but does not explicitly differentiate itself from similar sibling tools like create_experiment.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides explicit guidance on usage, including identifying specific failure modes (409, 400) and dictating the necessary follow-up actions (polling get_experiment_status or get_experiment_logs).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
start_generationA
Start generating a poisoned dataset with one of the 4 attacks (see list_available_attacks). Runs as a background job -- poll get_generation_status with the returned dataset id. Constraints (enforced server-side): 'lc' requires clean_model_id (an uploaded model checkpoint); 'sa' and 'htb' require source_class AND pre-generated .npy files already present on disk -- generation fails at runtime without them. Each attack only supports specific base_dataset values (see list_available_attacks).
| Name | Required | Description | Default |
|---|---|---|---|
| eps | No | ||
| vis | No | ||
| name | Yes | ||
| seed | No | ||
| device | No | cpu | |
| attack_type | Yes | ||
| description | No | ||
| base_dataset | No | cifar10 | |
| poison_ratio | No | Percent of samples to poison | |
| source_class | No | Required for sa and htb | |
| target_class | No | ||
| clean_model_id | No | Dataset-table ID of an uploaded clean model checkpoint. Required for lc only. | |
| model_architecture | No | resnet18 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare the generic write/non-idempotent/non-destructive profile; the description adds real substance on top: it is a background job, poll with the returned id, and constraints are enforced server-side with named runtime failure modes ('generation fails at runtime without them'). It stops short of describing what artifact is produced or whether a partial/failed job leaves state behind.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with purpose, then polling behavior, then constraints — the right order for an agent deciding whether to call. It is dense but every clause carries information; the nested parentheticals around 'lc'/'sa'/'htb' are the only mildly awkward part.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 13-parameter async mutation tool with no output schema, the description supplies the essential missing pieces: async lifecycle, the id needed for polling, and the attack-specific prerequisites that would otherwise cause silent runtime failures. No output schema exists, so the return value is adequately covered by 'returned dataset id'.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 23% across 13 parameters, so the description must compensate — and it does cover the highest-stakes ones: attack_type prerequisites (lc needs clean_model_id; sa/htb need source_class plus pre-generated .npy files) and base_dataset compatibility. However, eps, vis, seed, device, poison_ratio, target_class, model_architecture, name, and description get no explanation at all, leaving the majority of the surface undocumented in both schema and description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb+resource ('Start generating a poisoned dataset') and immediately scopes it to one of four attack types. It is clearly distinguishable from siblings like get_generation_status, cancel_generation, and run_experiment, and it names the reference tool (list_available_attacks) that enumerates the attack options.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states this is asynchronous and tells the agent to poll get_generation_status with the returned dataset id, and to consult list_available_attacks for attack/base_dataset compatibility. What's missing is a clear 'do not call' condition (e.g., don't call while another generation for the same dataset is running) or a direct comparison against run_experiment, which is a plausible alternative for similar work.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_configurationADestructive
Update fields on an existing configuration. Only the fields you pass are sent to the API and changed -- omitted fields keep their current value (the backend uses exclude_unset semantics). Do not pass a field just to keep it the same. Pass null for dataset_id, clean_dataset_id, or model_id to detach that reference.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | ||
| seed | No | ||
| device | No | ||
| cv_model | No | ||
| lr_gamma | No | ||
| model_id | No | ||
| momentum | No | ||
| n_groups | No | ||
| nesterov | No | ||
| config_id | Yes | ||
| optimizer | No | ||
| scoring_k | No | ||
| batch_size | No | ||
| dataset_id | No | ||
| lr_schedule | No | ||
| num_workers | No | ||
| target_class | No | ||
| total_epochs | No | ||
| weight_decay | No | ||
| learning_rate | No | ||
| phase_i_zscore | No | ||
| clean_dataset_id | No | ||
| threshold_method | No | ||
| training_setting | No | ||
| batch_level_epochs | No | ||
| model_architecture | No | ||
| poisoning_threshold | No | ||
| sample_level_epochs | No | ||
| confidence_threshold | No | ||
| batch_level_start_epoch | No | ||
| sample_level_start_epoch | No | ||
| phase_ii_feature_importance | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds substantial context beyond annotations: exclude_unset partial-update semantics, the fact that omitted fields retain current values, and that null detaches dataset_id, clean_dataset_id, or model_id. Annotations already disclose readOnly=false and destructive=true, so the remaining gap is minor.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences, front-loaded with the core action, followed by the non-obvious partial-update and null-detach rules. Every sentence carries operational value and there is no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the mutation model well and there is no output schema to explain. For a 32-parameter update tool with no parameter descriptions, though, it is incomplete: an agent still lacks semantic guidance for most domain-specific fields.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage across 32 parameters, the description compensates for the three nullable reference fields and the overall partial-update semantics. It still leaves most domain-specific parameters (e.g., phase_i_zscore, batch_level_epochs, poisoning_threshold) undocumented beyond their type and range constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Update fields on an existing configuration.' This clearly distinguishes it from create_configuration and get_configuration by name alone, though it never explicitly contrasts itself with those siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives useful usage rules for partial updates ('Only the fields you pass are sent... Do not pass a field just to keep it the same') and null-detach behavior. However, it never says when to choose this tool over create_configuration or get_configuration, so guidance is implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
24 tool updates
v0.1.0- First observed
analyze_score_distribution - First observed
cancel_experiment - First observed
cancel_generation - First observed
compare_experiments - First observed
create_configuration - First observed
create_experiment - First observed
get_configuration - First observed
get_dashboard_summary - First observed
get_dataset - First observed
get_default_configuration - First observed
get_experiment_logs - First observed
get_experiment_results - First observed
get_experiment_status - First observed
get_generation_status - First observed
get_provenance_records - First observed
get_sample_image - First observed
get_sanitized_dataset - First observed
list_available_attacks - First observed
list_configurations - First observed
list_datasets - First observed
list_experiments - First observed
run_experiment - First observed
start_generation - First observed
update_configuration
TDQS
Scored across 24 tools
The tools are highly distinct, adhering strictly to resource-action separation (e.g., list_datasets vs get_dataset, start_generation vs get_generation_status). Even within workflows, the specialized nature of the PoisonDash operations ensures minimal confusion.
Naming is extremely consistent, utilizing a predictable `action_resource` pattern across all operations. The use of standard verbs like `get`, `list`, `create`, `update`, `start`, and `cancel` maintains high predictability.
With 24 tools, the set is on the heavier side, but this count is justifiable given the significant complexity of the underlying ML domain. Each tool provides a necessary and distinct piece of functionality for the end-to-end data poisoning workflow.
The tool surface is remarkably complete, covering the full lifecycle: data generation, configuration management, experiment initiation, execution tracking (logs/status), result analysis, and data retrieval. There are no obvious dead ends in the core workflow.
Maintenance
Related MCP Connectors
Paid deterministic data-quality and execution-verification tools for AI agents.
Machine-readable utilities and datasets for AI agents.
Context and control for any AI. Agent identity, scoped access, and proof across tools.
AI Agent Utility Layer for agent safety, web monitoring, crypto research and tool discovery.
Related MCP Servers
- AlicenseBqualityDmaintenanceEnables testing AI safety classifier robustness against query decomposition, obfuscation, and multi-agent attacks. Provides tools for full evaluation pipelines, query previews, and status checks.46MIT
- AlicenseBqualityDmaintenanceEnables deterministic security testing of AI agents that use tools by serving synthetic MCP environments with poisoned data, fake secrets, and privileged actions. Records agent tool calls and evaluates security invariants (e.g., canary leaks, forbidden access, approval binding) without an LLM judge or real systems.8MIT
- AlicenseNot gradedqualityAmaintenanceEnables agents to track ML experiments with a reproducible write contract, internal git snapshots, and a queryable knowledge graph, plus a live web UI for humans.Apache 2.0
- AlicenseNot gradedqualityCmaintenanceEnables active deception for AI agents by injecting decoy honeypot tools that detect indirect prompt injection attempts, instantly terminate hostile sessions, and seal cryptographic incident evidence.Apache 2.0