Skip to main content
Glama

⚗️ Alchemist-MCP

A fail-closed MCP verification layer that gates DFT-to-MLIP (MACE / RuNNer) migration-barrier workflows behind scientific, software, and numerical checks — so a broken calculation can never present itself as a valid one.

Python >=3.11 MCP 2.x 227 tests passing engineering-demo-v1

Why Alchemist-MCP?

  • Fail-closed by construction. Every tool returns only PASS | WARN | FAIL | QUARANTINE | NOT_RUN — there is no code path that upgrades incomplete or unverified evidence into an apparent pass.

  • Verification before training. MACE and RuNNer adapters can only ever build a dry-run CommandPlan; execute=True / dry_run=False is refused unconditionally at the MCP boundary.

  • Provenance-first data intake. Quantum ESPRESSO parsing preserves atom order, canonical units, pseudopotential hashes, and paired input/output SHA-256 identity, with fail-closed quarantine on anything incomplete or ambiguous.

  • Honest status reporting. The project distinguishes engineering validation (proven by tests) from real scientific validation (pending real DFT data) everywhere, including in this README.

Related MCP server: colabfit-mcp

Architecture

flowchart TD
    A[Quantum ESPRESSO DFT] --> B[Scientific verification]
    B --> C[Software verification]
    C --> D[Numerical verification]
    D --> E[Frozen dataset]
    E --> F[MACE]
    E --> G[RuNNer 2G]
    F --> H[Common evaluation]
    G --> H
    H --> I[Reference / literature validation]

What is validated today?

Area

Status

Evidence

QE parsing, units, provenance, freeze/split

Engineering — validated

227 automated tests

MACE / RuNNer adapters (dry-run only)

Engineering — validated

227 automated tests

Common evaluator + reference-comparison contract

Engineering — validated

227 automated tests

MCP server (8 tools, official SDK client)

Engineering — validated

Live stdio client run

rMD17 engineering trial (real, non-DFT dataset)

Engineering — validated

reports/engineering_demo_validation.md

Gate 1 — Scientific Correctness (real DFT data)

NOT_RUN

No production migration-barrier DFT dataset yet

Gate 2 — Software Correctness

PASS

227 tests, ruff, mypy clean

Gate 3 — Numerical Correctness (real data)

NOT_RUN

No production HPC data yet

Gate 4 — Reference / Literature Validation

NOT_RUN

Blocked on Gates 1 and 3

Engineering validation proves the software does what it claims on real and synthetic inputs. It is not, and must not be read as, scientific validation of any migration-barrier result.

Validated Engineering Demo

A real, non-synthetic 175 MB rMD17 azobenzene dataset is exercised end to end, purely as a software/engineering trial:

SHA-256 verification
   -> NPZ schema/finiteness validation
   -> deterministic seeded selection (<= 1000 samples)
   -> kcal/mol -> eV unit conversion
   -> non-periodic extended-XYZ export
   -> ASE read-back
   -> dry-run MACE CommandPlan (never executed)

Every output is hard-labeled trial_label = "ENGINEERING_TRIAL_ONLY" and scientific_gate_eligible = false: azobenzene has no defect, no migration path, and no QE provenance, so this trial can never satisfy Gate 1, Gate 3, Gate 4, or training_eligibility. Full commands and live run output: reports/engineering_demo_validation.md.

MCP Tools

Tool

Purpose

Fail-closed behavior

get_pipeline_status

Lifecycle/gate snapshot

Never executes work

inspect_dft_dataset

Read-only QE dataset inspection

Structured quarantine errors per file

verify_dft_dataset

Run Scientific + Software + Numerical checks

Missing evidence -> NOT_RUN/FAIL

freeze_verified_dataset

Immutable, versioned manifest

Missing evidence/duplicates -> FAIL, no file written

run_mace_pipeline

MACE dry-run command plan

execute=True always refused

run_runner2_pipeline

Al-only RuNNer dry-run command plan

Non-Al data or execution request -> FAIL

compare_models

Energy/force/migration-profile comparison

Incomplete evidence -> NOT_RUN, never partial PASS

validate_reference

Compare against DFT CI-NEB + literature

Unverified evidence -> NOT_RUN; mismatch -> FAIL

Quick Start

# clone and enter the repo
git clone <repo-url> alchemist-mcp && cd alchemist-mcp

# install the locked environment (Python >=3.11)
uv sync --extra dev --locked

# run the test suite
uv run pytest -q

# lint and type-check
uv run ruff check .
uv run mypy src

# start the MCP server over stdio
uv run python -m alchemist_mcp.server

Verification philosophy

Every gate resolves to exactly one of PASS | WARN | FAIL | QUARANTINE | NOT_RUN. There is no implicit success: missing, incomplete, or unverifiable evidence always resolves to NOT_RUN or QUARANTINE, never PASS. A critical FAIL blocks the next scientific action outright. Training (MACE/RuNNer) is gated behind Scientific, Software, and Numerical verification of the input data, and final scientific claims additionally require independent reference/literature validation and human review. Fixture and trial-dataset successes verify software contracts only — they never substitute for real scientific, numerical, or literature validation.

Project status

Current milestone: engineering-demo-v1 — the software pipeline, MCP surface, and a real (non-DFT) end-to-end engineering trial are implemented and fully tested. Real scientific Gates 1, 3, and 4 remain pending genuine production DFT evidence; no MACE or RuNNer training has been executed.

Documentation

Current limitations

  • No production migration-barrier DFT dataset exists in this repository; Gates 1, 3, and 4 are NOT_RUN for real data.

  • The rMD17 trial is engineering-only and cannot satisfy any scientific gate.

  • No MACE or RuNNer training has ever been executed, real or trial — only dry-run command construction.

  • The known legacy Al 2x2x2-vs-3x3x3 finite-size delta (0.1105 eV) exceeds the versioned <= 0.03 eV Gate 3 threshold: a documented Gate 3 FAIL, not a passing result.

  • MCP Inspector (interactive dev tool) is unverified non-interactively; the official MCP Python SDK stdio client covers startup, discovery, schema, and invocation instead.

License

No license file is currently included in this repository. All rights reserved by the author unless a license is added.

Available Tools

8 tools
compare_modelsC

Compare model arrays only when ordered common-test record IDs match.

ParametersJSON Schema
NameRequiredDescriptionDefault
record_idsYes
dft_energy_eVYes
mace_energy_eVYes
mace_record_idsYes
runner2_energy_eVYes
migration_path_idsNo
runner2_record_idsYes
mace_migration_path_idsNo
dft_migration_profiles_eVNo
dft_forces_eV_per_angstromNo
mace_migration_profiles_eVNo
runner2_migration_path_idsNo
mace_forces_eV_per_angstromNo
runner2_migration_profiles_eVNo
runner2_forces_eV_per_angstromNo

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

C2.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. It hints at an alignment/validation requirement but never states what happens on mismatched IDs (error vs. silent skip), whether it mutates stored data, or what constraints apply to the optional migration-path and force arrays.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence is not conciseness here but under-specification: for a 15-parameter tool it omits essentially all necessary context. The one clause present is front-loaded, which is the only structural merit.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Output schema exists, so return values need not be described. However, a 15-parameter, 6-required comparison tool with no annotations and no parameter documentation leaves the agent unable to supply correct arguments or know the failure mode.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% across 15 parameters, so the description must compensate and largely does not. The phrase 'ordered common-test record IDs' is the only hint that the id arrays must be positionally aligned, and it explains none of the energy, force, or migration-profile parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The verb 'Compare' and object 'model arrays' give a recognizable action, and the precondition sharpens intent. But it never says which models (dft, mace, runner2, per the 15 params) are being compared or what 'common-test record IDs' are, so an agent cannot distinguish it confidently from the sibling pipeline tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The clause 'only when ordered common-test record IDs match' is a gating condition, but no alternative tool is named and no positive when-to-use context is given. An agent is not told what to do if IDs do not match, or what this replaces among run_mace_pipeline/run_runner2_pipeline.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

freeze_verified_datasetC

Freeze only after Gate 1-3 eligibility is proven.

ParametersJSON Schema
NameRequiredDescriptionDefault
seedNo
dry_runNo
recordsNo
dataset_pathNodata/incoming
gate_evidenceNo

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

C2.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations at all, the description carries the full behavioral burden and mostly fails. Freezing is a state-mutating, likely irreversible operation, and nothing is said about side effects, idempotency, or reversibility; critically, the schema's dry_run default of true means a default call performs no write, and the description never notes this.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One short sentence with no filler and the precondition front-loaded, so it is structurally clean. But the brevity is achieved by omission rather than economy — it is under-specified rather than concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a five-parameter mutation tool with zero annotations and zero parameter documentation, the description is far too thin. The presence of an output schema excuses it from explaining return values, but the dry_run/seed/records semantics, the gate evidence format required, and the consequences of freezing all remain undocumented.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% across five parameters (seed, dry_run, records, dataset_path, gate_evidence), and the description explains none of them. The only faint link is 'Gate 1-3' versus the schema's gate 1-4 domain in gate_evidence, which is too oblique to substitute for documentation of dry_run, seed, or records.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose2/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description is a precondition clause rather than a statement of purpose: it says when to freeze, but never what freezing actually does (writes an immutable artifact? computes hashes? persists to dataset_path?). An agent must infer the operation from the name alone, and the sibling set (verify_dft_dataset, validate_reference) makes 'freeze vs verify' ambiguous without reading the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives one explicit prerequisite ('only after Gate 1-3 eligibility is proven'), which is real usage guidance for gating this call. However, it never routes to the sibling that produces that evidence (verify_dft_dataset / validate_reference) and gives no when-not guidance, so the agent must guess how to establish eligibility.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_pipeline_statusA

Return lifecycle and gate status without expensive execution.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. It does disclose a meaningful trait — that the call is cheap/non-executing — which is useful for a status check. However, it says nothing about permissions, whether it is purely read-only, or the freshness of the status, so the safety profile remains inferred.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler; the purpose verb and the key differentiator (no expensive execution) both land immediately.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be explained, and there are no parameters to document. The remaining gap is scope: siblings run specific pipelines (mace, runner2) while this tool takes no argument, and the description never clarifies which pipeline's status is returned or what 'gate' states mean.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters and schema description coverage is 100%, so there is no parameter meaning for the description to add. Baseline 4 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Return) and resource (lifecycle and gate status), so an agent knows this is a status read. The phrase 'without expensive execution' begins to separate it from the run_* pipeline siblings, though it never names them. 'Lifecycle' and 'gate' are domain jargon left undefined.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: by flagging that it avoids 'expensive execution,' the description suggests using this instead of actually running a pipeline to check state. There is no explicit when-to-use, no when-not-to-use, and no sibling named as the alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

inspect_dft_datasetB

Inspect QE outputs without mutating accepted data.

ParametersJSON Schema
NameRequiredDescriptionDefault
dataset_pathNodata/incoming

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

B3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It does disclose one meaningful trait — that the operation is non-mutating with respect to accepted data — which is a genuine safety signal. However, it says nothing about permissions, whether 'incoming' data is scanned, or how results are shaped, leaving notable gaps for an unannotated tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler. It is efficient, though arguably terse enough that it omits information an agent would need rather than trimming genuine redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be explained, and there is only one optional parameter, which keeps the surface small. Still, for a tool sitting among verify_dft_dataset and freeze_verified_dataset, the description is thin on what inspection yields and how the path argument is interpreted.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for the single parameter dataset_path, and the description never mentions it or its default ('data/incoming'). With the schema silent and the description not compensating, the agent has no textual guidance on what the path should point to.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Inspect QE outputs') and adds a scope qualifier ('without mutating accepted data') that distinguishes it from mutation siblings like freeze_verified_dataset. It does not, however, distinguish it from the closely related verify_dft_dataset or explain what 'inspect' covers versus 'verify'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use guidance, no exclusions, and no routing to alternatives. The phrase 'without mutating accepted data' hints that this is a read-only look at incoming data, but the agent gets no explicit signal about when to choose inspect_dft_dataset over verify_dft_dataset or get_pipeline_status.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_mace_pipelineC

Return a fail-closed MACE dry-run plan; never launches MACE.

ParametersJSON Schema
NameRequiredDescriptionDefault
modeNozero-shot
seedNo
dry_runNo
executeNo
output_dirNoartifacts/mace
dataset_pathNodata/incoming
python_executableNo.venv/bin/python

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

C2.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it does disclose a meaningful trait: fail-closed semantics and that MACE is never launched. However, it does not say what happens if execute=true or dry_run=false, whether any filesystem writes occur, or what auth/environment requirements exist for the python_executable path.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single tight sentence with the key constraint front-loaded and zero filler. It is arguably over-concise rather than wasteful, but nothing in it is expendable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists so return values need not be described, but a 7-parameter mutation-adjacent tool with no annotations and no parameter documentation leaves the agent materially under-informed about invocation and side effects.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Seven parameters with 0% schema description coverage and the description explains none of them. Fields like mode, seed, python_executable, and the dry_run/execute pair are left with no meaning, types, or allowed values beyond their bare names.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb ('Return') and resource ('MACE dry-run plan'), and the phrase 'never launches MACE' distinguishes it from the execution-oriented siblings run_runner2_pipeline and get_pipeline_status. It falls short of 5 because 'MACE' is never defined and the dry-run only framing leaves the role of the execute/dry_run parameters ambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use guidance, no conditions, and no named alternatives despite seven sibling tools covering pipeline status, DFT dataset work, and a second pipeline runner. The agent must infer selection criteria entirely on its own.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_runner2_pipelineC

Return a fail-closed Al-only RuNNer 2G dry-run plan; never launches RuNNer.

ParametersJSON Schema
NameRequiredDescriptionDefault
dry_runNo
executeNo
recordsNo
work_dirNoartifacts/runner2
executableNorunner
input_fileNoinput.nn
dataset_pathNodata/incoming
gate_evidenceNo
frozen_manifestNo

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

C2.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full disclosure burden. It usefully warns that this never launches RuNNer (contradicting the 'run_' in the name) and states a fail-closed posture, but it never clarifies what 'fail-closed' does on FAIL/QUARANTINE, nor how the schema's execute/executable parameters reconcile with the claim that RuNNer is never launched.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The single sentence is front-loaded and free of padding, which is a strength. However, one dense clause is under-sized for a tool with 9 parameters and no annotations, so brevity comes at the cost of appropriateness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The output schema exists, so return values need no explanation, but the description is far short of what a 9-parameter, zero-annotation tool requires. It omits parameter behavior, prerequisites, origin of gate_evidence/frozen_manifest, and execution-safety details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and the description explains none of the 9 parameters. Key inputs like records, gate_evidence, frozen_manifest, work_dir and executable are left entirely to the agent to guess, and the tension between dry_run (default true) and execute (default false) is unaddressed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and artifact: it returns a dry-run plan for RuNNer 2, scoped to Al-only and fail-closed. This lets an agent distinguish it from an actual execution tool, though the 'Al-only'/'2G'/'fail-closed' jargon is unexplained and no sibling (e.g. run_mace_pipeline) is named as contrast.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit when-to-use guidance and no named alternative. The only orientation is the implied contrast that this produces a plan rather than running anything; the agent must infer when to prefer this over get_pipeline_status, run_mace_pipeline, or freeze_verified_dataset.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

validate_referenceC

Validate barriers against supplied DFT/reference evidence without literature claims.

ParametersJSON Schema
NameRequiredDescriptionDefault
literatureNo
dft_provenanceNo
runner2_barrier_eVNo
runner2_provenanceNo
dft_cineb_barrier_eVNo
mace_zero_shot_barrier_eVNo
mace_zero_shot_provenanceNo
mace_fine_tuned_barrier_eVNo
mace_fine_tuned_provenanceNo

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

C2.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations, so the description carries the full behavioral burden, and it discloses almost nothing: not what validation performs, what a failure looks like, whether provenance hashes are checked, or whether evidence is mutated. The one scope remark ("without literature claims") is a faint behavioral hint but is contradicted in spirit by the optional literature input.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no padding, which is structurally good, but its brevity comes from under-specification rather than discipline, and the final clause muddies rather than sharpens the scope.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 9-parameter validation tool with zero annotations, an output schema (returns covered) and 0% schema description coverage, the description is drastically incomplete. An agent cannot infer which inputs are required in practice, how the literature and DFT evidence interact, or what constitutes a valid invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Nine top-level parameters with 0% schema description coverage, and the description adds no parameter meaning whatsoever — it never mentions literature records, dft_provenance, or any of the four barrier values. The nested $defs descriptions live in the schema, not the description, so nothing compensates for the coverage gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a verb+resource ("validate barriers against supplied DFT/reference evidence"), which is more than a tautology, but the trailing clause "without literature claims" is ambiguous and sits awkwardly beside a schema that accepts a `literature` array. No differentiation from siblings like verify_dft_dataset or compare_models.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use guidance, no prerequisites, no named alternatives among the seven sibling tools. The only usage-ish hint is the exclusionary phrase "without literature claims," which is left unexplained and is not framed as a selection condition.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

verify_dft_datasetC

Run available fail-closed verification over incoming QE outputs.

ParametersJSON Schema
NameRequiredDescriptionDefault
dataset_pathNodata/incoming

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

C2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. 'Fail-closed' is a useful hint, but the description doesn't say what is checked, what happens on failure, whether it mutates state, or what output is produced. With an output schema present, some return-value explanation is excused, but the verification behavior itself remains opaque.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single efficient sentence with no filler. It is appropriately sized but too sparse to be useful – concision is achieved partly through omission rather than precision.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is a dataset verification step in a pipeline with several siblings, yet the description omits fail semantics, expected inputs, prerequisites, and where it sits in the pipeline. The output schema may explain results, but the definition is insufficient for correct selection and invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description adds no meaning for dataset_path beyond the schema's own name. It does not state the expected path semantics, whether it accepts directories or file patterns, or what the default 'data/incoming' represents.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose2/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description restates the tool name ('verify') with vague qualifiers ('available fail-closed verification', 'incoming QE outputs') that an agent cannot action on. It does not state what is verified, what counts as pass/fail, or how it differs from siblings like validate_reference or inspect_dft_dataset.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines1/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use, when-not-to-use, or alternative guidance is provided. The sibling tools validate_reference and inspect_dft_dataset are obvious alternatives for similar verification/inspection needs, yet the description gives no routing help.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 8 tool updatesv0.0.1
    • First observedcompare_models
    • First observedfreeze_verified_dataset
    • First observedget_pipeline_status
    • First observedinspect_dft_dataset
    • First observedrun_mace_pipeline
    • First observedrun_runner2_pipeline
    • First observedvalidate_reference
    • First observedverify_dft_dataset

TDQS

B3/5.0

Scored across 8 tools

Disambiguation4/5

Tools target distinct operations (status, inspect, verify, freeze, run dry-runs, compare, validate). However, run_mace_pipeline and run_runner2_pipeline are very similar in nature (dry-run plans for different models), and inspect_dft_dataset and verify_dft_dataset could be confused as both deal with QE outputs. Descriptions clarify the differences, so ambiguity is minimal.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern (get_pipeline_status, inspect_dft_dataset, verify_dft_dataset, freeze_verified_dataset, run_mace_pipeline, run_runner2_pipeline, compare_models, validate_reference). Even multi-word nouns are handled uniformly, making the naming highly predictable.

Tool Count5/5

Eight tools provide a well-scoped surface for a specialized DFT/MACE/RuNNer workflow. Each tool corresponds to a distinct step or check in the pipeline, and no tool appears redundant or excessive.

Completeness3/5

The tools cover inspection, verification, freezing, dry-run planning, comparison, and reference validation, but there are notable gaps: no tool to actually execute a pipeline (only dry-runs), no tool to retrieve or manage datasets beyond inspection, and no tool to handle pipeline failures or updates. These gaps could hinder agents from performing end-to-end workflows.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    B
    quality
    A
    maintenance
    A local MCP reasoning gate that enables structured engineering thought through step-by-step reasoning, branching, merging, validation, and quality metrics without calling external LLM APIs.
    15
    22 npm
    Apache 2.0
  • F
    license
    Not graded
    quality
    C
    maintenance
    An MCP server for discovering ColabFit materials science datasets and training MACE interatomic potentials on local hardware, enabling AI assistants to search, download, train, and validate models.
    2
    -
  • A
    license
    A
    quality
    D
    maintenance
    An interactive MCP server for FAIRChem and ASE simulations that allows LLM agents to load a model once and steer relaxations, MD, NEB, phonons, and minima searches mid-flight, with live monitoring and code introspection.
    17
    1
    MIT