Skip to main content
Glama
io.github.tammy357/ts-second-opinion

io.github.tammy357/ts-second-opinion

by tammy357

evaluate_predictions

Compares transition-state predictions from any generator with reference XYZ and reaction SMILES, returning stereo-aware D-MAE, RMSD, success, and handedness metrics for DFT triage.

Instructions

Stereo-aware evaluation of any TS generator: multi-frame xyz text of predictions and of reference TSs (one frame per reaction, same order) plus the atom-mapped reaction SMILES. Returns D-MAE, RMSD with and without the mirror image allowed, success rate, share with correct handedness, and success with correct handedness.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "predictions_xyz": "<predictions_xyz>",
      "references_xyz": "<references_xyz>",
      "reaction_smiles": "<reaction_smiles>"
    },
    "name": "evaluate_predictions"
  }
}

Input Schema

NameRequiredDescriptionDefault
orderNomap
references_xyzYes
predictions_xyzYes
reaction_smilesYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.3.0

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden, and it does disclose the core computation: stereo-aware comparison and mirror-image-permitted RMSD. It also names the returned metrics, which is valuable without an output schema. It stops short of stating read-only behavior, prerequisites, or error handling.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loads the purpose before listing input requirements and return metrics. The final metric list is slightly dense but each clause contributes useful evaluation detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a specialized scientific evaluator with no annotations and no output schema, the description covers the main inputs and return metrics adequately. It remains incomplete because the order parameter and exact XYZ formatting expectations are not clarified.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It explains predictions_xyz, references_xyz, and reaction_smiles reasonably well, including frame correspondence and atom mapping, but it leaves the order parameter unexplained despite its default value of 'map'.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action and resource: stereo-aware evaluation of TS generator predictions against reference TS structures. It is clearly more than a tautology, though it does not explicitly distinguish itself from sibling validation tools such as benchmark_summary or gate_check.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The required input pattern implies when the tool is applicable: when you have multi-frame prediction and reference XYZ text plus atom-mapped reaction SMILES. However, it gives no explicit when-not conditions or guidance on choosing this over sibling evaluation or benchmarking tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.