Skip to main content
Glama

court_trial_round

Run evaluation trials across sample artifacts with Bailiff fact-checking to validate outputs and surface factual issues before final judging.

Instructions

Step 3c: Run evaluation trials across sample artifacts with Bailiff fact-checking.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "ours_path": "<ours_path>",
      "manifest_path": "<manifest_path>",
      "law_dir": "<law_dir>",
      "out_dir": "<out_dir>"
    },
    "name": "court_trial_round"
  }
}

Input Schema

NameRequiredDescriptionDefault
modeNo
modelNo
law_dirYes
out_dirYes
providerNo
cache_dirNo
ours_pathYes
budget_usdNo
ledger_pathNo
reuse_b_dirNo
appeals_pathNo
manifest_pathYes
previous_round_dirNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

C2.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are supplied, so the description carries the full behavioral burden, yet it discloses almost nothing: it does not say that the tool writes to out_dir, persists a ledger/cache, or that budget_usd means real spend. For a 13-parameter pipeline step that appears to consume money and produce artifacts, omitting cost and side-effect disclosure is a significant gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single short sentence front-loaded with its pipeline position — efficient with no filler. The economy comes at the cost of substance, though: nothing in it explains the operation beyond a label, so brevity here reflects under-specification rather than discipline.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 4-required/13-total-parameter tool with no annotations and no output schema, an agent needs far more than one clause: required-path semantics, cost implications, side effects, and where the results land are all missing. The step label gives some pipeline context, but the definition is not sufficient to invoke the tool confidently.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% across 13 parameters, including required ones like ours_path, manifest_path, law_dir and out_dir whose expected contents are entirely undefined. The description mentions no parameter at all, so it does not compensate in the slightest for the schema's silence.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and object ('Run evaluation trials across sample artifacts') and adds a qualifier ('with Bailiff fact-checking'), so the general shape of the operation is inferable. However, 'evaluation trials' and 'Bailiff' are domain jargon, and nothing distinguishes it concretely from siblings like court_judge or court_verify_evidence, which sound like they also evaluate or check artifacts.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'Step 3c' prefix is the only usage signal: it implies this runs at a specific point in a multi-step pipeline, after earlier steps and presumably before later ones. There is no explicit statement of prerequisites, when-not-to-use, or which sibling to pick instead, so the guidance is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.