Skip to main content
Glama

hwcontract

Your firmware is correct on paper and wrong on the wire.

Coding agents write WS2812 drivers, ESC bitstreams, and boot logs that pass review and then fail the moment the signal hits a real chip. hwcontract closes that loop. It captures what the hardware actually did and returns a verdict you can act on:

  • PASS: within spec

  • MARGINAL: in spec but too close to a rail. Works on your bench, dies on a cold board in the field. It fails the verdict: the judge will not ship it.

  • FAIL: out of spec, with the measured value and how far off it is

  • MISSING: a required observation is absent; this also fails the verdict

Two things a green verdict gives you beyond the table:

  • Every pulse is judged, not just the median. Captures carry the full pulse distribution; a glitchy tail that a median hides comes back as marginal or fail, with the violating-pulse count in the hint.

  • No verdict finer than the capture can see. A width measured from samples is only known to one sample period. FAIL needs the signal out of spec by more than that, PASS needs it inside by at least that; anything closer is MARGINAL with a hint to capture at a higher samplerate.

  • Evidence on every verdict: contract hash, capture hash, capture parameters, tool version, timestamp. A green build in CI traces back to the exact bytes that produced it.

No hardware in your hand? The demo below runs the whole thing on a real recorded signal, so you can see exactly what you get before wiring anything up.

hwcontract judging a real WS2812B capture, a DMA-broken SPI trace, and a serial boot log

Replay a real capture

pip install hwcontract
python3 -m hwcontract.demo ws2812b

This downloads a recorded 24-LED NeoPixel capture from sigrok (about 109 KiB) on the first run, verifies its SHA256, then caches it in $XDG_CACHE_HOME/hwcontract or ~/.cache/hwcontract. It needs network access once; subsequent runs can use --offline. Use --cache-dir DIR or --capture PATH for a copy of the same pinned capture. No sigrok installation or hardware is needed. Capture provenance and license.

The capture is judged against two contracts. Same signal, two verdicts (output excerpt; each judgment also prints its evidence hashes):

measured on the real WS2812B signal (300000 samples @24MHz):
  T0H 333 ns   T1H 833 ns   T1L 417 ns   T0L 917 ns   RESET 992250 ns

=== generic WS2812 contract -> FAIL ===
  T0H     350   333  PASS
  T0L     800   917  MARGINAL  within one sample (41.7ns) of max; 1842 of 4275
                               pulses within one sample of the window
  T1H     700   833  MARGINAL  within one sample (41.7ns) of max
  T1L     600   417  MARGINAL  33ns past min, within one sample (41.7ns);
                               capture at a higher samplerate to resolve
  RESET 50000 992250 PASS

=== matching WS2812B contract -> PASS ===
  (all five edges PASS)

Same hardware, two contracts: the generic one fails, the chip-specific one passes. The generic rows are MARGINAL rather than FAIL because at 24MHz each sample is 41.7ns, and every miss is smaller than that: the capture shows the signal is too close to the WS2812 limits, not that it is definitely past them. A WS2812B isn't a WS2812. Measure the real signal, hold it to a spec, and match the contract to the actual chip.

Related MCP server: agent-gate

See the temporal engine catch a DMA bug

python3 -m hwcontract.demo spi

100 synthesized SPI frames as raw CS/SCK/MOSI waveforms at 100MHz, reduced to pin edges and judged against the bundled spi-frame contract. Frame 77 has a simulated DMA ordering fault: chip-select asserts after the clock starts. Frame 42 settles MOSI 10ns before the sampling edge. Both come back with exact timestamps. An additional artificial B/E jsontrace roundtrip tests the importer; those annotations are not real sigrok SPI decoder output:

cs-precedes-first-clock  800  1  FAIL  trigger at 1540310ns: no gpio.cs.falling
                                          in [1530310ns, 1540310ns] (first of 1)
mosi-setup               800  1  FAIL  forbidden spi.mosi.* at 843300ns is 10ns
                                          before spi.sck.rising at 843310ns

This is an offline, deterministic pin-timing demonstration. It does not decode SPI payload bytes or run a loopback test. python3 -m hwcontract.demo serial also works offline with synthetic clean and panicking boot logs. Repository commands such as python3 demo/spi_dma_temporal.py remain compatibility wrappers. All demos exit zero when their expected PASS/FAIL outcomes occur; use the judge CLI or MCP verdict for a production check.

What you get

  • 28 bundled contract templates, including WS2812-family LEDs, DShot, servo/I2C/sensor/stepper timing, serial boot logs, and SPI pin ordering. They are a spec library, not 28 ready-to-use capture integrations. Verify the template against your exact part and its datasheet.

  • Working capture paths: WS2812-family pulse-width buckets, DShot600 via check_dshot, and serial logs. DShot150/300/1200 need rate-specific adapter thresholds. Servo, I2C, HC-SR04 and other timing templates need custom observation adapters producing the contract's named measurements.

  • Temporal assertions on normalized events. Raw pin transitions feed spi-frame; decoder annotations need contracts matching the decoder's actual sources and event types.

  • New specifications can be YAML-only when an existing adapter emits the required observations. A new protocol may also require adapter code and known-good/known-bad captures.

  • An MCP server your agent can call, or plain CLI commands you can run by hand.

  • Evidence on every verdict: contract hash, capture hash, capture parameters, tool version, timestamp. A green build traces back to the exact bytes.

  • Reasonable by default: timing edges are all measured in nanoseconds, serial contracts are Python regex, verdicts come back with the measured value and the delta so an agent knows exactly what to fix.

Install

pip install hwcontract              # judge + logic-analyzer adapter
pip install "hwcontract[serial]"    # + live serial capture (pyserial)
pip install "hwcontract[untrusted]" # + google-re2 (ReDoS-immune, for untrusted contracts)
pip install "hwcontract[all]"       # everything

Live logic-analyzer capture (check_ws2812 / check_dshot) also needs sigrok-cli on PATH. Judge-only tools (judge_contract, judge_serial) need nothing extra.

hwcontract --help
hwcontract --version
hwcontract --list-contracts         # names, kinds, capture requirements
hwcontract --list-contracts --json  # also includes absolute installed paths

These commands exit immediately. Bare hwcontract starts the MCP stdio server and waits for JSON-RPC requests on stdin.

Wire it into an agent

One stanza per client, add it once. After install, the hwcontract command is on your PATH.

Claude Code

claude mcp add hwcontract -- hwcontract

Codex CLI: ~/.codex/config.toml

[mcp_servers.hwcontract]
command = "hwcontract"

opencode / Cursor / Gemini / any stdio MCP client

{ "mcpServers": { "hwcontract": { "command": "hwcontract" } } }

Transport is stdio by default (local, no auth surface). For remote-only clients (e.g. ChatGPT connectors), run hwcontract --http 8791 and expose it via a tunnel with HWCONTRACT_TOKEN set for bearer auth.

Speaks MCP 2026-07-28, the stateless revision: per-request _meta, server/discover, no handshake. Clients that still open with initialize get the old shape back. Each request picks its own era, so nothing to configure.

If the client can't find hwcontract (PATH issues)

GUI apps and some agents don't inherit your shell PATH, so a bare hwcontract can fail with "command not found". Two robust fixes:

  • Use the absolute path: which hwcontract → put that full path in command.

  • Or invoke via Python (no PATH lookup for the script): command: "python3", args: ["-m", "hwcontract.server"]. Works from any directory once installed.

Contract paths: MCP tools accept short bundled names such as ws2812b, boot, or spi-frame, or their full filenames (ws2812b.contract.yaml). These names match the pytest plugin and Action; for DShot600 the name is dshot, and the mini LED name is ws2812b_mini. An existing project file wins. Relative paths resolve under HWCONTRACT_ROOT (default: the server's working directory). Arbitrary absolute paths and symlinks must stay within that root. Only exact installed paths returned by list_contracts may refer to bundled contracts outside it; this does not grant access to other package files. The lower-level judge and temporal module CLIs still take file paths.

The tools

Tool

Hardware?

What it does

list_contracts

no

List all 28 templates, kinds, installed paths and capture requirements.

judge_contract

no

Judge given observations against a timing contract. Replay / testing.

judge_serial

no

Judge a given log string against a serial contract's expect/forbid.

judge_events

no

Judge decoded events against temporal assertions (when/require/within, forbid/while/before).

check_ws2812

yes

Capture a live WS2812 line and judge it, one call.

check_dshot

yes

Same, for a DShot600 ESC signal.

capture_ws2812

yes

Just capture → observations (no judging).

check_serial

yes

Read a serial port for N seconds and judge the log.

Event contracts check relationships for every occurrence, with latency distributions and first-failure timestamps. The bundled spi-frame contract requires gpio.cs.falling/rising, spi.sck.rising, and spi.mosi.* raw pin edges. A sigrok SPI Data annotation does not supply those events. Follow the complete multi-channel CSV-to-edge recipe to judge an actual capture. Semantic decoder checks use separate contracts matching the imported decoder rows.

Use the judge matching the contract kind: timing, serial, or events. Wrong-kind calls are errors. Serial logs longer than 1,000,000 characters are rejected, not silently truncated. CSV inputs must contain valid digital samples; malformed rows and empty captures are errors, not evidence of a passing device.

Prefer plain pytest over MCP? pytest-hwcontract is a plugin that turns verdicts into tests: a FAIL, MARGINAL or MISSING edge fails the test with the verdict table in the message, JUnit included.

Gate CI on it

The repo ships a GitHub Action, so captures checked into the repo get judged on every PR:

- uses: MohibShaikh/hwcontract@action-v0
  with:
    timing: "ws2812b=captures/strip.csv"     # contract=capture-glob, bundled names work
    serial: "boot=logs/boot.log"
    samplerate: 24000000                     # for CSV captures (0/1 per line)
    junit: hwcontract-junit.xml              # shows in the tests tab

A FAIL, MARGINAL or MISSING edge fails the step, annotates the failing line, and writes JUnit. The action self-tests on every push to this repo with one clean and one deliberately broken capture.

Replaying an unchanged recorded capture checks the judge and contract against those recorded bytes. It does not test newly built firmware. To gate a firmware change, flash that build, collect a fresh capture/log, and judge it; retain the firmware revision, capture parameters and verdict evidence together.

How it fits together

  observers (capture)                  judge (this repo)
  ─────────────────────                ─────────────────
  logic analyzer  ─ pulse widths ─┐
  serial port     ─ log text ─────┼─►  contract × observation  ─►  PASS/MARGINAL/FAIL/MISSING
  sigrok jsontrace ─ events ──────┘         (judge.py / temporal.py)
  • judge.py. The pure judge for timing and serial, plus contract validation. No hardware, no framework, cached.

  • temporal.py. Cross-event temporal assertions: selectors, signed windows, latency distributions, first-failure timestamps.

  • jsontrace.py. Imports sigrok-cli's Google Trace Event JSON into normalized events.

  • sigrok_adapter.py. Turns a logic-analyzer capture into pulse-width distributions for WS2812 and DShot.

  • serial_adapter.py. Captures a serial log, or replays a saved one.

  • server.py. The MCP server, stdio and HTTP JSON-RPC, stdlib only.

  • *.contract.yaml. What "correct" looks like. Human-editable, and they double as regression tests.

The contract format

Timing (ws2812.contract.yaml, dshot.contract.yaml): pulse widths in ns

contract: ws2812
headroom_pct: 20         # in-spec but within 20% of a rail => "marginal"
edges:
  - {name: T0H, min: 200, typ: 350, max: 500}   # '0' bit high time

Serial (boot.contract.yaml) uses Python regex:

contract: boot
kind: serial
expect: ["IMU init OK", "boot v\\d+"]
forbid: ["panic", "Guru Meditation", "\\bnan\\b"]

Events (spi-frame.contract.yaml) assert relationships between decoded events — SVA-style temporal checks on raw pin edges or sigrok annotations:

contract: spi-frame
kind: events
assertions:
  - {name: cs-precedes-first-clock, when: spi.sck.rising,
     require: gpio.cs.falling, within: [-10us, 0ns]}
  - {name: mosi-setup, when: spi.sck.rising,
     forbid: spi.mosi.*, before: 20ns}

A new contract needs matching observations. Add an adapter when the existing capture paths do not emit them; YAML alone does not add protocol support.

Kill switch

Instantly disable every hardware-touching tool (captures) while leaving the pure judge tools working:

export HWCONTRACT_SAFE=1

Security

Every tool argument is treated as hostile, since the caller is an LLM that can be prompt-injected. Arbitrary contract paths are confined to the server working directory, with HWCONTRACT_ROOT overriding it; exact bundled contracts are allowlisted. driver/channel/port are charset-validated, samples/seconds/samplerate are clamped, sigrok-cli runs with a timeout, YAML is safe_load. Do not expose this server over the network without adding authentication.

Self-tests: no hardware, run from anywhere

hwcontract --selftest                       # full MCP round-trip
python3 -m hwcontract.judge --demo           # tiny synthetic self-test, includes MISSING RESET
python3 -m hwcontract.sigrok_adapter --demo
python3 -m hwcontract.serial_adapter --demo
pytest                                      # the tests/ suite; pip install -e .[dev]

CI runs the suite on every push and PR, and the PyPI publish job waits for it.

Available Tools

8 tools
capture_ws2812A

Capture the WS2812 data line via sigrok and return measured pulse-width observations (T0H/T1H/...).

ParametersJSON Schema
NameRequiredDescriptionDefault
driverNo
channelNo
samplesNo
samplerateNo

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses that the tool captures data and returns observations, implying a read-only operation. However, it lacks explicit safety details, prerequisites (e.g., sigrok setup, hardware requirements), and possible side effects, leaving some transparency gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence that front-loads the main action and purpose. It avoids redundancy and every part contributes meaning, making it highly concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 4 parameters, no output schema, and no annotations, the description is insufficient for the agent to fully understand how to use the tool. It does not explain parameter semantics, return format details (units, structure), or any prerequisites, leaving significant gaps for a tool of this complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and the description does not mention any parameter details. The parameter names (driver, channel, samples, samplerate) are self-explanatory to some extent, but the absence of units, allowed values, or relationships means the agent gains no additional meaning from the description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: capturing the WS2812 data line via sigrok and returning measured pulse-width observations (T0H/T1H/...). The verb 'Capture' and specific resource (WS2812 data line) distinguish it from sibling tools like check_ws2812 which likely validate rather than capture raw data.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a clear context: use this when you need to capture WS2812 signals for pulse-width analysis. It does not explicitly exclude alternatives or name sibling tools, but the specific context (capture vs. check/judge) implies a distinct use case, earning a score slightly below 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

check_dshotB

Capture a live DShot600 ESC signal AND judge it against a contract in one call.

ParametersJSON Schema
NameRequiredDescriptionDefault
driverNo
channelNo
samplesNo
samplerateNo
contract_pathYes

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must disclose side effects and prerequisites. It reveals the combined capture-and-judge behavior but omits any mention of hardware requirements, permissions, or failure handling, leaving a significant transparency gap for a tool combining two operations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence that front-loads the key actions and resource, with no filler. Every word contributes to the purpose, making it highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of a combined capture-and-judge tool with five parameters and no output schema, the description is insufficient. It does not explain the required contract_path, the role of driver/channel/samples/samplerate, or the format of the judgment result, leaving major operational gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description is fully responsible for explaining the five parameters, but it only mentions DShot600 and contract without detailing driver, channel, samples, samplerate, or contract_path. This leaves all parameters unspecified and does not compensate for the schema's lack of descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it captures a live DShot600 ESC signal and judges it against a contract, combining two actions in one call. The specific verb+resource ('capture...AND judge...DShot600 ESC signal') distinguishes it from separate capture-only or judge-only sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'in one call' phrasing implies it is used when both capture and judgment are needed at once, suggesting an alternative to separate operations. However, it does not explicitly state when not to use it or name alternatives like judge_contract, though the sibling context provides this implication.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

check_serialB

Read a serial port for N seconds AND judge its log against a contract's expect/forbid patterns.

ParametersJSON Schema
NameRequiredDescriptionDefault
baudNo
portYes
secondsNo
contract_pathYes

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full responsibility for behavioral disclosure. It mentions reading for N seconds and judging, but omits critical details such as blocking behavior, return format, pass/fail semantics, or any side effects. This is a significant transparency gap for a hardware-interfacing tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence with no filler. The core action ('Read a serial port') is front-loaded, and the compound purpose is communicated efficiently.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has no annotations, no output schema, and 0% parameter documentation, the description is too sparse. It fails to specify return values, error behavior, or how the contract 'judgment' is reported, leaving the agent without enough context for reliable invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It indirectly clarifies 'port', 'seconds', and 'contract_path' but completely ignores 'baud', which is a necessary serial configuration parameter. The partial coverage leaves the meaning of 'baud' ambiguous and the overall parameter semantics incomplete.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's verb and resource: it 'Read[s] a serial port for N seconds' and 'judge[s] its log against a contract's expect/forbid patterns.' This distinguishes it from siblings like judge_serial (which likely only judges) by explicitly combining capture and judgment.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no explicit guidance on when to use this tool versus alternatives like judge_serial or capture_ws2812. 'Read AND judge' implies a combined use case, but no clear prerequisites, exclusions, or alternative references are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

check_ws2812A

Capture a live WS2812 signal AND judge it against a contract in one call. The one-shot an agent reaches for.

ParametersJSON Schema
NameRequiredDescriptionDefault
driverNo
channelNo
samplesNo
samplerateNo
contract_pathYes

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden for behavioral disclosure. It indicates the combined operation (capture+judge) but does not disclose potential side effects, whether it blocks, or return format. Some transparency is present, but depth is lacking.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, immediately front-loaded with the core action. Every word earns its place, conveying purpose and usage without fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has 5 parameters, no annotations, and no output schema, yet the description only covers its overall purpose. Missing are parameter prerequisites, contract format, return values, and any setup requirements, making it incomplete for confident invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and the description mentions no parameter names or meanings. Parameters like 'samples' and 'samplerate' lack units or context, and 'contract_path' is not explicitly connected to the contract mentioned in the description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool captures a live WS2812 signal and judges it against a contract in one call. This combines the functionality of sibling tools capture_ws2812 and judge_contract, making its purpose distinct and specific.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'The one-shot an agent reaches for' implies this is the preferred combined tool, suggesting it should be used when both capture and judgment are needed. However, it does not explicitly name alternatives or state when to use separate tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

judge_contractB

Judge observations against a timing contract -> PASS/MARGINAL/FAIL/MISSING. No hardware; use for replay or when an adapter already captured.

ParametersJSON Schema
NameRequiredDescriptionDefault
observationsYes
contract_pathYes

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. It usefully discloses that no hardware is touched (offline/replay operation) and lists the four output verdicts, which is meaningful beyond the schema. It says nothing about error behavior when the contract file is missing or malformed, or whether the verdicts are mutually exclusive per observation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact clauses with the outcome list front-loaded and the usage note appended; nothing is wasted. The arrow notation packs the return contract into a few tokens, though the second clause is slightly clipped in phrasing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema and no annotations, so the description must carry everything, and it does not. The verdict vocabulary is covered, but the input contract (contract file format, observation object shape) is completely unspecified for a tool whose entire correctness depends on those two inputs matching.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% for two required parameters, so the description must compensate and does not. 'contract_path' is never clarified as a filesystem path or identifier, and the expected shape of 'observations' objects (timestamped samples? channel/value pairs?) is entirely unstated, leaving the agent to guess the input format.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('Judge') and resource ('observations against a timing contract') and enumerates the four possible verdicts, so the agent knows exactly what comes out. It implicitly separates itself from hardware-capture siblings via 'No hardware', though it does not distinguish itself from judge_events or judge_serial, which look near-identical in shape.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'use for replay or when an adapter already captured' gives a real condition for selecting this tool over the capture_*/check_* siblings. However, it never names an alternative explicitly and gives no guidance on when to prefer judge_events or judge_serial, which share the same judging verb.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

judge_eventsB

Judge decoded hardware events against an events contract's temporal assertions (when/require/within, forbid/while/before). Events: [{source, type, start_ns, end_ns, fields}].

ParametersJSON Schema
NameRequiredDescriptionDefault
eventsYes
contract_pathYes

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it does disclose the assertion kinds checked (when/require/within, forbid/while/before) and the event record shape. However, it says nothing about the outcome semantics: what happens on a violation, whether it is a pure read/verdict operation, or what permissions the contract file requires.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two dense sentences with the action front-loaded and no filler; the assertion list and event shape are packed into parentheticals efficiently. Slightly cryptic for readers unfamiliar with the assertion vocabulary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema and no annotations mean the description should say more about the verdict/return shape and error behavior, which it does not. The input side is reasonably covered, but an agent calling this cannot predict what a result looks like.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description's inline expansion of the events array (source, type, start_ns, end_ns, fields) is genuinely valuable and compensates for the empty items schema. It leaves contract_path completely unexplained, so the compensation is only partial.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (judge) and resource (decoded hardware events) and adds the contract's temporal assertion vocabulary, which pins down the operation precisely. It is distinguishable from judge_contract by scope (events vs. the contract itself), though it never explicitly contrasts the two.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit when-to-use guidance, no prerequisites, and no named alternative among the siblings (judge_contract, list_contracts). The phrase 'decoded hardware events' hints at a pipeline stage, but the agent must infer when this tool is the right choice.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

judge_serialA

Judge a captured serial log against expect/forbid patterns. No hardware; use for replay or logs another tool already captured.

ParametersJSON Schema
NameRequiredDescriptionDefault
logYes
contract_pathYes

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description discloses that the tool does not require hardware, which is a key behavioral trait. However, it does not explicitly state read-only behavior or side effects, leaving some ambiguity for the agent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the core action, and contains no unnecessary information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple with two string parameters and no output schema. The description conveys the purpose and usage context but does not specify the return value or output format, which could leave an agent uncertain about how to interpret the results.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has no descriptions (0% coverage). The description mentions 'serial log' and 'expect/forbid patterns', which gives partial meaning to the `log` and `contract_path` parameters, but it doesn't explain their format or relationship clearly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: judging a captured serial log against expect/forbid patterns. It also distinguishes itself from hardware-related tools by noting 'No hardware', making it stand out among siblings like capture_ws2812 and check_serial.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly specifies when to use this tool: for replay or logs already captured, and states it is not for hardware. This gives clear guidance on when to invoke it, though it doesn't explicitly name alternative tools like check_serial for live use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_contractsA

List bundled contract templates, names, kinds, installed paths, and capture adapter requirements. No hardware.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It does disclose the read-only, hardware-free nature ('No hardware.') and what the listing contains, but says nothing about pagination, permissions, or whether results are static bundled data versus dynamically discovered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with the verb and the returned fields, with the constraint ('No hardware.') placed last where it belongs. No filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-param listing tool with no output schema, the description usefully enumerates the returned fields, which compensates for the absent output schema. Minor gaps remain around what 'kinds' and 'capture adapter requirements' mean in practice.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is nothing for the description to disambiguate; baseline 4 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb+resource ('List bundled contract templates') with an enumeration of returned fields (names, kinds, installed paths, capture adapter requirements). It is distinguishable from judge_contract, though the distinction is inferred from the verb rather than stated explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The trailing 'No hardware.' implicitly tells the agent this call is safe without a device attached, but there is no explicit when-to-use statement and no named alternative (e.g. judge_contract) for the contrasting case.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 2 tool updatesv0.6.0
    • Addedjudge_events
    • Addedlist_contracts
  2. 6 tool updatesv0.1.2
    • First observedcapture_ws2812
    • First observedcheck_dshot
    • First observedcheck_serial
    • First observedcheck_ws2812
    • First observedjudge_contract
    • First observedjudge_serial

TDQS

B3.4/5.0

Scored across 8 tools

Disambiguation4/5

The set splits cleanly into judge_* (no-hardware judging of already-captured data), check_* (capture + judge in one call), and capture_ws2812 (capture only), and descriptions explicitly guide selection (e.g. check_ws2812 is 'the one-shot an agent reaches for'). Minor overlap remains between judge_contract (timing contract) and judge_events (events contract) and judge_serial, but each is scoped to a distinct contract kind or data source so boundaries are mostly clear.

Naming Consistency4/5

All names use a consistent snake_case verb_noun pattern (list_, judge_, capture_, check_). The only wrinkle is semantic: capture_ws2812 and check_ws2812 both 'capture' the WS2812 line but use different verbs, and there is no capture_dshot/capture_serial to mirror capture_ws2812, creating mild asymmetry despite uniform formatting.

Tool Count4/5

Eight tools is well within the ideal 3-15 range and each family (list, judge, capture, check) covers a genuine need. Slight redundancy exists since capture_ws2812 is partially subsumed by check_ws2812, but the raw-observation use case justifies keeping both.

Completeness3/5

Coverage spans listing, capturing, and judging for timing/events/serial/WS2812/DShot, but the contract lifecycle is thin: list_contracts exists with no create/update/delete or contract-authoring operation, and raw-only capture exists for WS2812 but not for DShot or serial. These are workable but notable gaps for a contract-driven validation toolset.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers