Skip to main content
Glama

sim_expect

Runs firmware in simulation and asserts that specific text or regex patterns appear on UART. Ideal for automated unit & integration testing in a Red-Green-Refactor agent loop.

Instructions

Runs firmware in simulation and asserts that specific text or regex patterns appear on UART. Ideal for automated unit & integration testing in a Red-Green-Refactor agent loop.

Input Schema

NameRequiredDescriptionDefault
chipNoTarget chip family. Default is s3 (ESP32-S3).
wifiNoAttach a virtual access point with DHCP/DNS/NAT. Spec like 'ssid=demo,psk=secret' (keys: ssid, psk, chan, bssid), or a bare SSID for an open network. Station gets 10.0.2.15.
boardNoBoard model: S3 'atech14' (default), 'waveshare-lcd4b', 'waveshare-amoled18-v2', 'waveshare-cam'; C6 'waveshare-c6-lcd147'; ESP32 'ttgo-tdisplay', 'esp32dev-loopback'; 'none' for a bare module. Omit on the C3, which has no boards. See sim_list_boards.
appBinNoDirect path to the application binary (.bin).
eventsNoOptional interaction events before checking patterns.
orderedNoWhether patterns must appear in exact chronological sequence (default: true).
patternsYesExpected console output. Plain text matches as a substring of a line ("Counter: 5"); write a regular expression between slashes ("/Counter: \\d+/", "/wifi.*connected/i").
ptableBinNoDirect path to partition-table.bin (needed with appBin for a real ROM boot).
maxSecondsNoMax emulated seconds to wait for patterns (default 10).
projectDirNoPath to the ESP-IDF project root or build folder. If omitted, uses current working directory.
bootloaderBinNoDirect path to bootloader.bin (needed with appBin for a real ROM boot).

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changedv0.2.6
    • changedInput schema / properties / patterns / description
      Previous value: -"List of expected strings or regex patterns on UART output."New value: +"Expected console output. Plain text matches as a substring of a line (\"Counter: 5\"); write a regular expression between slashes (\"/Counter: \\\\d+/\", \"/wifi.*connected/i\")."
  2. First observedv0.1.1

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry the full behavioral disclosure burden. It says the tool runs firmware and asserts UART patterns, but it does not explain failure behavior, whether state is modified, how events affect execution, or what the assertion result contains.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two concise sentences with the core action front-loaded. Every sentence contributes relevant information without unnecessary detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 11-parameter simulation tool with no annotations and no output schema, the description covers the central assertion purpose but leaves output and failure semantics thin. The schema fills in the parameter details, but the overall definition does not fully explain what an agent should expect from the assertion result.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the parameters are already well documented by the schema. The description restates the meaning of patterns but adds no meaningful parameter guidance beyond what the schema provides, making the baseline score appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action and resource: runs firmware in simulation and asserts that text or regex patterns appear on UART. This is clear enough to distinguish the core assertion behavior, but it does not explicitly differentiate the tool from siblings such as sim_run or sim_script.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives useful context by saying it is ideal for automated unit and integration testing in a Red-Green-Refactor loop, which implies when to use it. However, it names no alternatives and gives no explicit when-not-to-use guidance relative to the sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.