Skip to main content
Glama

cw_benchmark

Measure which SONAR capabilities and transports actually work with a few-second read-only sweep, then opt into write and plugin stages. Rerun after a SONAR update to confirm boundaries.

Instructions

Measure what this adapter can actually do against the running SONAR, and how fast: every capability grouped by transport, with the confirmation method for each result. Use it after a SONAR update to re-establish the boundaries. Measured: a full sweep takes 70s+, and not only because of the plugin scan — the write cases post a command and then wait for the UI to settle, and four of them account for ~55s of it. So the default answers from the read half (a few seconds) and every expensive stage is opt-in and named in skipped. Pass full: true for everything. The write stages can start playback and send MIDI to the selected destination; they do not insert tracks or buses.

Input Schema

NameRequiredDescriptionDefault
fullNoRun every stage: the write cases and the plugin inventory scan. Takes 70s+. The default read-only sweep answers in a few seconds and says which stages it skipped.
includeWritesNoRun the cases that post a command and wait for the UI. ~55s of the total.
includeMidiSendNoOnly meaningful when the write cases run.
includePluginScanNoExplicit override for the plugin-inventory stage; implies nothing about writes.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.2.0

TDQS

A4.1/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so well: it discloses full-sweep runtime, default read-half speed, skipped-stage reporting, and the side effects of write stages (starting playback and sending MIDI, but not inserting tracks or buses). This gives the agent critical safety and cost context before invoking it.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with purpose and usage, then gives performance and side-effect details that are relevant for a benchmark tool. It is somewhat long, but most sentences earn their place by clarifying cost or safety.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema and no annotations, the description provides enough context to call the tool correctly: it explains default vs full behavior, skipped stages, timing, and write-stage side effects. It stops short of describing exact return values, but that is a minor gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the input schema already explains each parameter in detail. The description adds some framing around default behavior and full:true, but largely repeats what the schema provides, making the baseline 3 appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: measuring what the adapter can do against the running SONAR, including speed and capability grouping by transport. It is clearly a benchmark/diagnostic tool, though it does not explicitly distinguish itself from nearby tools like cw_doctor or cw_status.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear usage context: run it after a SONAR update to re-establish boundaries, and use the default read-only sweep unless full coverage is needed. It also explains that expensive write stages are opt-in, but it does not name alternative tools or exclusion cases.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.