Skip to main content
Glama
SH4RKKK

ibm-i-5250-mcp

by SH4RKKK

run_tests

Replay Markdown-based 5250 screen regression tests to verify a program's display behavior after a code change. Reports which tests pass.

Instructions

Replay screen regression tests against a program and report what passed. A test is a Markdown file with 5250-do blocks (the same action lines screen_do takes) and 5250-expect blocks (checks against the screen that came back), so it reads as a specification. Use this after compiling a change to see whether the display behaviour it was supposed to keep still holds, or before making one to write the expected screens first and watch them fail. Opens its own session and closes it, unless you pass session to reuse one that is already open.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
pathNoa test file, or a directory of *.md tests to run in name order (default: "screen-tests")
filterNoonly run tests whose file name contains this, ignoring case
serverNowhich IBM i to use, named by a .env.<name> file. Omit for the default .env server
sessionNoreuse this already open session instead of opening and closing one. The session is left where the last test finished

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description carries the full disclosure burden. It covers the test file format, the outcome ('report what passed'), and important session lifecycle behavior: opens/closes its own session unless `session` is passed, and leaves the session where the last test finished. It does not disclose failure behavior or output format, but the added session context is genuinely useful and goes beyond the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences, each earning its place: the main action, the test format, the recommended usage timing, and the session behavior. The primary verb and result are front-loaded, and there is no repetition of schema properties or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with four fully documented optional parameters, the description is close to complete: it explains what a test is, when to run it, and the session side effects. The main gap is 'report what passed' — there is no output schema and no detail about how pass/fail results are presented, though this is a minor omission for a test runner.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema documents all four parameters with descriptions (100% coverage), so the baseline is 3. The description adds useful background on authoring test files with 5250-do/5250-expect blocks, which helps an agent understand the tool's domain, but it does not add parameter-specific meaning beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Replay screen regression tests against a program and report what passed.' It also distinguishes itself from siblings by explaining the 5250-do/5250-expect Markdown test format and how it relates to screen_do, making it clear this is the batch test-runner rather than an interactive screen tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit timing guidance: 'Use this after compiling a change... or before making one...' to verify preserved behavior or drive expected screens. It also explains when to reuse a session versus letting the tool open and close its own. It does not explicitly name when-not-to-use alternatives, which keeps it just below a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.