Step Engineer

A reusable, Step-5-Preview-specific agent harness for constrained search: explore better implementations and strategies inside a clearly defined environment, using measured feedback and fixed acceptance rules. Use its ready-made MCP server and CLI, or build your own interfaces around its reusable Python components. GPT, Grok, Claude, or another agent defines the task while Step explores the permitted tradeoffs.

The current implementation works on local implementation or policy files: it edits an isolated copy, measures candidates against fixed checks, preserves the best qualifying version, and returns a patch with verification evidence. A simulator can supply the environment and score; the deliverable is still reviewed code, not a deployed controller.

Your parent agent keeps its existing model and reasoning settings, including Ultra where supported. Step Engineer does not replace the parent agent, apply patches to the original project, or publish changes.

繁體中文說明 · Use cases · Task brief · Why Step? Evidence and charts · Build your MCP / CLI · Parent-agent instructions

When to delegate to Step

Use it when you can specify the environment, allowed decisions, objective, hard constraints, permitted tradeoffs, and final validation. A useful task has a feasible baseline and more than one plausible strategy. The parent fixes what counts as success; Step is free to search the implementation space.

Being near a limit is useful only when it improves the objective and remains valid on the intended workload. Real-world variation may require an explicit margin. Contradictory hard constraints require a revised task contract; Step must not silently relax them. See the use-case guide and copyable task brief before submitting a job.

Related MCP server: MCP ToolHub

Why a harness for Step?

Our specialization hypothesis is that Step is useful when explicit constraints, an objective evaluator, and room to explore better solutions meet. StepFun's GPU-kernel and training-data experiments motivate it. These are vendor evidence, not proof that Step outperforms other models or that proximity to a constraint is itself a measure of quality. We evaluate task outcomes, not rule exploitation.

Vendor-reported results, checked 2026-09-26: a fixed MLA workload on one H100, with a 24-hour budget per run. This compares best runs with different reasoning settings, not averages or equal-cost results. Official presentation; method, source data, and limitations.

Our own live case reduced fresh nine-pose composition time by 10.57%, with identical RGBA outputs in the tested cases. Two preceding attempts produced no patch. This is one local workload, not a comparative model benchmark. The evidence guide separates official results, our measurements, and untested hypotheses, and explains which model capabilities this harness currently uses.

How other agents use it

What the harness does, and why it exists

Step proposes the next change. The harness turns those proposals into a bounded, measured search: it controls the editable state, supplies execution feedback, and decides whether the saved result meets the caller's acceptance criteria. This runtime is the part developers reuse behind their MCP or CLI.

flowchart TD
    O["Orchestrator defines environment, allowed tradeoffs<br/>Objective, hard constraints, and holdout checks"] -->|"MCP / CLI / tool adapter"| C
    subgraph H["Step Engineer harness: reusable execution and validation"]
        C["1. Validate scope and snapshot files<br/>Keep the original project intact"] --> B["2. Measure the unchanged baseline<br/>Establish the comparison"]
        B --> L["3. Dispatch permitted tools<br/>Control reads, edits, and execution"]
        L -->|"evaluate_candidate"| E["4. Run supplied checks and benchmarks<br/>Measure validity and improvement"]
        E --> K["5. Save the best feasible candidate<br/>Keep measured progress"]
        K -->|"Feedback for the next attempt"| L
        L -->|"Normal loop stop"| V["6. Revalidate saved best; write evidence<br/>Accept a checked result, not the last draft"]
        K -.->|"Saved snapshot"| V
        G["Across the loop: time, tokens, tools, estimated cost<br/>Stop limits and final-validation time reserve"] -.-> L
    end
    L -->|"Context and tool results"| S["Step-5-Preview API"]
    S -->|"Proposed edits and tool calls"| L
    V --> R["Orchestrator reviews patch, metrics, usage, and stop reason"]

The diagram shows the normal optimization path. An invalid baseline stops before model iteration. Cancellation or an execution error records a termination result; it does not guarantee final validation or an accepted patch.

Ownership: the orchestrator supplies the objective, checks, benchmark, thresholds, and optional independent final checks. The harness executes these supplied evaluators and records runtime evidence; it does not design the tests or constitute a cross-model evaluation platform. Step proposes changes and reacts to feedback. The orchestrator reviews the result and decides whether to apply it.

Saving best is automatic after a qualifying measurement; restoring the working candidate requires the restore_best tool. Final validation uses a fresh copy of saved best, and accepted.patch contains changes only after acceptance. Without separate final_checks, the result reports independently_checked=false. Estimated cost limits and the validation-time reserve do not guarantee an exact bill or a successful final check.

The final-validation allowance is deducted from both model-request and development command deadlines. Reaching that exploration deadline stops further tool dispatch and hands the saved best to final validation. Baseline and final checks use the remaining job deadline. The allowance retains a 40% job-time cap, and process cleanup adds overhead, so it remains a best-effort reservation.

Where MCP and CLI fit

MCP and CLI are entry points to this runtime. The built-in CLI calls Harness directly; MCP and the documented custom wrappers use JobService for allowed source roots, background jobs, polling, and cancellation. The caller owns the process/session lifetime. Cloud models issue tool calls through a local host; they do not directly access your filesystem. See custom interfaces.

The current release provides reusable Python components and examples; it does not generate a new MCP/CLI project automatically. Candidate execution currently requires macOS. Larger-context and multimodal model capabilities are documented separately from what this text-based worker exposes.

What it provides

• A Step API client with bounded requests, usage accounting, and no automatic retries.

• An optimization loop with explicit file permissions, correctness checks, repeated benchmarks, and separate final validation.

• A macOS process sandbox for fixed, parent-owned commands.

• CLI and asynchronous MCP stdio interfaces.

• Tool schemas and local dispatch for OpenAI Responses, Grok Chat Completions, and Claude tools.

• A fully local, scripted demonstration that requires no API key.

The default worker setting is medium, with up to 16,384 output tokens and 300 seconds per request, and 250,000 total tokens per job. These are a starting configuration, not a guarantee of improvement. In one live workload, two high attempts exhausted their output limits without producing a patch; a medium attempt produced a verified improvement. See the case study for the failures, measurements, and limits of that comparison.

Requirements

• macOS with /usr/bin/sandbox-exec. Candidate execution fails closed on unsupported systems; there is no unsandboxed fallback.

• Python 3.11 or newer. Python 3.12 is recommended for the tested setup.

• uv to install the locked dependencies.

• For live optimization: a Step API key with model access and sufficient provider quota.

The sandbox supports the installed Python runtime and narrow system paths. Other runtimes or compilers may need additional setup. Validate the required toolchain before delegating a job; do not disable isolation to work around a missing runtime.

Start with the offline demo

Clone the repository, then run the demo:

git clone https://github.com/ShemYu/step-engineer.git
cd step-engineer
uv sync --frozen --python 3.12
uv run step-engineer run examples/batch_aggregation/job.json --offline-demo

This runs the real harness and sandbox with a handwritten, scripted solution. It demonstrates the pipeline; it is not a Step model benchmark. The example documentation explains the contract and verification.

Configure a live worker

Copy the credential template and restrict access:

cp .env.example .env.local
chmod 600 .env.local

Edit .env.local locally and set the placeholder to your own key:

STEP_API_KEY=YOUR_STEP_API_KEY
STEP_BASE_URL=https://api.stepfun.ai/v1

Never commit this file, paste the key into a model prompt, or include credential files in a job. The client accepts the documented Step endpoint only. STEPFUN_API_KEY is a fallback variable; an existing process environment takes precedence over --env-file.

uv run step-engineer --env-file .env.local doctor
uv run step-engineer --env-file .env.local run examples/batch_aggregation/job.json

doctor checks local configuration and sandbox availability. It does not verify authentication, quota, or model access. Live submission sends selected source and tool feedback to StepFun and can incur charges.

The bundled live job uses medium, at most 12 model requests, 24 tool calls, 600 seconds, 250,000 total tokens, and an estimated US$0.50. Its per-request limits are 300 seconds and 16,384 output tokens.

Delegate from a parent agent

A local MCP host can launch the stdio server:

uv run --project /absolute/path/to/step-engineer step-engineer \
  --env-file /absolute/path/to/private/step.env \
  --runs-dir /absolute/path/to/optimization-runs \
  serve --allow-root /absolute/path/to/your-project

All paths above are placeholders. Repeat --allow-root for each authorized source root. If omitted, only the bundled example is allowed. No HTTP port is opened.

Keep the MCP session alive for the job. Closing the server cancels its active jobs; restarting it does not resume them or send new model requests.

GPT, Grok, or Claude can be the parent, but a local application must execute their tool calls. A cloud model cannot directly access this machine's filesystem or localhost. The integration guide covers Codex, Claude Code, and the provider-neutral Python adapter. Adapter schemas and dispatch have local tests; paid end-to-end tests against all three parent providers are not claimed.

Define a job

Copy the example job, then inspect the full schema:

uv run step-engineer schema

Commands use argv arrays, without shell expansion. {python} selects the installed interpreter; {workspace} and {scratch} expand to isolated directories. The command workspace is read-only. Write build products and temporary files to scratch.

elapsed_seconds is measured externally and includes process startup, input preparation, and cleanup. A trusted benchmark may emit custom metrics as its last JSON line:

{"metrics":{"recall":0.98,"qps":4200}}

Program output cannot override external elapsed_seconds. Missing or non-finite metrics, failed checks, timeouts, and output/storage limit violations reject the measurement. A custom stdout score is still only as trustworthy as its evaluator.

Budgets and acceptance

Generic defaults are 12 model turns, 40 tool calls, 600 seconds, 250,000 total tokens, an estimated US$1, and four evaluations without improvement. Each request is capped at 16,384 output tokens and 300 seconds. The bundled example tightens the tool and estimated cost limits.

max_request_seconds accepts 5–600 seconds. Its effective limit is also capped by the job's remaining time after a final-validation reserve. Token and estimated cost reservations can stop a job before its nominal maxima. Longer output limits include reasoning tokens, not just visible answers.

Cost estimates use US$1 per million input tokens and US$2.70 per million output tokens, checked on 2026-09-26, and ignore cache discounts. They are conservative local estimates, not a billing guarantee. Interrupted or timed-out requests can still be billed; uncertain usage is marked, and requests are not automatically retried.

The harness first measures the unchanged snapshot. Only a qualifying, better evaluated candidate becomes best. Final validation uses a fresh copy of that saved version. An unmeasured last edit never replaces the saved best. Budget exhaustion and acceptance are reported separately: a run may exhaust its budget while an earlier measured candidate still passes final verification.

Improvement-threshold comparisons allow score-scale floating-point roundoff at the boundary, while still requiring a strictly better score. This tolerance does not relax metric constraints or compensate for benchmark noise.

max_no_improvement counts completed candidate evaluations that do not improve the saved best, including infeasible candidates. A new best resets the count; reads, writes, and rejected tool arguments do not increment it. Reaching the limit stops the rest of the tool batch and proceeds to final validation.

The guard-repair case study records four bounded Step attempts and the subsequent developer-agent repairs and parent review, including failures and costs. It is not a model benchmark.

Artifacts are written under runs/<run_id>/ in the current working directory by default. Set --runs-dir explicitly when integrating a host:

Inspect accepted, final_validation, and stop_reason together. Without separate final_checks, the harness reruns normal checks and the benchmark but reports independently_checked=false. The original source directory remains unchanged; applying a patch is a parent-owned decision.

Data and isolation boundaries

Live jobs send selected visible source contents and tool feedback to StepFun. Exclude credentials, unrelated data, and material you cannot share with that provider. Final-only files are withheld from the worker's development tools, but are visible to the program being tested during final validation.

The runner blocks network access, prevents writes to the command workspace, restricts filesystem reads, and constructs a clean child environment without API keys. Output is capped at 16 KiB per stream; scratch has a 16 MiB per-file limit and a monitored 64 MiB aggregate limit. Scratch is removed after commands.

This is a guardrail for personal engineering work, not a VM or a hostile multi-tenant execution service. Escaped descendants and tampering inside the same interpreter require stronger isolation. Source copies, reports, patches, or test output can contain sensitive project data; review them before sharing. Raw model reasoning is kept in memory for tool continuity, not saved in reports.

Development checks

uv sync --frozen --python 3.12
uv run pytest -q
uv run ruff check src tests tools examples/resource_allocation

Tests cover provider HTTP mocks, state transitions, path boundaries, real macOS sandbox execution, and MCP stdio. Passing them establishes harness behavior; it does not establish a model's quality on an unseen workload.

References

• Step 5 Preview: official model capabilities and limits

• Step 5 Preview: official experiments and results

• Step quickstart

• Step Chat Completions

• Step tool calls

• Step reasoning

• Step pricing

• Evidence, chart data, and the complete official-documentation directory

• MCP Python SDK v1

This project uses the MCP Python SDK v1 interface.

License

MIT. The private application assets described in the case study are not included or licensed by this repository.