The codecalc server provides a comprehensive suite of MCP tools for AI models to execute code, perform exact/symbolic math, solve logic problems, manage sessions, and analyze code—all within a sandboxed, resource-limited environment.
Code Execution: Run code in 31 languages (Python, Node, Go, Rust, C, C++, etc.) with CPU/memory/output/timeout limits and network blocking (execute_code, execute_code_stream). Stream partial output, compare execution across languages (compare_execution), and list available languages (list_languages).
Sessions & Workspaces: Start persistent sessions with stateful REPLs (Python/Node) or workspace directories (session_start, session_stop, session_list). Manage files (session_files, session_read_file, session_write_file), run multi-file programs (session_run), list artifacts (session_artifacts), and install packages (install_package).
Symbolic Mathematics: Evaluate, integrate, differentiate, simplify, and solve equations with sympy (evaluate_expression, simplify_expression, solve_expression, limit_expression). Check algebraic equivalence (algebraic_equiv) and solve systems of linear equations (solve_linear).
Exact Arithmetic & Programmer Tools: Perform exact rational arithmetic (calc_exact), compare thresholds (compare_threshold), calculate percentages (percentage). Analyze bits (bit_analysis, bitop), convert bases (base_repr, radix_convert), inspect floats (float_repr), check integer widths (int_widths).
Statistics & Probabilities: Compute mean, median, stdev, CV (calc_stats), percentiles (percentiles), and hash collision probabilities (collision_probability).
Logic & SMT: Generate truth tables for boolean expressions (truth_table) and check SMT-LIB2 satisfiability with Z3 (z3_check).
Code Analysis & Optimization: Estimate Big-O complexity via tree-sitter or benchmarking (analyze_complexity, benchmark). Translate code between languages with verification (translate_code), optimize code with proof and speedup requirements (optimize_code), extract functions (extract_function), and compare edge cases (compare_edge_cases).
Units, Constants & Time: Convert dimensional units (convert_units, list_units), look up physical constants (physical_constants), convert data sizes (data_sizes), humanize durations (human_duration), and convert epoch timestamps (epoch_time).
Runtime Management: Check runtime versions and available updates (runtimes_status), update language runtimes (update_runtimes).
LLM-Assisted Operations: Fetch documentation from Context7 (context7_docs). Translate, optimize, and compare code using LLM gateway when configured.
Enables running Bun JavaScript/TypeScript code with sandboxing, resource limits, and stdout/stderr capture.
Enables compiling and running C++ code with sandboxing, resource limits, and output capture.
Enables running Deno TypeScript/JavaScript code with sandboxing, resource limits, and output capture.
Enables running Elixir code with sandboxing, resource limits, and output capture.
Enables running Erlang code with sandboxing, resource limits, and output capture.
Enables compiling and running Fortran code with sandboxing, resource limits, and output capture.
Enables running Haskell code with sandboxing, resource limits, and output capture; scaffolds a temporary Nix project.
Enables compiling and running Kotlin code with sandboxing, resource limits, and output capture.
Enables running Lua code with sandboxing, resource limits, and output capture.
Enables running Perl code with sandboxing, resource limits, and output capture.
Enables running PHP code with sandboxing, resource limits, and output capture.
Enables running Python code with sandboxing, resource limits, output capture, and a persistent REPL session.
Enables running Ruby code with sandboxing, resource limits, and output capture.
Enables compiling and running Rust code with sandboxing, resource limits, and output capture.
Enables running SQLite SQL scripts with sandboxing, resource limits, and output capture.
Enables compiling and running Swift code with sandboxing, resource limits, and output capture.
Enables running TypeScript code (via the TypeScript compiler) with sandboxing, resource limits, and output capture.
Enables compiling and running Zig code with sandboxing, resource limits, and output capture.
Enables running Zsh shell scripts with sandboxing, resource limits, and output capture.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@codecalc what is 0.1 + 0.2 exactly?"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
codecalc is an offline, self-hosted MCP server that gives an AI agent a calculator, a code runner, and a logic checker — so it gets a correct answer instead of a guessed one. It runs code in 31 languages, does exact symbolic math, solves SMT/logic problems, and measures complexity, all exposed as 49 MCP tools.
Fastest path: uvx 'codecalc[full]' setup --write registers codecalc with your MCP client automatically. New to MCP, or want more detail first? See QUICKSTART.md, or the Install section below.
Three things nobody else offers together cleanly:
Offline-core — ships no model, no API key, no gateway, no telemetry.
The core opens no sockets; network access is opt-in and only where a
specific tool's job needs it (the Piston provider, install_package, the
runtime-update tools, executed code unless no_net, and a one-time
in-process grammar download on first analyze_complexity — full breakdown
in the network-boundary table below).
Safe execution of untrusted code — an opt-in strict isolation boundary (gVisor+Docker on Linux, AppContainer on Windows) layered above the default rlimit sandbox, fail-closed and attested.
Verification tools — verify_translation proves a port to another
language behaves identically, verify_optimization proves an optimization
preserved behavior, and z3_check proves or refutes logic with an SMT
solver.
Use it when you want a free, local, private, hardened code-runner and verifier that an MCP agent can call directly — no vendor account, no cloud spend, nothing leaving the machine except where a tool's job explicitly requires it.
Reach for something else when you want managed cloud scale instead of self-hosting (a hosted sandbox like E2B or Modal), or when you're not self-hosting at all and the model vendor's built-in code interpreter already covers what you need.
Related MCP server: Alexandria MCP
codecalc is not a general cloud sandbox and not a vendor code interpreter. It overlaps with several things and beats them in only one narrow place — forcing a model to measure a claim instead of asserting it. Where that isn't what you need, one of these is the better tool, and this table says so plainly.
You want… | Better fit | Why |
To just run some Python/JS quickly, zero setup | Your model vendor's built-in interpreter | Already there, already sandboxed, nothing to install. Anthropic's code-execution tool has internet access "completely disabled" and cannot install packages at runtime; OpenAI's hosted containers have no outbound network access by default, with an org-level |
Heavy or multi-tenant workloads, managed scale | A cloud sandbox (E2B, Modal, Daytona) | Per-tenant Firecracker/gVisor isolation codecalc does not claim by default |
Pure arithmetic or symbolic math, nothing else | A small calculator or SymPy MCP | Lower token cost; none of the 31-language runtime machinery |
A model to stop guessing numbers, equivalence, and speedups — locally, privately, with graded evidence | codecalc | Exact rationals, |
Do not reach for codecalc if you need multi-tenant or network-exposed isolation (its threat model is explicitly single-operator, local, stdio), if zero-setup convenience matters more than measurement, or if a hosted interpreter already covers your case. It earns its keep only when the correctness of the claim — not just "it ran" — is the point.
codecalc setupThe fastest path to a working MCP connection, without reading the rest of this section:
uvx 'codecalc[full]' setup # prints what it would do — nothing on disk changes
uvx 'codecalc[full]' setup --write # applies it: merges your client's config, copies the skillIt detects which MCP client is installed (Claude Desktop, Claude Code,
Cursor, VS Code, Zed — pass --client=NAME if none or several are found),
reuses codecalc doctor's own backend/extras/grammar-cache checks, prints the
exact config block in that client's own JSON shape with absolute paths
already filled in, runs two real canaries (execute_code, evaluate_expression)
to prove the connection would work, and ends in one verdict: ready /
degraded / not-ready. --write is the only mode that changes anything —
it MERGES the codecalc entry into your existing client config (every other
server stays exactly as it was) and backs up the original to
<path>.codecalc-bak first. codecalc --help lists every subcommand.
Published ascodecalc 0.12.0 on PyPI (pip install codecalc) and the
codecalc-exec 0.12.0 executor on crates.io
(#91). Every release
artifact carries a keyless sigstore build-provenance attestation — verify
one with gh attestation verify <file> --repo The-40-Thieves/codecalc; PyPI
wheels additionally carry PEP 740 attestations.
Where | What you get | Link |
PyPI |
| |
crates.io | the | |
GitHub Releases | wheels for every platform, the executor binaries, the | |
MCP registry (official) | the | |
Smithery | hosted listing and one-click client config | |
Glama | hosted listing and the score badge above | |
MCPB (Claude Desktop) | the drag-and-drop bundle, attached to every GitHub Release | see GitHub Releases, above |
Docker MCP Catalog | the | submitted as docker/mcp-registry#5025; listed at hub.docker.com/mcp/server/codecalc once merged |
Not yet listed: PulseMCP and mcp.so do not carry a codecalc entry yet; the Docker MCP Catalog entry is pending review. PulseMCP's own submission page (checked 2026-09-08) says it is not accepting new submissions and that publishing to the official MCP registry — already done, row above — is what it indexes from once submissions reopen, so there is nothing to submit there today. mcp.so takes a submission through its own form. See docs/distribution.md for the exact steps, kept there rather than here because submitting is an action for whoever runs it, not a fact about the current release.
The published install (simplest — no build step, and what most people want):
uvx 'codecalc[full]' # run it directly, no environment to manage
# or
pip install 'codecalc[full]' # into your own virtualenvFrom source, if you would rather build the executor yourself:
git clone https://github.com/The-40-Thieves/codecalc
cd codecalc
uv sync --all-extras # or: pip install -e '.[full]'
cargo build --release --manifest-path executor/Cargo.toml
mkdir -p bin # bin/ is gitignored, so a fresh clone has none
cp executor/target/release/codecalc-exec bin/
uv run codecalc doctor # verify: backend should read `rust`Without the cargo build, everything still runs on the pure-Python fallback —
doctor will say so, and the network table below says what that costs.
Why [full]. The base install is the MCP surface and the sandbox executor:
31 language runtimes, sessions, packages, ~32 MB. The symbolic half — sympy and
z3 — is 88.6 MB measured, and a caller who only runs code should not download an
SMT solver to do it. So it is an extra:
install | size | what you get |
| ~32 MB | execute_code, sessions, packages, complexity-free tools |
| +83 MB | evaluate_expression, solve, limits, truth tables, z3, units |
| +5 MB installed, +89 MB fetched on first use | analyze_complexity via tree-sitter |
| ~120 MB | everything |
Nothing fails silently: a tool whose extra is missing returns
{"ok": false, "error": "sympy is not installed. It ships in the 'symbolic' extra: pip install 'codecalc[symbolic]' ..."}, and codecalc doctor lists
which extras are present before you make a call.
Four names for the capability sets above, plus the two that live outside
pyproject.toml entirely — a Docker image and an opt-in isolation boundary.
The invariant that makes "edition" a meaningful word here: in the edition
that lists a tool, that tool is functional — never listed-but-missing-its-extra.
A tool an edition doesn't have returns the dependency_missing contract error
naming the extra that provides it (see above), not a silent failure or a
tool that appears to exist and doesn't work.
Edition | Install | What you get |
Full |
| The recommended local product: the native (Rust) executor, symbolic tools ( |
Core |
| Execution + non-symbolic tools only — the base install in the table above. Every symbolic/parsing tool is still listed by |
Docker |
| The MCP server itself, packaged to run as an ordinary container. Core-shaped by default: ships |
Strict Host | opt-in; | Not an install, a boundary: the gVisor |
codecalc doctor reports which of these you're actually running (backend,
extras present, strict_runtime prerequisites) — read it before assuming a
capability rather than after a tool call surprises you.
.github/workflows/release.yml publishes a platform-tagged wheel per target
(Linux x86_64/aarch64 musl, macOS x86_64/aarch64, Windows x86_64), each
carrying the matching codecalc-exec binary and — where the platform has
one — its --no-net shim, so executor.backend() == "rust" on install
without a manual build step. No wheel for your platform, or installed from
source instead? Everything still runs; see the network table above for what
falls back and to unenforced in that case.
Point an MCP client at the installed command. The key differs by client —
mcpServers for most, servers for VS Code, context_servers for Zed — so
these are given separately rather than as one snippet to adapt:
Claude Desktop — ~/Library/Application Support/Claude/claude_desktop_config.json
(macOS), %APPDATA%\Claude\claude_desktop_config.json (Windows) · Cursor
(.cursor/mcp.json) and Claude Code (.mcp.json) use the same shape:
{ "mcpServers": { "codecalc": { "command": "uvx", "args": ["codecalc[full]"] } } }VS Code — .vscode/mcp.json, top-level key is servers:
{ "servers": { "codecalc": { "command": "uvx", "args": ["codecalc[full]"] } } }Zed — ~/.config/zed/settings.json, key is context_servers:
{ "context_servers": { "codecalc": { "command": "uvx", "args": ["codecalc[full]"], "env": {} } } }Windows paths need doubled backslashes in JSON. If you installed into a venv
rather than using uvx, point at the interpreter directly:
{ "mcpServers": { "codecalc": {
"command": "C:\\path\\to\\venv\\Scripts\\python.exe",
"args": ["-m", "codecalc"] } } }Run codecalc doctor to print a config block with the absolute paths of your
install already filled in.
Install the skill too. The tools cannot help a model that never reaches for
them — a model confident about 0.1 + 0.2 does not feel uncertain, it feels
finished. codecalc/SKILL.md ships inside the package and says when calling is
mandatory (any non-integer, any comparison you will state, anything past 2^53,
any number stated as a claim), when it is noise (2 + 3 + 4 needs no tool), and
how results must be reported — passed: true means "equivalent on N inputs",
never "verified". codecalc doctor prints its path; copy it into your client's
skills directory. check_claims.py gates it, so it cannot name a tool that does
not exist or a field no tool returns.
Not sure what your install actually resolved? Ask it, rather than finding out from a tool call later:
codecalc doctor # or: python -m codecalc doctorThis is the install verification step. It exits 0 when the install can
execute — a writable workspace and a resolved backend — and 1 when it cannot,
so it works unchanged in a Dockerfile, a provisioning script or a CI job. A
missing optional extra or an uninstalled Haskell does not fail it: those are
facts about the host, not a broken install, and a check that goes red for them
is one people learn to ignore.
It prints the execution backend and the binary behind it, whether installs are
confined, the status of every one of the 31 runtimes, whether the workspace is
writable, and a client config block with absolute paths filled in. All of that
is otherwise discoverable only by making a tool call and reading backend,
unenforced, or a failure.
codecalc doctor --json # the same report, for scripts
codecalc doctor --deep # actually RUN each runtime, and read its version--json emits the report and nothing else, against a published schema
(docs/contract/doctor-v1.schema.json)
carrying the same contract_version and the same policy as a tool result.
Each runtime reports one of four states, and the difference between two of them is which measurement was actually taken:
state | means |
| codecalc knows the language; nothing for it resolves here |
| its command resolves and is executable — not run |
| resolves but cannot run, or was run and failed |
| actually executed here and answered — |
status_basis says which pass produced them. Without --deep nothing is ever
reported available, because nothing was executed, and claiming otherwise for a
binary that was merely found on PATH would be a stronger measurement than was
taken.
Under --deep, a runtime whose version probe never gets an answer (a spawn
failure or a timeout) is unhealthy too; a nonzero exit alone only counts
when the flag used is one confirmed correct for that command (go version,
lua -v, zig version — none of them speak GNU --version, so a bare
nonzero exit there is reported as merely unmeasured, not broken). A captured
failure lands in probe_error, never in version, which holds a version
string or nothing. A compile-then-run language whose run step needs a
SECOND, different tool (kotlin: kotlinc compiles, but run launches
java directly) reports installed only when BOTH resolve; detail names
whichever half is missing.
Building the Rust core yourself, or running from a checkout? See "Build the Rust core" and "Run the server" below.
One-click install: both buttons register uvx codecalc[full] (the
recommended Full edition) and require uv to be installed.
The shortest version of the config above — this registers codecalc as a
stdio MCP server. The console entry point is codecalc, so uvx codecalc
launches it directly:
{
"mcpServers": {
"codecalc": { "command": "uvx", "args": ["codecalc"] }
}
}Installed with pip install codecalc instead? Point at the resolved
command with no args:
{
"mcpServers": {
"codecalc": { "command": "codecalc" }
}
}CodeCalc's core opens no sockets. No model gateway or telemetry is built
in. tests/test_offline.py asserts this for the top-level core modules. The
opt-in Piston provider is the deliberate exception: its wire client lives under
codecalc/provider_adapters/ and is registered only when
CODECALC_PISTON_URL is configured.
That is a claim about the package, not about every tool call, and the difference is worth stating rather than leaving a reader to discover:
layer | reaches the network? |
CodeCalc core | No HTTP client, model gateway, or telemetry. One dependency exception: |
configured Piston provider | Yes, explicitly. Calls only the operator-supplied |
| Yes, by design. It runs uv / npm / gem / cargo, which fetch from their registries. Installer hooks also run outside the sandbox — see SECURITY.md |
| Yes. They shell out to mise / rustup / swiftly / npm, which check remote versions |
code you execute | Yes, unless |
| Yes, before the sandboxed step, through the confined |
These distinctions are stated precisely on purpose: a guarantee described more broadly than it is enforced is exactly the failure mode this project works to avoid, so "offline-core" is scoped to what the structural test can actually support rather than claimed as a blanket "no network calls".
A PEP 723 block alone, with no dependencies argument, can trigger the
install above. execute_code/session_run read the block out of the source
text itself — a caller who passes no dependencies argument at all still gets
a confined uv/npm subprocess and real egress if the code they submit
happens to carry a # /// script block, whenever the refusal rule above does
not apply. This is logged distinctly (dependency_install_implicit in the
audit trail, alongside install_denied) so an operator can tell "source text
alone triggered this" from an explicit install_package/dependencies= call.
To disable it: no_net=True on the call, or a deny-network/strict
CODECALC_CAPABILITY_POLICY — either one refuses before any fetch, block or
no block. execute_code_stream and run_submit read the block the same way
execute_code does. compare_execution is the one holdout: it fans out
across several languages with no per-language install plumbing behind it, so
it REJECTS an explicit dependencies argument with a validation error
rather than approximating one, and DISCLOSES rather than silently drops an
inline PEP 723 block it finds in a snippet — that row's result carries
dependencies: {"status": "unsupported", "reason": ...} instead of
installing from it.
Two ceilings govern a dependency-bearing run, not one. The run's own
timeout bounds the sandboxed step; it says nothing about installing
dependencies FIRST, outside the sandbox. A separate, fixed budget
(codecalc.dependencies.DEFAULT_DEPENDENCY_INSTALL_BUDGET_SECONDS, 120s,
aggregate across every dependency of one run) bounds that step instead —
exceeding it refuses the run with a stamped timeout naming the budget,
before the run's own timeout clock even starts. A sessionless run's
dependency workdir is also held to a disk quota — reusing
CODECALC_SESSION_DISK_QUOTA_MB (below), the same cap a session workspace
already has — and a run that grows past it after a successful install is
refused with a stamped resource_exhausted naming the measured size and the
cap.
The grammar download, stated plainly, because it is the one that is easy to
miss. The other three paths above go through a CHILD PROCESS, which is what
tests/test_offline.py says it cannot see. This one does not:
tree-sitter-language-pack ships a ~5 MB extension and fetches each grammar on
first use, in-process, into a local cache — 28 grammars, 89 MB, about 15
seconds on a cold cache. So the first analyze_complexity call for a given
language opens a socket from inside the server.
It is verified (the pack checks a signature and raises on a checksum mismatch), it is cached, and it never happens again for that language. So the offline-core claim is scoped to steady state: this first-use grammar fetch is the one in-process exception, which is why it is called out here rather than glossed over.
For an offline or egress-restricted install, warm the cache first — it is one
command, and afterwards nothing here reaches the network. If you installed
codecalc (pip install/uvx, not a source checkout), scripts/ did not come
with it, so use the shipped console script instead:
codecalc-prefetch-grammars # installed: fetch all 28 grammars
codecalc-prefetch-grammars --print-cache-dir # installed: the directory to copyBuilding from source? The script still works and calls the same code:
python scripts/prefetch_grammars.py # fetch all 28 grammars
python scripts/prefetch_grammars.py --print-cache-dir # the directory to copycodecalc doctor reports whether that cache is populated, so this is
discoverable before it matters rather than after a tool call degrades.
Layer | Language | Why |
Executor core ( | Rust | Sandbox + rlimits + process-group kill + JSON CLI. No |
Logic layer ( | Python | sympy (symbolic math, equation solving) and z3 (SMT) have no Rust equivalents |
MCP server ( | Python | the official |
Python orchestrates; Rust executes; sympy/z3 reason. Each layer does what it's best at. The Rust binary is preferred automatically; a pure-Python executor is the fallback if the binary is missing.
No modern instruction-set requirements — rustc targets a generic CPU by
default and nothing overrides it. (executor/.cargo/config.toml explains why
-C target-cpu=generic is deliberately NOT written there: it would be a
no-op that reads like a guarantee.)
Static musl builds run on any Linux regardless of glibc version:
bin/codecalc-exec-x86_64-musl, bin/codecalc-exec-aarch64-musl (~430K each;
the exact size moves with every toolchain bump, so it is not pinned here)
Size-optimized profile (opt-level="z", LTO, panic=abort, stripped) —
measured, not assumed: against an otherwise identical opt-level=3 build,
z came out 1.02 ± 0.26 times faster on the executor's own path (i.e. no
detectable difference) while being 16% smaller. The executor spends its time
in syscalls, not arithmetic, so there was nothing for a higher optimisation
level to speed up.
Lazy sympy/z3 imports. Both are imported on first use, so a session that
only executes code never pays for them. This claimed "~40ms, not ~600ms" for a
long time while being wrong in both directions: the server took 1.9s to
start, and sympy was not actually lazy — units.py imported it at module
scope and server.py imports units, so every start paid 437ms for it.
Deferring that took spawn-to-first-response from 1888ms to 1243ms
(measured, median of 7). The remaining ~870ms is the mcp SDK's own import,
which is not ours to remove.
The fork-bomb measurement is taken once, and only when it is needed.
Sizing RLIMIT_NPROC means reading /proc/<pid>/status for every process on
the machine. That walk used to run during argument parsing and again for every
step: a C compile-and-run opened 1767 status files on a 590-process box to
answer one question three times, and --lang notalanguage paid the full cost
to produce a one-line error. Measured lazily and cached, an error costs 1.1ms
instead of 13.3ms and a compiled run 78ms instead of 104ms.
list_languages probes runtime availability and reports which languages
actually work on the machine (graceful degradation on minimal installs)
cd executor
cargo build --release # native
cargo zigbuild --release --target x86_64-unknown-linux-musl # static x86_64 (uses zig)
cargo zigbuild --release --target aarch64-unknown-linux-musl # static arm64
# Copy the executable AND its --no-net shim together. build.rs rebuilds the
# shim whenever blocknet.c changes, but the executor looks for it beside the
# BINARY, so installing only the binary leaves the previous shim in place — and
# a stale shim silently enforces the old policy while every "is it there?"
# check still passes. Copy both or neither.
mkdir -p ../bin # bin/ is gitignored, so a fresh clone has none
cp target/release/codecalc-exec target/release/blocknet.so ../bin/Requires: Rust 1.97+, a C compiler for the --no-net shim (the build warns and
carries on without one; on macOS, or a Linux kernel without seccomp support,
--no-net then reports itself in unenforced rather than pretending — a
Linux kernel with seccomp support enforces it in-kernel either way), and
cargo-zigbuild
for the static cross-builds (zig is used as the linker; no x86_64 GCC needed).
Every session file is also exposed as an MCP resource:
codecalc://session/<session_id>/files/<path> — images render inline for the
model, text returns as text, other files download.
Graphical results in MCP Apps hosts: verify_translation and
verify_optimization also carry an MCP Apps
ui:// view (ui://verify-translation/view.html,
ui://verify-optimization/view.html) — a per-case diff table and a per-size
timing chart, respectively, rendered inline by a host that supports the
extension. Both are self-contained (inline CSS/JS, no network, no external
assets); a host without MCP Apps support sees exactly today's text/structured
result, unchanged.
Exact arithmetic & programmer-mode: exact rationals, threshold checks, bit analysis, binary64 introspection.
Tool | Description |
| EXACT arithmetic: |
| Exact threshold verdict with shortfall: |
| Exact share and percentage of PART/TOTAL (rationals accepted) |
| mean, median, sample stdev, CV (CV > 0.2 = noise swamps the effect) |
| p50/p90/p95/p99 by nearest-rank AND interpolation; warns n<100 |
| Birthday-bound hash collision: 1e5 items/32 bits ≈ 0.69, 1e6/64 ≈ 2.7e-8 |
| Byte sizes both ways: KiB/MiB (binary) AND KB/MB (decimal) |
| Humanised duration + per-day/per-30d rates |
| Epoch s/ms/µs/ns → ISO 8601 UTC, implausible readings suppressed |
| Any base 2..36, fractions included, non-termination flagged ( |
| What binary64 actually stores: exact value, raw bits, ULP, neighbours, representable-or-not |
| Programmer-mode integer facts and operations, selected by |
| Are |
| Symbolic algebra, selected by |
Core tools
Tool | Description |
| 31 languages with extension, compile flag, runtime availability |
| Execution-provider identity, interface version, host class, and machine-readable capabilities |
| Run code in any language → stdout/stderr/exit_code/verdict (OK/TLE/MLE/OLE/RTE)/cpu_ms/peak_memory_kb; per-call limits ( |
| Provider-selected execution using the same canonical limits as |
| python3 only. Runs the same sandboxed executor |
| python3 only. Decides, with z3, which if/elif/else arms and while/for(range, static bounds) loops in ONE function can ever be taken for ANY input — |
| Submit code for background execution; returns a |
| Poll a background run: status while running, the full |
| Cancel a background run; idempotent on an already-terminal run, honest about providers that cannot cancel mid-flight |
| Persistent session; python3/node get a stateful REPL worker (variables/imports persist across calls), other languages a workspace dir |
| Session lifecycle |
| Workspace file tools, jailed to the session dir; listings support |
| Multi-file programs: execute an entry file that imports other session files (helper.py, data/...) in the workspace |
| List files created by executed code (results, images, CSVs) |
| Archive a session's workspace to a snapshot stored OUTSIDE the jailed workspace ( |
| Install packages (uv pip/npm/gem/go/cargo...) into a session or shared cache |
| Prove a port is equivalent: you write the translation, the executor runs both versions on the same inputs and reports match / diverged / inconclusive per input. A pass is graded |
| Prove an optimisation: you write the candidate, the executor confirms it still agrees with the original AND times both — accepted only if equivalent and measurably faster. Accepted is graded |
| Pull a named function + its dependency closure (imports, referenced helpers) into a standalone program and run it (ast-exact for python3, best-effort elsewhere) |
| Run the same logic in N languages on edge-case inputs (empty, zero, negative, float precision) and flag behavioral divergence |
| Dimensional unit conversion via sympy: length, mass, time, speed, energy, power, force, pressure, temperature (°C/°F/K), volume, area, data, frequency |
| 22 physical constants with values (c, h, N_A, k_B, G, g, m_e, R, ...) |
| All 140+ unit aliases for convert_units |
| Symbolic math: |
| Boolean algebra: |
| SMT-LIB2 satisfiability + model. An |
| Structured matrix ops: det/inverse/eigenvalues/transpose/rank/trace on a |
| Static Big-O estimate from code structure, parsed with tree-sitter (every supported language). Reports |
| Empirical Big-O: runs code at increasing N, fits growth curve |
| Same code across N languages side-by-side |
| Non-mutating update check: current vs latest for every language runtime, which package manager owns it, and the command that would run |
| Update runtimes. Dry-run by default ( |
The 2026-09-08 merge (0.11.0) folded two lexically-overlapping clusters — flagged by Glama's public review as indistinguishable from a description alone — into one enum-selected tool each, keeping every mode's own parameters, annotations and result shape. The eight old names stayed registered as thin aliases for one minor release so nothing broke mid-upgrade, then were removed in 0.12.0 (CHANGELOG.md). Calling one of them now gets the MCP SDK's own unknown-tool error, not a result:
Retired name | Replacement |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
verify_translation, verify_optimization and z3_check return grade +
grade_basis (+ grade_rules_version) on top of their own result. The grade
names how strong the evidence for a success actually is; it is derived from
evidence those tools already emit, in codecalc/grades.py — the verifiers
never assign their own grade.
Grade | Means | Emitted by |
| Two independently authored programs were both actually run and their outputs agreed. |
|
| Z3 returned |
|
| Reserved: the claimed computation ran and produced the reported result, with no independent second opinion. Not currently emitted by any tool above — every one of them also clears the | — |
| Explicit non-grade for a mismatch, an inconclusive comparison, a rejected optimisation candidate, a measurement failure, a Z3 | any of the above, on a non-success |
z3_check's sat verdicts are graded ungraded, not solver_proven, even
though sat is just as decisive a verdict as unsat. The ticket's motivating
pattern is proving a property P by asserting not-P and checking unsat; a
caller running that pattern who gets sat back has learned P is FALSE, and
solver_proven on that result would let a reader who skims grade without
result mistake a counterexample for a proof. sat's grade_basis says so
explicitly: satisfiability was decided, but solver_proven is reserved for
unsat so a counterexample can never wear a proof grade. Widening sat back
into solver_proven later is additive; narrowing it after callers depend on
the wider behaviour would not be, so this ships narrow now. Full reasoning:
codecalc/grades.py's module docstring.
algebraic_equiv is deliberately NOT graded: it compares two expressions via
sympy.simplify(a - b) == 0, a CAS transformation rather than a decision
procedure with a checkable certificate, and it is one simplifier's opinion
rather than two independent implementations agreeing. None of the three
grades describes that evidence honestly.
Every language is mapped to its package manager, and codecalc can update its own runtimes:
Manager | Languages | Update command |
mise | python3, node, bun, deno, ruby, go, erlang, elixir, gleam, zig, java, kotlin, sqlite, duckdb, gradle |
|
rustup | rust (stable/nightly toolchains) |
|
swiftly | swift |
|
apt | c, c++, fortran, csharp, php, perl, lua, tcl, r, jq, bash, zsh |
|
npm | typescript/tsc |
|
uv | mojo |
|
nix | haskell (on-demand) | nothing persistent |
runtimes_status is always safe. update_runtimes refuses to mutate unless
apply=True is passed explicitly — and it only touches the package manager
that owns each language (never the Rust sandbox, which has no update powers).
One of those managers is elevated: apt updates system packages, so its command
starts with sudo. apply=True is an argument a connected model controls, so
that branch takes a second key the model does not have — the host must set
CODECALC_ALLOW_RUNTIME_APPLY=1. Without it the apt command is reported as
skipped with ok: false and the variable named, while the unprivileged managers
still run. sudo -n already fails closed where a password is required; this
covers the passwordless-sudo rule common on developer machines and CI images,
which is exactly where -n does not stop it.
cd /path/to/codecalc && .venv/bin/python -m codecalc.server
# stdio transport — register with any MCP client
# The identical tool/resource registry over stateless Streamable HTTP:
.venv/bin/python -m codecalc.server serve-http --host 127.0.0.1 --port 8000Streamable HTTP binds to loopback by default. Bearer-token auth
(CODECALC_HTTP_TOKEN) is required for any non-loopback bind — serve-http
refuses to start on a routable address if neither it nor --oauth-issuer (below)
is set, and the static-token comparison is constant-time — and optional on
loopback, where an MCP client spawning the process is already inside the trust
boundary. Setting a token does not change the single-operator threat model: put
an authenticating reverse proxy and the stronger process/container isolation
described in SECURITY.md in front of it before exposing it beyond one
operator's own machine.
For hosted use only, serve-http also accepts --oauth-issuer URL (or
CODECALC_OAUTH_ISSUER) as an alternative to the static token — off by
default; the static-token path above is unchanged when it is unset, and
setting the variable costs nothing outside serve-http itself: doctor,
--help, serve-strict, and the bare stdio server never touch the network
over it, only serve-http's own startup does. The issuer and JWKS URLs must
be https:// unless the host is loopback (for local testing); an issuer that
is plain http:// on a real host, or cannot be reached at all, fails
serve-http's startup outright with a message on stderr rather than starting
a server no token could ever pass. Given a reachable issuer, serve-http
validates each bearer token as a JWT against that issuer's own JWKS
(RS256/ES256; the JWKS URL is discovered once from
<issuer>/.well-known/openid-configuration, or pinned with
--oauth-jwks-url) and checks its issuer, audience, expiry, and
not-before. It also serves RFC 9728 Protected Resource Metadata at
/.well-known/oauth-protected-resource/mcp, and a request with a missing or
invalid token gets WWW-Authenticate: Bearer resource_metadata="..." pointing
at it, per the MCP authorization spec (2025-06-18 and later). --oauth-audience
defaults to this server's own resource URL; --oauth-scopes "s1 s2" requires
every named scope on the token, checked by the SDK's own auth middleware. If
both a static token and an issuer end up configured at once, the issuer
wins — a request bearing the static token's exact value is rejected like any
other invalid bearer value, and a warning naming both settings is printed to
stderr at startup. codecalc runs no /authorize or /token endpoint of its
own (it is a resource server only, never an authorization server), so there is
no dynamic client registration surface here either; a 2026-07-28-era client
that needs one uses a Client ID Metadata Document against its OWN
authorization server, not against codecalc.
Point an MCP client at it:
{ "mcpServers": { "codecalc": { "command": "/path/to/codecalc/.venv/bin/python",
"args": ["-m", "codecalc.server"],
"env": {
"PYTHONPATH": "/path/to/codecalc",
"CODECALC_RUNTIME_PATH": "/path/to/mise/shims:/usr/local/bin:/usr/bin:/bin"
} } } }Protocol revision 2026-07-28, on the official mcp SDK 2.0. Not fastmcp:
fastmcp 3.x pins mcp>=1.24,<2.0 and so cannot reach this revision at all.
Verifying that is less obvious than it looks. mcp.types.LATEST_PROTOCOL_VERSION
reads 2026-07-28 regardless of what a given connection negotiated, and the
same server answers on either protocol depending only on how you connect:
client | negotiated | cache hints |
|
| dropped |
|
| applied |
So tests/test_mcp_protocol.py asserts the negotiated value from a real
connection. The legacy path still works — backward compatibility is a feature —
it just must not be mistaken for the new protocol.
Worth noting for anyone reading the spec's headline change: 2026-07-28 removes
protocol-level sessions, and directs servers needing cross-call state to use
"explicit, server-minted handles passed as ordinary tool arguments". That is
exactly what codecalc's session_id already is.
Every result carries contract_version, currently 1.6.0. The published
schema is docs/contract/result-v1.schema.json
and the policy behind it — what MAJOR/MINOR/PATCH may change, the twelve-month
deprecation window, worked success/failure/timeout examples, and the migration
path from unversioned servers — is in
docs/contract/README.md.
For in-process Python use, the supported protocol-neutral service boundary—and
the session/storage internals that are deliberately not public—is documented in
docs/embedding.md.
Two things a caller should know before reading anything else:
ok means "ran and exited 0". A program that behaves exactly as intended
and exits 3 comes back ok: false, exit_code: 3, verdict: "RTE". To tell
a failed program from a failed request, read verdict — a request that
never reached a runtime has no verdict at all, and has a code instead.
code is the branch target, not error. Eight stable values; the prose
in error is free to improve and is not a contract. An unrecognised code
must be treated as internal — that is what lets a 1.x client survive a
2.0.0 server, though adding a code is still a MAJOR change, because the
published enum is closed and a strict validator rejects the result first.
Truncation reports a size, not just a flag. output_truncated says output
was cut; stdout_bytes / stderr_bytes say by how much — the bytes the
program actually produced, before the cap. A 200 000-character print under
max_output_kb=1 returns 1 039 bytes of stdout and stdout_bytes: 200001,
so a caller can size a retry instead of guessing. null there means not
measured (nothing ran); a program that printed nothing reports 0.
The schema is JSON Schema 2020-12 — the dialect MCP 2026-07-28 defaults tool
outputSchema to — so a client can validate our results with it directly.
scripts/check_contract.py regenerates it from codecalc/contract.py and fails
on a diff, and separately re-derives both backends' verdict vocabularies from
main.rs and executor.py: check_parity.py compares the two backends' key
sets and is structurally blind to a new verdict value, which would leave the
published enum short and make a strictly validating client reject a good result.
All optional. codecalc runs with none of these set.
Variable | Default | What it does |
| (unset) | Bearer token for the Streamable HTTP transport ( |
|
| What the HTTP transport's auth metadata advertises as its own URL. Only consulted when |
| (unset) | Same as |
| this server's own resource URL ( | Same as |
| (unset) — discovered from | Same as |
| (unset) | Same as |
| the server's own | The |
|
| Override the sandbox binary. Without one, codecalc falls back to a pure-Python executor — |
| (unset) | Fail-closed: refuse to start if no usable |
|
| Default execution-provider ID. Explicit |
| (unset) | Register the non-local open-source Piston v2 provider at this absolute HTTP(S) base URL. No public service is contacted by default. |
| (unset) | Exact value for Piston's |
| (unset) | Activate the current OS's |
| (unset) | Exact value for the strict service's |
|
| Durable metadata-only journal backing |
|
| Admission cap for |
| (unset) | Permit |
|
| Where session workspaces live. Keep this codecalc-private. |
|
| How old (and untouched) a marker-less, session-shaped directory must be before |
| (unset) | Deny-by-default allowlist for |
| (unset) | Idle-expiry for stateful (python3/node) session workers: a session untouched for longer than this is reaped — worker killed via the same teardown |
|
| Per-session ceiling on total workspace disk. |
|
| Global ceiling on disk summed across every session workspace on this host — closes the gap where staying under the per-session quota by opening many sessions would otherwise be unbounded. Same enforcement points and |
|
| Per-write size ceiling for anything a session write path creates — independent of the total quotas above, so one runaway file cannot hide under a generous session/global total. A WRITE-time cap; distinct from |
|
| Per-session ceiling on the number of artifact files — catches a session writing one byte at a time into thousands of tiny files, a shape no byte-sized cap alone bounds. Only a write that creates a NEW file is checked; overwriting an existing one always succeeds regardless of the count. |
|
| Refuse a session write when the HOST's free disk space drops below this — protects the host even when every quota above is generous, since a shared host can be driven low by something that is not a codecalc session at all. Measured with |
|
|
|
|
| Per-session ceiling on the number of snapshots kept at once — catches many small snapshots the byte cap alone would not, the same "count cap alongside the byte cap" shape |
| (unset) | Capability broker. Unset, no brokering — a job's capabilities run as requested (today's behaviour); the execution receipt still discloses them under |
|
| Append-only JSON-lines audit stream for broker decisions and security-relevant side effects (denied capability, refused install, cleanup). Each event carries a source-safe timestamp, the run/session id, the decision and reason, and never the executed source or a credential. Set to a path to relocate it; set empty to disable. Best effort — a write failure never fails a run. |
|
| Fork-bomb guard. |
| (unset) | Escape hatch: pin |
The strict service runs on Linux x86_64 or ARM64 with Docker Engine, cgroup v2,
and an explicitly registered gVisor runsc runtime. Its executor image must be
pinned by @sha256: digest on the execution path. That image is published to
GHCR (ghcr.io/the-40-thieves/codecalc-exec, multi-arch amd64+arm64) by the
publish-executor-image workflow, which an operator dispatches
(workflow_dispatch); the workflow commits the immutable digest into
docker/executor-image.lock, and published_strict_image() resolves it as the
production default. Until that first dispatch no digest is pinned and the
execution path fails closed — it never falls back to the mutable local
diagnostic tag (codecalc-exec:strict), which doctor and the conformance
suite keep using. The default systrap platform works without KVM, so the same
authenticated service can be used from Linux, macOS, and Windows; strict clients
never fall back to native local execution.
Provisioning and running any of the three strict backends in production —
the gVisor+Docker host, Windows AppContainer hardening, and the macOS/Windows
remote-client configuration — is covered in
docs/deployment/README.md, separate from the
provider interface itself in
docs/contract/provider-v1.md.
Both backends resolve CODECALC_RUNTIME_PATH identically, and
scripts/check_parity.py fails CI if the Rust and Python copies of that
contract ever drift — including if a machine-specific home directory finds its
way back into the default.
codecalc's tools/list returns 49 definitions. Measured with o200k_base as a
proxy on the served JSON, that is 78,586 bytes / 20,630 tokens of descriptions
and input schemas (up from 64,643 bytes / 17,277 tokens at the same 49 tools),
and every client pays it before the first user message. The number
has grown with the descriptions, not the count: the disambiguation sentences
and the per-mode text on bits/symbolic are what a selection-accuracy-first
server spends its tokens on. The latest jump (+13,943 bytes, +3,353 tokens) is
every one of the 152 tool PARAMETERS gaining its own description in the
input schema (Annotated[<type>, Field(description=...)]) — the docstrings
above did not change, so scripts/tool_select_eval.py's selection-accuracy
numbers (it scores only name + docstring, never the input schema) are
unaffected by this change.
A follow-up pass then edited the DOCSTRINGS themselves, now that every
parameter's own syntax/default/range lives in its schema description and no
longer needs restating in prose: measured with the same o200k_base proxy
(a mcp.Client.list_tools() dump, by_alias=True, exclude_none=True, one
tool per JSON object), 77,237 bytes / 20,439 tokens — down from the
152-parameter figure above (-1,576 bytes, -412 tokens), short of the
~19,000-token target this pass aimed for. What moved which way: the
execute_code/execute_code_stream/run_submit/session_run cluster and six
other tools (truth_table, session_files, algebraic_equiv,
compare_threshold, percentage, percentiles) each gained one
disambiguation/usage sentence naming a sibling tool, verify_optimization's
docstring was cut to about 60% of its length, and pure schema-restating
sentences ("languages = comma-separated subset", explicit default/range
call-outs, repeated operation lists) were removed across the file — but
several of THOSE removals had to be partly reverted once
scripts/tool_select_eval.py showed they deleted discriminating
vocabulary BM25 actually leans on (see scripts/data/tool_select_baseline.json,
regenerated by this pass: full 151→150/234, dev 128→130/184, core
79→78/116 top-1 hits, every surface within the eval's own
DEFAULT_EPSILON_HITS), which is most of why the net reduction is smaller
than the additions alone would suggest.
A per-tool icons field (2025-11-25+) was tried and measured, not assumed:
one tiny inline data:image/svg+xml;base64,... glyph per tool GROUP, under
300 bytes even for the largest of six — small per icon, but Tool.icons is a
per-TOOL field, so each of the tools repeats its group's full base64
payload on the wire, and base64 tokenizes far worse than prose under a BPE
encoder. Measured on the full served tools/list payload: +11,540 bytes,
+6,665 tokens (o200k_base) — real and non-trivial on a server whose whole
pitch (see "Reducing the tool surface" below and docs/design/ 2026-08-10-tool-facade.md) is that tool SELECTION accuracy matters more than
saving a few tokens elsewhere. Removed. codecalc's MCPServer still carries
one SERVER-level icon plus a website_url — both ride on initialize, once
per connection, not once per tool, so they do not touch tools/list
at all: measured before/after, the served tools/list payload is
byte-identical (59,902 bytes / 15,952 tokens either way, as measured at the
time on the then 52-tool surface) — +0 on the
number this section exists to track.
codecalc does not hide its tools behind a discovery facade, and that is
deliberate: the tool surface is where per-operation approval prompts, audit
names and typed schemas live, and collapsing 49 tools into one dispatcher makes
install_package and percentage look like the same permission to a client
that approves by tool name. The cost is real, but the client is the better place
to solve it, because the client can defer definitions without giving up the
schemas or the per-tool boundary.
If you are paying too much for codecalc's definitions:
Claude Code defers every MCP tool by default — tool search is on by
default, with no token floor codecalc needs to clear. auto loads a
server's tools upfront only while their definitions total under 10% of the
context window and defers all of them once that 10% is reached; false
loads everything upfront regardless of size (Claude Code MCP docs,
https://code.claude.com/docs/en/mcp, retrieved 2026-09-07). calc_exact,
execute_code, verify_translation, verify_optimization, and
list_languages carry _meta["anthropic/alwaysLoad"] (per that same doc,
"your 3-5 most frequently used tools") so they stay loaded even when a
client defers everything else; install_package and update_runtimes
carry _meta["anthropic/requiresUserInteraction"], which forces a
permission prompt on every call regardless of the session's permission
mode — both change the host and both fetch from a registry.
execute_code, execute_code_stream, session_run, compare_execution,
and run_inspect carry _meta["anthropic/maxResultSizeChars"] = 499520
(2 * 240 KiB + 8_000), the truncation hint for the one tool family whose
output can legitimately approach it — run_inspect carries it because
its terminal reply, once a run_submit-started run finishes, is the same
envelope execute_code returns. This bounds the serialized TEXT content
block only — the JSON result as the string a client renders as the tool's
reply — not the whole MCP response: every one of these five tools except
session_run also declares outputSchema, so the SDK additionally
attaches structuredContent with the same JSON ("MCP server developers
can configure custom output limits for individual tools by specifying
_meta['anthropic/maxResultSizeChars'] in the tool's listing, up to a
hard maximum of 500,000 characters", same doc as above, describes the
text result specifically) — so a typed tool's total wire payload
approaches twice this hint. session_run's inlined artifact content
blocks (image/text/link, up to 8 within a 4 MiB encoded budget — see its
own docstring) are likewise separate blocks outside this bound. 240 KiB
per stream is a hard CEILING max_output_kb is clamped to on every tool
that accepts it (execute_code, execute_code_stream, run_submit),
separate from the 64 KiB DEFAULT 0 selects — chosen as the largest
round-KiB ceiling that keeps the TEXT-block hint under that 500,000-char
limit; there is no other ceiling on that parameter today, and raising it
further would push the hint over that limit. A run whose real output
needs more than 240 KiB belongs in a session instead: leaving
max_output_kb at its default with session_id set spills oversized
output to a full-fidelity file, readable in full via session_read_file,
rather than truncating it. compare_execution takes no max_output_kb
of its own, but accepts an unbounded number of snippets, so a
many-language comparison can still legitimately exceed the hint.
Claude API, via the MCP connector, takes defer_loading once on the
toolset's default_config, or per tool in configs. Deferred definitions stay
out of the system-prompt prefix, prompt caching is preserved, and a matching
tool is expanded into its full definition when the model searches for it.
OpenAI's Responses API has the same knob under a different name:
defer_loading: true on an MCP server tool definition (OpenAI Responses MCP
tool guide, https://developers.openai.com/api/docs/guides/tools-connectors-mcp,
retrieved 2026-09-07).
VS Code caps a single chat request at 128 enabled tools and groups excess tools behind "virtual tools" above a configurable threshold (VS Code agent tools docs, dated 2026-09-02, https://code.visualstudio.com/docs/copilot/agents/agent-tools). Windsurf / Cascade caps at 100 total tools (Cascade MCP docs, https://docs.devin.ai/desktop/cascade/mcp, retrieved 2026-09-07).
The MCP specification itself has no deferral mechanism — no tool search,
grouping, tags, or toolsets; a server can only publish ttlMs/cacheScope
hints and paginate tools/list (MCP spec 2026-07-28,
https://modelcontextprotocol.io/specification/2026-07-28/server/tools,
retrieved 2026-09-07). A client without one of the mechanisms above pays the
full cost regardless of what codecalc does.
Any client can filter which of the 49 tools it exposes to the model. Nothing here requires codecalc to change.
A server-side facade remains under consideration for clients with no such
mechanism (docs/design/2026-08-10-tool-facade.md), and is not implemented.
Trimming a description to cut this cost is exactly the change
scripts/tool_select_eval.py exists to gate: an offline, labeled eval of
whether a deterministic lexical (BM25) selector still picks the right tool
for a plain-language ask, scored against the live tools/list text.
Measured v1 baseline (196 hand-labeled prompts, none containing their own
target tool's name — see the script's own docstring): 60.71% top-1 /
75.51% top-3 accuracy on the full surface (62.75% / 63.0% top-1 on dev /
core respectively). It is a lexical proxy, not a model — see the script's
module docstring for exactly what a green run does and does not prove.
The checked-in baseline PINS the exact labeled corpus by content hash
(prompt_set_sha256); a --baseline compare against a corpus that no
longer hashes to it fails with a distinct "corpus changed" error rather than
silently scoring a smaller, easier prompt set against the old numbers. And
because a tool can be top-1-wrong against full's 51 distractors (zero
headroom to lose) while still having real headroom against core's much
smaller distractor set, both the regression compare and the ablation
self-check (replacing real descriptions with a generic stub, one tool at a
time, across every candidate tool — no sampling) run separately against all
three of full/dev/core, wired into CI via
tests/test_tool_select_eval.py so the gate is proven live, on every
surface, on every run — not just at the PR that added it.
BM25 is a lexical proxy, not a model — scripts/tool_select_llm_eval.py is
the model-driven half, calling a real chat model over a live gateway with
the identical tool catalog and labeled corpus; it is opt-in (workflow_dispatch,
advisory rather than a hard gate) rather than wired into every PR, and its
measured numbers live in docs/tool-selection-eval.md next to BM25's own.
For an operator who would rather not configure every client, codecalc also has
a first-party knob: CODECALC_TOOLS registers only a chosen slice of the
56-tool surface, so a client that never enables tool search still pays for a
smaller tools/list.
On a client with no deferral mechanism of its own, the client's own allow-list
does the same job from the other end — OpenAI's allowed_tools, Gemini CLI's
includeTools/excludeTools, or Codex CLI's enabled_tools/disabled_tools
all narrow what a given session sees without touching the server.
Every tool also now carries a ToolAnnotations hint (readOnlyHint,
destructiveHint, idempotentHint, openWorldHint — see
codecalc/server.py's GROUP_ANNOTATIONS/TOOL_ANNOTATION_OVERRIDES tables
for the value on each of the 49). Codex CLI's writes approval mode
(v0.144.0+) reads readOnlyHint directly: a tool marked readOnlyHint: true
skips the approval prompt, everything else still asks. That covers the whole
calculator group (19/19 pure) plus the read-only members of the mixed
groups — list_languages/list_execution_providers/runtimes_status/
branch_reachability in execution, z3_check/algebraic_equiv in
verification,
session_list/session_files/session_read_file/session_artifacts/
run_inspect in sessions, and analyze_complexity in analysis — without
codecalc doing anything client-specific; the annotation is the same hint
every MCP client reads, writes just happens to be the mode that consumes it.
This is not the facade the section above declines to build. Every tool a
group activates keeps its own name, its own typed input schema and its own
per-tool approval prompt — a group that is not active simply never registers
its tools with the MCP SDK at all, so they are absent from tools/list and
rejected by tools/call, not merely hidden behind a dispatcher a client could
still invoke by guessing the name.
Every tool belongs to exactly one group:
Group | Tools |
|
|
|
|
|
|
|
|
|
|
|
|
CODECALC_TOOLS takes a comma-separated list of group names, preset names, or
both:
Preset | Expands to |
|
|
|
|
| every group (the default) |
CODECALC_TOOLS=calculator # just the calculator (19 tools)
CODECALC_TOOLS=core # same thing, by preset name
CODECALC_TOOLS=calculator,execution # two groups, unioned
CODECALC_TOOLS=dev # a coding-assistant slice (35 tools)Unset or empty registers every group — 49 tools, same as today —
so nothing changes for an operator who does not set this. An unknown group or
preset name is a loud startup failure naming the bad value and every known
group/preset, never a silent fallback to "everything" or "nothing": either
direction would turn a typo into a footgun nobody notices until it matters.
codecalc doctor prints the active groups, the full group→tools mapping, and
how many tools this process actually registered, whatever CODECALC_TOOLS is
set to.
Client-side deferred loading (the section above) and this env var compose
cleanly: point a client with no deferred-loading mechanism at a
CODECALC_TOOLS-restricted process, or use both — a smaller declared surface
still benefits from being deferred.
Each file is a standalone script that prints one PASS/FAIL line per
assertion and exits non-zero if any failed — no test runner, no plugins.
cd /path/to/codecalc
# everything. `|| break` used to be `|| break` alone, which stopped at the
# first failure AND left the loop exiting 0 — a red suite reported success to
# anything wrapping this command. This form runs them all and carries the
# failure out.
fail=0
for f in tests/test_*.py; do PYTHONPATH=. .venv/bin/python "$f" || { echo "FAILED: $f"; fail=1; }; done
for f in scripts/*.py; do PYTHONPATH=. .venv/bin/python "$f" || { echo "FAILED: $f"; fail=1; }; done
[ "$fail" -eq 0 ] # the exit status of the whole run
# or individually
PYTHONPATH=. .venv/bin/python tests/test_smoke.py # every language, via the Rust executor
PYTHONPATH=. .venv/bin/python tests/test_mcp_all.py # every tool over MCP stdio, answers checked
PYTHONPATH=. .venv/bin/python tests/test_executor_sweep.py # sandbox regressions71 test files and 19 CI-invoked scripts, 2184 assertions. "CI-invoked"
means referenced by path (scripts/<name>.py) from a job in
.github/workflows/*.yml — scripts/check_claims.py derives the count that
way and gates it, so a script wired into a workflow without this sentence
changing, or this sentence bumped without a workflow change, fails the build.
Nothing in the suite
needs the internet, so none of it is ever skipped for lack of a network.
It can skip for lack of a capability, and that is correct rather than a regression: a machine without a symlink privilege, without a given language runtime, or without a built native executor cannot exercise the cases that need them. The suite reports three distinct outcomes — the property holds, the property is broken, and this machine cannot exercise it — and every skip names its real cause. A nonzero skip count on Windows or in fallback mode is the healthy result; what would be wrong is a skip reading as a pass.
This paragraph previously claimed zero skips unconditionally. That became
false the moment the suite learned to distinguish the third outcome, and
nothing gated it: check_claims.py gates the counts below, not the prose
around them. The counts are gated by
scripts/check_claims.py: they were written by hand once and were stale within
three pull requests, which is exactly the failure the rest of that script
exists to prevent. Four of the files are regression suites named after the
sweep that produced them — test_bug_sweep, test_executor_sweep,
test_python_sweep, test_network_modules — and each one's docstring states
the defect it locks out and how it was reproduced, because a regression test
whose reason has been forgotten is the first one deleted.
Two rules the suite holds itself to, learned from breaking both:
Assert the value, not the shape. Three of these files once had no
assertions at all: they called tools, printed the output and exited 0. They
caught a crash and never a wrong answer — a runtimes_status total replaced
with -999 passed, printing total = -999.
Don't pin what varies. benchmark and compare_execution rank by
measured time, so their winner moves under load; their structure is asserted
and their timing is not. runtimes_status is checked against itself — the
summary must agree with the data it summarises — so it holds on any machine
rather than describing this one.
Linux, macOS and Windows. The three do not offer the same primitives, and the
executor reports which ones it could not apply in an unenforced array on
every result rather than letting a caller assume they all held.
The native table below describes the local provider and is not a hostile-code
security boundary. On macOS, <host>-strict instead uses the explicitly
configured Linux strict service: the macOS binary performs provider selection,
attestation, supervision, and result validation, while untrusted code executes
inside the remote cgroup/namespace/seccomp/Landlock boundary. A missing or
incomplete service fails before source leaves the Mac and never falls back to
native execution.
Symbolic evaluation carries the same idea. Every symbolic tool runs SymPy in
a forked child under CPU and memory ceilings with a wall clock the parent
enforces, so an expression nobody anticipated is still bounded — SymPy's own
maintainers abandoned their attempt at a safe= flag as "security theater", so
the screen in safe_expr.py buys time and the child buys the bound. Where
there is no fork, the result reports expression_bound_not_enforced_without_fork
rather than implying a guarantee.
A second field, output_error, covers the other way a result can be wrong:
absent means stdout/stderr are what the program produced, present means at
least one of them is not, and names which stream and the OS error. That
distinction did not exist until #80
— an output file that could not be read came back as a program that printed
nothing, on a run reported as successful. ok now accounts for it on both
backends.
Guarantee | Linux | macOS | Windows |
Wall-clock timeout | yes | yes | yes |
Kill the whole process tree |
|
|
|
Fork-bomb guard |
|
| Job |
Memory ceiling |
| reported unenforced¹ | Job |
CPU-time ceiling |
|
| Job |
Open-file ceiling |
|
| reported unenforced |
Output cap | yes | yes | yes (on read) |
| seccomp-bpf filter⁶ (falls back to |
| reported unenforced |
Stateful sessions | yes | yes | yes |
¹ Darwin accepts setrlimit(RLIMIT_AS) but does not enforce address space the
way Linux does, so setting it would buy an illusion.
² Dynamically-linked programs only — a statically linked binary (Go, by default)
ignores it.
⁴ Applied via JOB_OBJECT_LIMIT_PROCESS_TIME, which Windows has supported
since XP — this was reported as cpu_limit_unavailable_on_windows until
2026-08-08, and the table said the same, so code and docs agreed with each other
and disagreed with Windows. It is not identical to RLIMIT_CPU and the
difference is reported rather than glossed: it counts user-mode time only, so
a process burning kernel time is not capped by it, and the system checks
periodically rather than immediately. Runs on Windows carry
cpu_limit_counts_user_time_only_on_windows in unenforced to say so.
³ Weaker still on macOS, in two ways. SIP and the hardened runtime strip
DYLD_INSERT_LIBRARIES for protected and hardened-signed binaries (most signed
interpreters), and dyld interposing does not reach calls made inside the
shared cache where libSystem lives — a program's own connect() is intercepted,
a system framework opening a connection internally is not. Treat macOS no_net
as a speed bump, never as isolation.
⁶ Linux only. The executor installs a seccomp-bpf filter in the sandboxed
child that refuses the socket(AF_INET/AF_INET6) SYSCALL in-kernel — not a
libc symbol, so ctypes/dlsym and raw syscall() calls cannot route around
it the way they can around the LD_PRELOAD shim. AF_UNIX still works. Falls
back to the shim (with its symbol-level bypass, disclosed in unenforced as
no_net_best_effort_shim) when the kernel refuses the filter.
Both are exercised by the suite on every platform. The fork-bomb probe measures
the EAGAIN boundary precisely but needs os.fork, so it is POSIX-only; a second
probe SPAWNS processes instead, which is the portable operation, and pins the
ceiling low through CODECALC_MAX_PROCESSES so it costs two dozen short-lived
processes rather than walking up to the fallback. Verified to track the limit
rather than something incidental: a headroom of 24 bounds it at 22 children and
a headroom of 300 bounds it at 298.
⁵ Windows' ActiveProcessLimit is scoped to the job rather than to the uid,
so it avoids the failure mode that broke 14 of 31 runtimes on Linux. CodeCalc
now supplies that job at process creation, makes it non-nestable with the
minimal JOB_OBJECT_UILIMIT_EXITWINDOWS restriction, and allowlists only the
three standard I/O handles inherited by the child.
Measured on Windows 11 Pro: 400 of 400 spawns succeeded against a ceiling of
24, reproduced from two unrelated launchers including Task Scheduler. This is
not a failed API call — SetInformationJobObject and AssignProcessToJobObject
both return success and the correct limit reaches the job. It is topology.
ActiveProcessLimit is not one of the limits combined across a nested job
chain; those take the most restrictive value, while this one comes from the
process's immediate job. A post-creation AssignProcessToJobObject places
the child somewhere in that chain rather than at its end: measured, the child's
immediate job reported 0x3000 / APL 0 while codecalc's reported
0x230A / APL 24, so codecalc's ceiling was never consulted.
No parent-side Win32 call returns another process's immediate job or its
effective ActiveProcessLimit, so this cannot be closed by inspection. Every
compatibility run that assigns the child after creation therefore carries
process_limit_enforcement_unverified_on_windows in unenforced. Four further
strings can each positively prove a failure; none can prove success, so their
silence does not imply enforcement.
Creation-time assignment is the default. It was verified on Windows 11 Pro with
a direct Python runtime: 23 children succeeded against a total limit of 24 and
the next spawn failed with WinError 1816. Runtime launchers that require an
inner job now fail rather than silently escaping the limit; configure a direct
runtime executable. CODECALC_WIN_JOB_AT_CREATION=0 retains the old path only
as an explicitly unverified compatibility escape hatch.
AppContainer security isolation is a DIFFERENT guarantee from the Job Object's
resource limits. The Job Object above caps resources — memory, process count,
user-mode CPU — and each run names in unenforced which of those did not bind.
The optional AppContainer backend adds a security boundary layered on the same
creation-time topology: a least-privilege AppContainer profile
(CreateAppContainerProfile, no capability SIDs, so no network), launched with
SECURITY_CAPABILITIES in the same STARTUPINFOEX attribute list as the job
assignment. Access is granted two ways, deliberately split. The sandbox
workdir is granted to the run's own AppContainer SID — per-run, so
concurrent runs cannot reach each other's workdirs, and it vanishes with the
ephemeral directory. The interpreter directory is granted read+execute to the
fixed ALL APPLICATION PACKAGES SID (S-1-15-2-1) as an explicit,
non-inheritable ACE applied per file across the tree — because a real
interpreter's pre-existing files are inheritance-protected and no inheritable
grant reaches them. That interpreter grant is persistent and cached (a marker
in codecalc's own state dir; the several-thousand-file walk runs once per
interpreter): a deliberate trade-off that leaves a read-only ACE, readable by any
AppContainer on the machine, on a public interpreter — rather than re-walking
every run. The intended property is that a payload cannot read the user profile,
write outside its workdir, or reach the network. It is OFF by default (opt in
with CODECALC_WIN_APPCONTAINER=1) and fails closed — if profile creation,
SID derivation or an ACL grant fails, the launch is refused rather than dropped
to an unconfined process. The isolation has been verified on a Windows 11 box
(AppContainer SID present, user-profile secrets unreadable, writes confined to
the workdir, network denied, ambient privileges reduced to the two benign ones
Windows keeps), yet every run that takes this path still emits
appcontainer_isolation_unverified_on_windows: a Server-SKU CI runner cannot
exhibit AppContainer behaviour, and the guarantee ultimately depends on the
deployment's OS and configuration, so the shipped default stays conservatively
disclosed rather than claiming a universal proof.
Two things degrade rather than fail on a given platform: languages whose runtime
is absent (list_languages reports available: false), and the shell-wrapped
plans — gleam and haskell — which need a POSIX shell to scaffold a project
and report available: false on Windows outright rather than resolving through
a bash that cannot run them. csharp left that set: .NET 10 runs a single .cs
file directly, so it is shell-free on every platform.
available/status above is a claim about resolution: did this machine
find the command on PATH. It is not a claim about reliability: has
codecalc's own CI ever actually run this language and checked the output. The
two are orthogonal, and they disagree in practice — a review's own smoke test
found the rust and csharp host toolchains failing on a machine where both
rustc and dotnet resolved cleanly. list_languages, runtimes_status, and
codecalc doctor all report a tier alongside resolution to make that gap
visible instead of silent:
Tier | Meaning |
| A CI job genuinely executes this language and asserts on its real output, on every PR. Currently |
| Declared, with a local smoke fixture ( |
| A registry entry never validated on any runner, anywhere, not even locally. None today. |
codecalc doctor's text output prints both axes side by side rather than
folding tier into the resolution summary, so a best_effort runtime that
happens to be installed on your machine reads as exactly what it is:
resolved, unverified by codecalc, may be broken.
Fresh temp dir per run, deleted on exit (source + binaries + outputs). The
deletion is identity-checked: the directory's device and inode are
recorded at creation and re-checked before removal, because executed code runs
with that directory as its cwd and can rename another one into its place. A
caller-supplied --workdir is a session workspace and is never deleted.
If the filesystem supplies no file index to identify the directory by, the
deletion is refused rather than performed unverified, so temp directories
accumulate there instead of the wrong one being removed. That trade is stated
because it is the one this guarantee actually makes: it was previously
implemented in the Rust executor only, and the Python fallback deleted
unconditionally, which CI caught on Windows.
rlimits: CPU (timeout+8s), address space 2TiB (V8/JVM need huge VA), file size 256MiB, 256 FDs, core dumps off
The timeout is a total budget: compile and run share it, so --timeout 10
cannot take twenty seconds. duration_ms is the run alone; compile_ms and
total_ms are reported separately.
Wall-clock timeout kills the whole process group (SIGKILL). So does
SIGTERM to the executor — PR_SET_PDEATHSIG reaches only the direct child, so
a group kill is what covers its descendants, and the executor is the only
participant that knows the group id.
Output capped at 64KiB per stream, on every path including stateful sessions.
Exceeding it is reported as OLE, and the file-size rlimit is kept strictly
above the cap so that overflow stays detectable — tying the two together
turned a truncated 4MB output into a silent verdict: OK.
Fork-bomb guard via RLIMIT_NPROC, sized from the measured ambient task
count plus headroom rather than a fixed number. This is a mitigation, not
isolation: the budget is shared with every other process your user owns, so
concurrent executions draw on the same pool. cgroup v2 pids.max is the real
per-sandbox answer and needs delegated cgroup access a stdio MCP server cannot
assume — reach for it when this moves behind a container.
no_net blocks the network, not every socket: it refuses AF_INET and
AF_INET6 and forwards everything else, so AF_UNIX local IPC keeps working.
No network namespace isolation (single-host tool; containerize for untrusted code).
Note, 2026-09-07: this bullet describes the pre-#242 state. Since
#242 (2026-08-21),
Linux additionally enforces no_net in-kernel via a seccomp-bpf filter —
see the no_net row in SECURITY.md's "Known limitations" table for the
current per-platform breakdown. A full network namespace is still only the
strict (gVisor) backend's job; this note does not change that.
Every result carries a backend field ("rust" or "python") so a caller
never has to infer which sandbox actually ran from an absent key — that was
possible to confuse with an older build that never reported it at all. The
pure-Python fallback cannot provide everything above: it has no no_net
shim (reported in unenforced, not silently dropped), and peak_memory_kb
comes back None rather than a number, because ru_maxrss is a
process-lifetime high-water mark this path has no way to attribute to one
run. CODECALC_REQUIRE_NATIVE=1 turns "running on the fallback" into a
startup failure instead of a guarantee you have to notice was quietly
weaker.
A session is a persistent workspace; python3 and node additionally get a
long-lived REPL worker so variables and imports survive between calls. What that
does and does not buy you:
workspace session | stateful worker | |
Fresh sandboxed process per call | yes | no — one worker serves every call |
| applied | reported in |
| per call | applied once, at worker start |
Output cap + | yes | yes |
Per-call wall clock | yes | yes — a worker that blows it is killed |
A worker cannot take a per-call rlimit after the fact, and --no-net is decided
at exec time. Rather than accept those arguments and drop them, the result lists
them in unenforced — the same field the executor already uses to say "asked
for, not applied". Omit session_id, or use a workspace session, when a ceiling
has to be real.
The worker protocol does not share a file descriptor with executed code, and
every response carries the id of the request it answers. Both matter: sys.stdout
is a Python-level rebind that a subprocess writes straight past, and a corrupted
stream that is not resynchronised returns every later call the previous call's
result — a well-formed answer to a different question.
The channel differs by platform and the guarantee does not. POSIX hands the
worker an out-of-band pipe; Windows has neither pass_fds nor preexec_fn, so
the worker appends responses to a file whose path arrives in the environment.
Either way a child spawned with inherited stdio writes to fd 1 and cannot reach
the protocol. Tests force the file-backed channel on every platform, because an
unexercised fallback is one that works until it is needed.
Two CLI-only commands for an operator running a long-lived server, not MCP tools — they don't count toward the tool surface above:
codecalc status # read-only snapshot: sessions, disk usage, quotas, audit log
codecalc status --json # the same report, for scripts
codecalc cleanup # DRY RUN (the default) — marker-based only
codecalc cleanup --write # actually removes marker-based candidates
codecalc cleanup --write --include-unmarked # ALSO sweep old, unmarked, session-shaped dirsstatus reports SESSION_ROOT, how many sessions exist and which of them
are idle-expired (the on-disk .codecalc-session-expired marker the
idle-TTL reaping leaves behind), per-session and global workspace disk
usage, the configured disk quotas and current headroom, the audit log's
path and size, and a one-line runtime reliability-tier summary. It changes
nothing — no session is started, stopped, reaped, or written to.
cleanup reclaims disk from session directories under SESSION_ROOT.
--dry-run is the default — nothing is removed until --write is passed.
Because cleanup runs as a SEPARATE process from any server that may be
using SESSION_ROOT right now, it has none of that server's in-memory
bookkeeping to consult — only what is on disk.
Directory mtime is deliberately NOT trusted as a liveness signal. An
earlier version of this feature did trust it, and an adversarial review
proved that wrong live: a REPL worker doing purely in-memory work touches
no file at all, and even an in-place file overwrite bumps only that file's
own mtime, never its parent directory's — so a genuinely-active worker
session can look, by directory mtime alone, identical to an abandoned one.
The real signal is a per-worker-session liveness lockfile: the server
writes its own pid into the session directory the moment a stateful
(python3/node) worker starts, and removes it the moment that worker is
actually gone (reaped or session_stop). cleanup checks this for every
candidate and refuses outright — regardless of marker, age, or the mtime
floor below — whenever the lockfile names a pid that is still alive. That
is what makes a session any running codecalc server is using is never
deleted true for worker sessions.
By default, cleanup considers ONLY directories carrying the idle-expiry
marker — the risk-free path, since a marker only ever exists after
sessions.py's own idle-TTL reaper has already closed that specific worker
for good (session ids are never reused). --include-unmarked additionally
sweeps old (CODECALC_CLEANUP_ABANDONED_AGE_HOURS, default 24h),
session-shaped, marker-less directories — the one path with real residual
risk, because a workspace-only session (no worker) never gets a
lockfile to check against, so this path falls back to age + a hard
recency floor (nothing modified in the last few minutes is ever touched)
as a heuristic, not a proof. Turn it on deliberately, and never point
CODECALC_SESSION_ROOT at anything but a codecalc-private directory —
the "looks like a session dir" name filter is loose, not strict.
Other safety properties, unconditional on every path: only a direct child
of SESSION_ROOT is ever a candidate (never SESSION_ROOT itself); a
symlink there is refused, never followed; and removal itself is
identity-checked (device/inode, re-verified immediately before the
delete) the same way session_stop's own workspace teardown is, so a
directory swapped out from under a stale scan is refused rather than
deleted.
python3, node, bun, deno, typescript, ruby, php, perl, lua, tcl, r, elixir, erlang, bash, zsh, mojo, swift, c, cpp/c++, rust, go, fortran, zig, java, kotlin, csharp, gleam, haskell, sqlite, jq, awk — 31 runtimes.
codecalc does not install any of them. It runs whatever is already on
CODECALC_RUNTIME_PATH, and list_languages probes each one and reports which
actually resolved, so a minimal machine degrades to the subset it has rather
than failing opaquely.
Java uses single-file source launch (JEP 330). Kotlin compiles to a jar.
gleam/haskell scaffold a temp project (gleam new / nix-shell); csharp runs the file directly (.NET 10 file-based apps).
benchmark uses the stdin-N contract: code reads N from stdin, work sized by N.
Five workflows, each documented inline with what it gates and — where a tool was considered and rejected — why it is not there.
Workflow | Gates |
| clippy |
|
|
|
|
|
|
|
|
Two conventions run through all of them, both borrowed from harder-won experience:
Actions are pinned by commit SHA and downloaded tools by SHA-256. A tag is mutable; a digest is not.
Every scan asserts it scanned something. A linter pointed at a renamed directory, a dependency scanner with no lockfile to read, and a clean repo all produce the same output — exit 0. Each gate counts its inputs first and fails if the count is implausible.
Apache-2.0. See LICENSE.
Contributions require a DCO sign-off (git commit -s); dco.yml enforces it.
Are two expressions algebraically identical? 'is (ab)/c the same as a(b/c)?' answered exactly. Use symbolic(op="simplify"), not this, to see one expression's own simplified/factored/expanded forms rather than compare two; use verify_translation to compare running PROGRAMS, not expressions. Caveat: symbolic identity says nothing about float rounding, integer truncation or modular overflow.
| Name | Required | Description | Default |
|---|---|---|---|
| a | Yes | First symbolic expression to compare for algebraic identity | |
| b | Yes | Second symbolic expression to compare for algebraic identity |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint, idempotentHint, and destructiveHint, so the description's burden is reduced. It adds a meaningful behavioral caveat: the tool answers based on exact symbolic identity and says nothing about numeric execution semantics such as floating-point rounding, integer truncation, or modular overflow.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded with the core purpose. Every sentence earns its place: the first gives the exact question and example, the second routes to siblings, and the third provides an important correctness caveat. There is no filler or restatement of the title.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter, read-only comparison tool with full schema coverage and an output schema, the description is complete enough. It gives the intended use, differentiates against the most likely sibling tools, and flags the main limitation the agent needs to know before relying on the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for both parameters, so the baseline is 3. The description adds context about comparing two expressions and gives a usage example, but it doesn't define a specific expression grammar or syntax beyond what the schema already indicates.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a precise purpose—testing whether two expressions are algebraically identical—and grounds it with a concrete example. It also names sibling tools for closely related but different tasks, allowing an agent to disambiguate without opening each schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit routing: use symbolic(op="simplify") when dealing with a single expression's simplified/factored/expanded forms rather than a comparison, and use verify_translation when comparing running programs. The caveat about float rounding, integer truncation, and modular overflow also tells the agent when not to rely on the result.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Estimate the asymptotic (Big-O) time complexity of a code snippet via structural analysis.
| Name | Required | Description | Default |
|---|---|---|---|
| code | Yes | Source code snippet to analyze structurally for its asymptotic time complexity | |
| language | No | Language `code` is written in; default 'python3' | python3 |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With annotations already indicating readOnlyHint, openWorldHint=false, and idempotentHint, the description adds value by stating 'structural analysis', implying the code is not executed. This goes beyond the annotations and helps an agent understand the tool's non-executing behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, compact sentence with no filler. It front-loads the core purpose and the method in a way an agent can parse immediately.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the annotations, full schema parameter documentation, and the presence of an output schema, the description is sufficient for a tool of this complexity. It could mention limitations or rough heuristics, but 'estimate' already signals approximate results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the parameters are already well documented. The description only reinforces the 'code snippet' nature and does not add significant new meaning beyond the schema, placing it at the baseline for this dimension.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Estimate') and a specific object ('asymptotic (Big-O) time complexity'), and clarifies the method ('via structural analysis'). This clearly distinguishes it from execution or benchmarking siblings like execute_code and compare_execution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description makes clear that the tool is for static, structural complexity estimation rather than runtime measurement. It provides useful context for when to choose it over execution-based tools, though it does not explicitly name alternatives or edge cases where it should not be used.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Empirically measure time complexity by running code at increasing input sizes.
Contract: the code must read an integer N from stdin (first line) and do work
sized by N. codecalc runs it at each size in sizes and fits the growth
curve to estimate Big-O (O(1), O(log n), O(n), O(n log n), O(n^2)...).
Example python: 'import sys\nn=int(sys.stdin.readline()); s=0\nfor i in range(n): s+=i\nprint(s)'
| Name | Required | Description | Default |
|---|---|---|---|
| code | Yes | Program that reads integer N from stdin's first line and does work sized by N | |
| sizes | No | Comma-separated input sizes to run at, e.g. '100,1000,10000,100000' | 100,1000,10000,100000 |
| timeout | No | Wall-clock seconds allowed per size before that run is killed | |
| language | No | Language `code` is written in; default 'python3' | python3 |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that user-supplied code is executed repeatedly at increasing input sizes and that results are fit to a growth curve. This goes beyond the annotations and gives the agent a clear model of what happens. It could mention side-effect risks of executing arbitrary code, but the annotations already flag non-readOnly, and the contract is explicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact, front-loaded with the main purpose, and every sentence earns its place. The contract is stated crisply and the example is short but highly informative. No fluff or repeated schema content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description does not need to explain return values. It covers the core contract, the execution model, the fitting behavior, and provides a concrete example. This is fully sufficient for an agent to invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3, but the description adds real value beyond the schema by defining the stdin contract (read integer N from first line) and providing a working Python example. This clarifies exactly what 'code' must do, which the schema only implies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource ('Empirically measure time complexity by running code at increasing input sizes') and clarifies the output (Big-O estimate). It clearly distinguishes this tool from static analysis or other execution tools by emphasizing empirical measurement via runs at multiple sizes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states the intended use case clearly: measure empirical time complexity and estimate Big-O from actual runs. It gives a concrete contract and example, so an agent knows when to choose it. It does not explicitly name alternatives or exclusions, but the context is strong enough to route correctly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Programmer-mode integer facts and operations, selected by mode —
replaces the four former standalone tools bit_analysis, bitop,
int_widths and base_repr, retired in 0.12.0 (CHANGELOG.md). Every mode
returns exactly its former tool's own result, plus mode (additive).
mode="analysis" (was bit_analysis) — facts about a single N: popcount,
bit length, trailing zeros, power-of-two check, next power of two. Used
by this mode: n (required), align (optional).
mode="op" (was bitop) — combine two integers a/b with
and/or/xor/nand/nor/xnor/not/shl/shr/sar/rol/ror at a fixed width
(8/16/32/64). Every result shows unsigned,
signed (two's complement), hex, octal and binary. shr is logical
(zero-fill); sar is arithmetic (sign-propagating) — 0x80 shr 1 = 0x40
(+64) but 0x80 sar 1 = 0xC0 (-64); rol/ror rotate bits around the width
instead of shifting them out. A left shift that drops bits says
OVERFLOW and shows the unbounded answer. Used by this mode: a, op
(required), b (required unless op="not"), width (optional).
mode="widths" (was int_widths) — which widths (i8..i64/u8..u64) hold
n, and the wrapped value where they do not; flags anything past 2^53
as unable to round-trip through a JS number or JSON float. Used by this
mode: n (required).
mode="repr" (was base_repr) — hex/oct/bin of n; with width, two's
complement and signed-overflow detection. Used by this mode: n
(required), width (optional).
| Name | Required | Description | Default |
|---|---|---|---|
| a | No | First operand for mode='op'; required by that mode | |
| b | No | Second operand for mode='op'; required unless op='not' | |
| n | No | The integer to inspect; required by modes 'analysis', 'widths', and 'repr' | |
| op | No | Bit operation for mode='op': and/or/xor/nand/nor/xnor/not/shl/shr/sar/rol/ror | |
| mode | Yes | Which fact/operation to compute: 'analysis', 'op', 'widths', or 'repr' (each has its own required params) | |
| align | No | Alignment boundary for mode='analysis'; reports padding needed to reach it | |
| width | No | Bit width for mode='op' (8/16/32/64, default 64) or mode='repr' (omit to skip width analysis) |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes well beyond the readOnlyHint/idempotentHint/destructiveHint annotations. It discloses output shape ('plus mode'), result formats (unsigned/signed/two's complement/hex/octal/binary), shift semantics (logical shr vs arithmetic sar), overflow behavior, and the 2^53 round-trip caveat. It thoroughly explains observable behavior without contradicting the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long because the tool unions four former tools, but every sentence earns its place. It is front-loaded with the tool's purpose and follows a consistent mode-by-mode structure, closing with concise examples that clarify subtle shift behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a seven-parameter, multi-mode tool, the description is complete. It identifies all modes, their former names, their relevant parameters, the output shape, overflow and wrapping behavior, and precision caveats. The output schema's presence plus these details cover what an agent actually needs to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although the schema already covers all parameters well, the description adds mode-dependent semantics: which parameters each mode requires, the acceptable widths (8/16/32/64 default 64), and the b-optional-when-op='not' exception. These constraints are not fully inferable from the schema alone, so the description adds real value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states exactly what the tool computes: programmer-mode integer facts and operations selected by mode, and it enumerates the four modes with concrete behaviors and their former tool names. This gives an agent a clear, discriminable purpose and distinguishes the tool from broader sibling tools like radix_convert or float_repr.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Each mode is given a distinct use case and its own parameter requirements, which effectively tells the agent when to use mode='analysis', 'op', 'widths', or 'repr'. It does not, however, explicitly contrast the tool with sibling alternatives such as radix_convert or calc_exact, so it stops short of fully explicit when-to-use-when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Which if/elif/else arms and while/for loops of this python3 function can ever run, which are dead code, and what inputs reach each — decided with z3, without running the program.
Use trace_execution instead to see what happened on one run. Use z3_check, not this, when you already have an SMT-LIB2 script to solve directly rather than Python source to translate.
Unannotated parameters default to int. Each branch reports
verdict (reachable/dead/unknown), a witness when reachable, and
boundary_inputs (min/max/equality-edge for each comparison in its own
guard) — every input dict is shaped to drop straight into
compare_edge_cases's test_inputs. Refuses, naming the construct and
line, prior to any z3 call: floats, attribute access, comprehensions,
try/except, imports, data-dependent loop bounds, and anything else
outside + - * // %, and/or/not, == != < <= > >=, and
abs/min/max/len on int/bool/str. A for loop of at most 32
iterations is unrolled exactly; a longer for, or a while, is
checked one iteration at a time — a branch can still come back
reachable there, but never dead, and anything past the loop that
depends on what it computed comes back unknown rather than a guess.
| Name | Required | Description | Default |
|---|---|---|---|
| code | Yes | Python3 function source to analyze for reachable/dead branches | |
| inputs | No | Parameter name -> 'int'/'bool'/'str', to narrow or override an unannotated parameter's inferred type | |
| timeout | No | Wall-clock seconds before the z3 solver call is abandoned | |
| language | Yes | Source language of `code`; only python3 functions are analyzed | |
| max_branches | No | Max branches to analyze before stopping; default 64 |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnly/idempotent annotations, the description discloses substantial non-obvious behavior: it never runs the program, refuses unsupported constructs before any z3 call, unrolls short for-loops exactly, treats long for-loops and while-loops one iteration at a time, and returns unknown rather than guessing for dependent code after such loops. This is far more than the annotations alone provide, and nothing contradicts them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Although long, the description is densely informative with no filler. It front-loads purpose, then alternatives, then output shape, then limitations, and every sentence carries operational weight needed to invoke the tool correctly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of the tool and the presence of an output schema, the description is complete: it explains the analysis mode, the refusal behavior, loop-handling semantics, output fields, and how outputs connect to compare_edge_cases. An agent has enough context to choose, call, and interpret this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already covers all five parameters with clear descriptions, giving a baseline of 3. The description adds meaningful parameter-related behavior by stating that unannotated parameters default to int and by enumerating exactly which Python constructs the code parameter accepts or refuses, which directly affects how to supply both code and inputs.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a precise verb and object: it identifies which if/elif/else arms and loops of a Python3 function can run, which are dead code, and what inputs reach each branch, using z3 without executing the program. It also distinguishes itself from sibling tools by naming trace_execution and z3_check as alternatives, so an agent can select it correctly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit when-to-use and when-not-to-use guidance: use trace_execution for observing a single run, and use z3_check when the input is already an SMT-LIB2 script rather than Python source. It also documents the unsupported-construct boundary, so the agent knows in advance when this tool will refuse.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Use calc_exact, not evaluate_expression, for a literal arithmetic expression with no symbols in it. EXACT arithmetic: 0.1 + 0.2 == 0.3 is True here (False in plain Python).
Everything is an exact rational, integers are arbitrary precision. Supports
/ // % ** comparisons, bitwise ops (& | ^ << >> ~) on integers, and whitelisted math functions (sqrt, log, sin, ...) plus pi/e/tau. Use BEFORE asserting any computed number: thresholds, ratios, overflows, 'X is N% of Y'. Examples: '2**64 - 1', 'comb(52,5)', '0.1+0.2 == 0.3', '0xff & 0x0f'.
| Name | Required | Description | Default |
|---|---|---|---|
| expr | Yes | Literal arithmetic expression with no symbols, e.g. '2**64 - 1', 'comb(52,5)', '0.1+0.2 == 0.3' |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses key behavioral differences from plain Python: exact decimal arithmetic ('0.1 + 0.2 == 0.3' is True), arbitrary precision integers, supported operations, and math functions. These are not implied by the annotations (readOnlyHint: true, idempotentHint: true) and are essential for an agent to predict results correctly.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The critical guidance is front-loaded with the 'not evaluate_expression' contrast, and the rest is dense but purposeful. The example list and supported-operator inventory are useful; only slight redundancy exists between 'EXACT arithmetic' and the later mention of exact rationals.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description fully equips an agent to use the tool correctly: it explains when to use it, what input restrictions apply, what arithmetic guarantees are provided, common use cases, and concrete examples. Combined with the output schema and safe read-only annotations, nothing essential is missing for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although the schema already describes the expr parameter with example strings, the description significantly expands its semantics by defining what constitutes a valid expression ('literal expression with no symbols'), explaining exact rational behavior, and enumerating supported operators and functions. This goes well beyond the schema's baseline coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific directive: use calc_exact, not evaluate_expression, for a literal arithmetic expression with no symbols. It names the resource (literal arithmetic expressions), the verb (calculate exactly), and explicitly distinguishes itself from its sibling tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly contrasts calc_exact with evaluate_expression and specifies when to choose it: 'Use BEFORE asserting any computed number: thresholds, ratios, overflows, 'X is N% of Y''. This gives clear, actionable selection criteria beyond generic tool descriptions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Mean, median, sample stdev, and coefficient of variation (CV) for a sample of numbers. Pairs with percentiles for distribution shape (p50/p90/p95/p99) on the same sample, and with benchmark or verify_optimization, which are common sources of the timing samples this tool summarizes. CV > 0.2 flags run-to-run noise that swamps the effect. Returns n/mean/median/stdev/cv plus a cv_note.
| Name | Required | Description | Default |
|---|---|---|---|
| nums | Yes | Sample of numbers to summarize (mean, median, sample stdev, coefficient of variation) |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover safety (readOnly, idempotent, non-destructive), and the description adds meaningful behavior beyond that: it specifies sample stdev (not population), states that CV > 0.2 flags run-to-run noise, and discloses the return shape including n/mean/median/stdev/cv plus cv_note. No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences each earn their place: the first states the core statistics, the second positions the tool within the sibling family and common data sources, and the third gives the interpretation rule and return fields. No filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With one simple parameter, full schema coverage, rich annotations, and an output schema present, the description's detail is appropriately matched. It explains the tool's role, its interpretation, and its return value sufficiently for correct invocation. Nothing essential is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents the nums parameter as 'Sample of numbers to summarize (mean, median, sample stdev, coefficient of variation)'. The description adds no new parameter-level meaning beyond what the schema states, falling to the baseline for fully covered schemas.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with the exact computed quantities: mean, median, sample stdev, and coefficient of variation for a sample of numbers. It clearly identifies the resource (a numeric sample) and operation, and implicitly distinguishes from sibling percentiles by describing itself as a complement, not a duplicate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context for when to use the tool: it pairs with percentiles for distribution shape and summarizes timing samples commonly sourced from benchmark or verify_optimization. It stops short of explicitly listing exclusions or a 'when not to use' rule, but the role is unambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Birthday-bound hash collision probability: 1 - exp(-n^2 / (2*2^b)).
Sizes hashes: 1e6 items into 64 bits is ~2.7e-8; 1e5 into 32 bits is ~0.69 — the answer to 'can I truncate this to 8 hex chars?' (no).
| Name | Required | Description | Default |
|---|---|---|---|
| bits | Yes | Width of the hash in bits, e.g. 32, 64, 128 | |
| items | Yes | Number of items being hashed |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish that the tool is read-only, idempotent, and non-destructive. The description adds the exact formula and examples, so the computation is fully transparent and deterministic with no hidden behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded: a formula, two illustrative examples, and one practical takeaway. Every sentence contributes to understanding the tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter pure calculation tool with strong annotations and an output schema, the formula plus examples fully equip an agent to invoke it correctly. Nothing critical is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema fully documents both parameters, and the description adds meaning by showing how items and bits map into the formula. The concrete examples make parameter semantics vivid and unambiguous.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly defines the tool's purpose: computing birthday-bound hash collision probability. It includes the exact formula and concrete example outputs, making it easy to distinguish from other numeric or statistical sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a practical use case: deciding whether truncating a hash to a given bit width is safe. It gives clear context for when to use the tool, though it does not explicitly name alternatives or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run the same logic in N languages on edge-case inputs and flag divergence.
Default inputs cover empty, zero, negative, and float-precision cases: ['', '0', '1', '-1', '10', '100', '0.1\n0.2']. Returns a per-input matrix plus a divergences list where languages disagree on identical input.
| Name | Required | Description | Default |
|---|---|---|---|
| inputs | No | Inputs to run every snippet on; omit for the default set covering empty/zero/negative/float cases | |
| snippets | Yes | Language name -> code; provide one correct snippet per language, implementing the same logic |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false and openWorldHint=true. The description adds that it runs code and returns a per-input matrix plus divergences list, and specifies the default inputs. This provides some behavioral context, but it does not disclose potential side effects, permissions, or rate limits, so it partially covers the burden.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no wasted words. The first sentence states the core purpose, and the second provides crucial details about inputs and output format. Information is front-loaded and efficiently structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and the description's explanation of the return structure (matrix and divergences list), the description covers the essential usage. It does not mention limitations or prerequisites, but for a code-execution tool with 2 parameters, it is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema defines both parameters. The description adds the explicit default input list (['', '0', '1', '-1', '10', '100', '0.1\n0.2']), which is concrete and helpful, going beyond the schema's summary.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Run the same logic in N languages on edge-case inputs and flag divergence,' which is a specific verb, resource, and outcome. It clearly differentiates its focus on edge-case inputs and divergence detection from other comparison tools in the sibling list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No mention of when to use this tool versus alternatives. It does not provide any 'use this when...' or 'instead of...' guidance, leaving the agent to infer its applicability from the description alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run the same code in multiple languages side by side.
Returns per-language stdout/stderr/exit/duration plus which was fastest. Example: {"python3": "print(67)", "node": "console.log(67)"}
This tool fans out across every language with no per-language install
plumbing behind it; use install_package/execute_code(dependencies=...)
beforehand instead. A # /// script block in a snippet is likewise never
installed, but is DISCLOSED, not dropped: a python3 row that carries one
gets dependencies: {"status": "unsupported", "reason": ...}.
| Name | Required | Description | Default |
|---|---|---|---|
| stdin | No | Text piped to every snippet's standard input; empty means no input | |
| timeout | No | Wall-clock seconds allowed per language before that run is killed | |
| snippets | Yes | Language name -> code; each snippet must be a complete, valid program in its own language | |
| dependencies | No | Not supported here; any truthy value is refused — install packages beforehand instead |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only set readOnlyHint=false, openWorldHint=true, etc., without detailing specific behavior. The description goes far beyond annotations by disclosing that the tool 'fans out across every language' without install plumbing, that dependencies passed to this tool are refused (via schema) and that script blocks are disclosed but not installed. It also details the returned fields (stdout/stderr/exit/duration, fastest), adding meaningful behavioral transparency not inferable from annotations alone.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact: two paragraphs with a clear example. The primary purpose is front-loaded, followed by crucial caveats about dependencies and script blocks. No redundant phrases; every sentence carries actionable information. The example is concise and illustrative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (multi-language execution, nested snippets, dependency handling) and that it has an output schema (though not shown), the description covers all essential points: what it does, what it returns, how dependencies are treated, and an example. The agent has enough to invoke it correctly and interpret results. No significant gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage and already describes each parameter (stdin, timeout, snippets, dependencies). The description adds value by giving a concrete example of the snippets object and clarifying that each snippet must be a complete valid program. It also explains that dependencies are not supported and that script blocks yield a specific status. While the schema covers the basics, these additions sharpen understanding without contradicting the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description begins with a clear, specific statement of function: 'Run the same code in multiple languages side by side.' It further clarifies the output (stdout/stderr/exit/duration plus fastest) and gives an example. It distinguishes itself from siblings by explicitly referencing `install_package` and `execute_code(dependencies=...)`, making the tool's scope unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use alternatives: 'use `install_package`/`execute_code(dependencies=...)` beforehand instead' for dependency installation. It also explains the behavior for `# /// script` blocks, advising that they are disclosed rather than installed. This provides clear, actionable context for when to choose this tool versus its siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Exact threshold check with a verdict and the shortfall when it fails. Use calc_exact, not this, when you want the computed VALUE rather than a threshold comparison.
a OP b. Both sides are evaluated exactly and printed as fractions — a
threshold comparison written out cannot be gotten backwards. Example:
('1/25', '>', '0.05').
| Name | Required | Description | Default |
|---|---|---|---|
| a | Yes | Left-hand numeric expression, evaluated exactly | |
| b | Yes | Right-hand numeric expression, evaluated exactly | |
| op | Yes | Comparison operator: one of ==, !=, >, >=, <, <= ('=' also accepted for ==) |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds valuable behavioral context: both sides are evaluated exactly and printed as fractions, and the comparison 'cannot be gotten backwards.' It also mentions the verdict and shortfall output. This goes beyond the annotations without contradicting them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded: the core purpose and the key alternative are stated in the first two sentences. The example and the 'cannot be gotten backwards' note are useful and not redundant. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has an output schema, so return values are already documented. The description covers the core behavior, the alternative tool, and an example. It could mention edge cases like invalid operators or division by zero, but for a simple comparison tool with full schema coverage and an output schema, the description is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all three parameters (a, b, op) with descriptions. The description adds the example ('1/25', '>', '0.05') and clarifies that both sides are evaluated exactly, but it doesn't add much beyond what the schema already provides. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Exact threshold check') and resource (threshold comparison), and explicitly distinguishes it from calc_exact by saying 'Use calc_exact, not this, when you want the computed VALUE rather than a threshold comparison.' This clearly differentiates it from the most similar sibling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly names the alternative tool (calc_exact) and the condition that selects it ('when you want the computed VALUE rather than a threshold comparison'). It also explains the exact-evaluation behavior and gives an example, leaving no ambiguity about when to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Convert a value between units (dimensional analysis via sympy).
Supports metric/imperial length, mass, time, speed, energy, power, force, pressure, temperature (°C/°F/K), volume, area, data sizes, frequency. Examples: ('60','mph','km/h'), ('100','celsius','fahrenheit'), ('1','gb','mib'). Use list_units for the full alias table.
| Name | Required | Description | Default |
|---|---|---|---|
| value | Yes | Numeric quantity to convert, in `from_unit` | |
| to_unit | Yes | Target unit alias, e.g. 'km/h', 'fahrenheit', 'mib'; see list_units for all aliases | |
| from_unit | Yes | Source unit alias, e.g. 'mph', 'celsius', 'gb'; see list_units for all aliases |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish safety (readOnly, idempotent, non-destructive), so the description adds context by mentioning 'dimensional analysis via sympy' and specifying the supported measurement categories. It stops short of describing behavior on invalid units or edge cases, but with annotations carrying the safety profile and an output schema present, this is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and well-structured: a one-line purpose, a category list, a few clarifying examples, and a pointer to list_units. Every sentence earns its place, and the core purpose is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a moderate-complexity conversion tool, the description covers supported domains, gives representative examples, points to the alias reference, and benefits from a 100%-covered schema and an output schema. Nothing essential for invoking the tool correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description reinforces parameter meaning with examples like ('60','mph','km/h') and points to list_units for aliases, but it does not add new per-parameter semantics beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Convert a value between units (dimensional analysis via sympy)'. It enumerates the supported categories and gives concrete examples, making the tool's purpose unmistakable and distinguishing it from siblings like list_units and data_sizes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly implies this tool is for value conversion and explicitly routes users to list_units for the full alias table, which is a useful alternative-selection signal. However, it does not explicitly state when not to use convert_units or how to choose among potentially overlapping siblings such as data_sizes or human_duration.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Byte counts for a plain integer, both binary and decimal — the gap
between them is where '291 MB' and '277 MiB' silently disagree by 5%.
For units other than bytes, use convert_units. For a duration, not a
byte count, use human_duration. Returns bytes plus binary and
decimal dicts of unit -> value.
| Name | Required | Description | Default |
|---|---|---|---|
| n | Yes | Byte count to express in both binary (KiB/MiB/GiB/TiB, /1024) and decimal (KB/MB/GB/TB, /1000) units |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering safety. The description adds the return structure ('bytes' plus 'binary' and 'decimal' dicts) and clarifies the meaning of the two unit systems, beyond what annotations provide. It doesn't mention edge cases like negative integers, but that's more parameter semantics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, with no fluff. It front-loads the core purpose, then provides usage routing, and ends with the return format. Every sentence adds value and the structure is easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple, one-parameter, read-only tool with full schema coverage and an output schema, the description covers everything needed: purpose, usage boundaries, return format, and unit definitions. Nothing critical is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter n has a description covering its semantics completely: it specifies the exact units used (KiB/MiB/GiB/TiB vs KB/MB/GB/TB) and the divisors (/1024 vs /1000). Schema coverage is 100%, and the description reinforces the parameter's purpose.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns byte counts for a plain integer in both binary and decimal units, and explicitly names the sibling tools it is not (convert_units for other units, human_duration for durations). This distinguishes it from all alternatives without needing to inspect schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit when-to-use guidance: for byte counts only, and directly routes to alternatives for other cases. The example of the 5% discrepancy adds context for when the distinction matters, leaving no ambiguity about appropriate invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Epoch seconds/millis/micros/nanos to ISO 8601 UTC (implausible readings suppressed).
| Name | Required | Description | Default |
|---|---|---|---|
| n | Yes | Epoch timestamp to convert to ISO 8601 UTC; units (seconds/millis/micros/nanos) are inferred from magnitude |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnly=true, idempotent=true, destructive=false. The description adds useful behavioral context: output is ISO 8601 UTC and implausible readings are suppressed. This goes beyond the schema and annotations, though it doesn't precisely define what counts as implausible.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One terse sentence carries all essential information: input forms, output format, and the suppression caveat. No filler or repetition of the schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Input schema fully documents the single parameter incl. unit inference, and there is an output schema. The description adds the key behavioral caveat about implausible readings. Only minor ambiguity about what counts as 'implausible', but the tool is simple and well covered by schema/annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds the key behavioral detail that units are inferred from magnitude (also in schema) and that implausible values are suppressed. It doesn't duplicate the schema's parameter description, and what it adds is meaningful, so slightly above baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific operation (convert epoch time) and the concrete resource transformation (seconds/millis/micros/nanos to ISO 8601 UTC). It clearly distinguishes the core use case, though it does not explicitly contrast with sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to prefer this over other tools, nor any exclusions/prerequisites. The description only states what it does, not when it should be selected; usage context must be inferred from the schema and annotations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Use evaluate_expression, not calc_exact, for something other than
plain arithmetic on literal values. Symbolically evaluate to a value or
closed form via sympify: 'integrate(x2, x)', 'sqrt(144) + 210'. Not
simplification — for simplified/factored/expanded forms, use
symbolic(op="simplify"). Returns value (if the result is a number) or
the evaluated expression, plus type.
| Name | Required | Description | Default |
|---|---|---|---|
| expression | Yes | Symbolic math expression to evaluate via SymPy, e.g. 'integrate(x**2, x)', 'sqrt(144) + 2**10' |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is known. The description adds that the tool evaluates symbolically ('sympify'), is not simplification, and returns a `value` or evaluated expression plus `type`. This goes beyond the annotations by clarifying the function's side-effect-free, symbolic nature and its return structure, though it doesn't discuss potential edge cases or failure modes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—two sentences with no filler. It front-loads the most critical information (when to use it, what it does, examples) and then adds the exclusion against simplification. Every phrase earns its place, making it easy for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter tool with a well-covered schema and a provided output schema, the description is complete. It covers the tool's niche (symbolic vs. numeric, vs. simplification), its behavior (returns value/type), and routes the agent to alternatives. Nothing essential is missing for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the `expression` parameter is already documented with the same examples. The description reinforces the parameter's meaning but does not add significant new semantic depth beyond what the schema provides — it repeats the examples and adds the sympy context, but no extra syntax or edge-case guidance. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Symbolically evaluate to a value or closed form via sympify.' It gives concrete examples ('integrate(x**2, x)', 'sqrt(144) + 2**10') and immediately differentiates from calc_exact with 'something other than plain arithmetic on literal values.' This makes the tool's scope unambiguous and distinguishes it from its sibling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance is provided: 'Use evaluate_expression, not calc_exact, for something other than plain arithmetic on literal values' and 'Not simplification — for simplified/factored/expanded forms, use symbolic(op="simplify").' These two sentences tell an agent exactly when to use this tool versus the two closest siblings, leaving no ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Execute code in language in a sandbox. Use this, not
execute_code_stream/run_submit/session_run, for one program whose result
you can wait for within a 120s cap.
Returns stdout, stderr, exit_code, duration_ms, cpu_ms, peak_memory_kb, verdict (OK/TLE/MLE/OLE/RTE).
session_id: run inside a session workspace (see session_start); with a
stateful session (python3/node) interpreter state persists across calls.
Also reports artifacts_created (files just created/modified) since
that workspace outlives the call; a sessionless run has none. See
session_run for the same field plus inline content blocks.
max_output_kb: the 240 KiB hard ceiling is what the
anthropic/maxResultSizeChars this tool advertises in its _meta
already assumes — the cap leaves no headroom to raise past it without
the real result exceeding that hint. A run whose real output needs
more than 240 KiB belongs in a session instead: leave max_output_kb
at its default (0) with session_id set (below), and oversized output
SPILLS to a full-fidelity file readable via session_read_file rather
than truncating — see the spill paragraph further down. An EXPLICIT
max_output_kb, even under the 240 KiB ceiling, is honoured as a
literal cap with no spill.
no_net: Linux enforces in-kernel via a seccomp-bpf filter. macOS /
no-seccomp kernel: best-effort symbol shim, disclosed in unenforced
when that's the only guarantee that held. See SECURITY.md.
compact: never drops unenforced, output_error,
artifacts_created, or dependencies — if a guarantee you asked for
was not applied, or a declared install failed, a compact result still
says so.
dependencies: merged with a PEP 723 # /// script block for
python3, deduped by normalized name with this argument winning; the
only source for node. A block ALONE, with no
dependencies argument, is enough to trigger an install — see
SECURITY.md. Installed BEFORE the sandboxed step, through the same
confined install_package path — never inside the sandbox — and
refused (capability_not_requested) without installing anything when
no_net=True or the capability policy denies or strictly limits
network. Bounded by a fixed install-time budget separate from
timeout (120s aggregate across every dependency;
codecalc.dependencies.DEFAULT_DEPENDENCY_INSTALL_BUDGET_SECONDS) and,
session-less, by CODECALC_SESSION_DISK_QUOTA_MB on the run's own
workdir — either one exceeded refuses the run with a stamped, coded
error naming the ceiling. See the dependencies field on the result.
With session_id set and max_output_kb left at its default, output that
would otherwise be truncated is instead SPILLED: the inline
stdout/stderr still carry the same truncated prefix as before, and
stdout_spill/stderr_spill name a codecalc://session/{sid}/files/...
resource carrying the fuller stream (session_read_file or the resource
route reads it back) — capped at 4 MiB, ..._spill_capped: true if even
that was not enough to hold everything. Passing an EXPLICIT
max_output_kb is honoured as a literal ceiling with no spill, same as
before. Session-LESS runs (no session_id) have no workspace to spill
into and keep the old truncate-and-drop behaviour.
| Name | Required | Description | Default |
|---|---|---|---|
| code | Yes | Source code to run in `language` | |
| stdin | No | Text piped to the program's standard input; empty means no input | |
| no_net | No | Block outbound network access for this run; best-effort on platforms without seccomp | |
| compact | No | Drop diagnostic fields (timings, workdir, platform) from the result; safety disclosures are always kept | |
| max_cpu | No | Per-call CPU-time ceiling in seconds; 0 means no explicit limit is set | |
| timeout | No | Wall-clock seconds before the run is killed as TLE; clamped to a 120s ceiling | |
| language | Yes | Runtime to execute in, e.g. 'python3', 'node'; see list_languages for the full catalog | |
| provider | No | Execution backend id to use (see list_execution_providers); default picks automatically | |
| session_id | No | Run inside this session's workspace (from session_start) instead of a throwaway sandbox | |
| dependencies | No | Packages to install before running, e.g. ['requests==2.31.0']; merged with any PEP 723 block | |
| max_memory_mb | No | Per-call memory ceiling in MiB; 0 means no explicit limit is set | |
| max_output_kb | No | Stdout/stderr capture cap in KiB per stream; 0 uses the 64 KiB default, hard-clamped to 240 |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes far beyond the sparse annotations (readOnlyHint=false, openWorldHint=true) to disclose sandbox containment, the full verdict set (OK/TLE/MLE/OLE/RTE), seccomp-bpf vs best-effort network enforcement with the 'unenforced' disclosure, output spill mechanics, dependency install refusal conditions, and the install-time budget. No contradiction with annotations; in fact it enriches what openWorldHint=true implies about network and dependency behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very long, but it is well-structured with bold lead-ins and bullets, and the core purpose/usage line is front-loaded before any parameter detail. Given the tool's complexity (12 parameters, spill behavior, security nuances, sibling differentiation), most sentences earn their place, though the spill and dependency sections are dense enough that a committed reader must work to extract the key rules.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 12-parameter execution tool with a rich output schema and many execution siblings, the description is exceptionally complete. It covers return-value verdict semantics, edge cases (TLE clamping, truncation vs spill, capped spill with _spill_capped flag, dependency refusal codes), and security guarantees. Even though an output schema exists, the description adds verdict meanings and behavioral consequences that the schema alone would not convey.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 100% schema coverage, the description adds substantial meaning the schema lacks: max_output_kb's relationship to the anthropic/maxResultSizeChars hint and the literal-cap-vs-spill distinction; session_id's state persistence and artifacts_created reporting; no_net's platform-dependent enforcement; compact's guaranteed-retained fields; and dependencies' PEP 723 merging with normalized-name dedup winning toward the argument. This far exceeds the baseline 3 for covered schemas.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The opening sentence 'Execute code in language in a sandbox' states a precise verb, resource, and containment context. It then names the siblings it is NOT ('Use this, not execute_code_stream/run_submit/session_run'), differentiating it from the cluster of execution tools that share this domain.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit routing guidance is front-loaded: this tool is for 'one program whose result you can wait for within a 120s cap,' and oversized output is explicitly redirected to session_run/session_read_file instead. It also states the exact conditions that should drive an agent away from this tool (output > 240 KiB, need for spill/artifacts), leaving nothing to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Execute code and STREAM progress + partial output as it runs. Use this, not execute_code/run_submit/session_run, for the same run when you want output while it runs, up to a 300s cap.
Reports progress notifications to the client while the program runs, so agents can see output before the process finishes. Returns the same result shape and applies the SAME ceilings as execute_code: max_memory_mb, max_output_kb and max_cpu are forwarded to the executor exactly as execute_code forwards them, including the same 240 KiB per-stream clamp.
dependencies: same as execute_code's (PEP 723 merge, no_net/policy
refusal, 120s budget, workdir quota) — installed before streaming
starts; a refusal/failed install is the stream's only event.
| Name | Required | Description | Default |
|---|---|---|---|
| code | Yes | Source code to run in `language` | |
| stdin | No | Text piped to the program's standard input; empty means no input | |
| no_net | No | Block outbound network access for this run; best-effort on platforms without seccomp | |
| max_cpu | No | Per-call CPU-time ceiling in seconds; 0 means no explicit limit is set | |
| timeout | No | Wall-clock seconds before the run is killed; clamped to a 300s ceiling (longer than execute_code's) | |
| language | Yes | Runtime to execute in, e.g. 'python3', 'node'; see list_languages for the full catalog | |
| provider | No | Execution backend id to use (see list_execution_providers); default picks automatically | |
| dependencies | No | Packages to install before running, e.g. ['requests==2.31.0']; merged with any PEP 723 block | |
| max_memory_mb | No | Per-call memory ceiling in MiB; 0 means no explicit limit is set | |
| max_output_kb | No | Stdout/stderr capture cap in KiB per stream; 0 uses the 64 KiB default, hard-clamped to 240 |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the annotations by explaining progress notifications, the same result shape and ceilings as execute_code, the 240 KiB per-stream clamp, and dependency installation timing. It also states that a refusal/failed install is the stream's only event, which is non-obvious behavior an agent needs to know.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense and front-loaded, with the core purpose and routing in the first sentence. It is longer than minimal, but the additional paragraphs each carry essential behavioral and dependency details rather than filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Complete for a streaming execution tool: it covers when to use it, how it behaves, and key parameter quirks, while the output schema handles return value details. Referencing execute_code for exact ceilings avoids duplication and keeps the definition self-consistent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents all parameters. The description adds meaningful extra semantics for dependencies (PEP 723 merge, no_net/policy refusal, 120s budget, workdir quota) and confirms that max_memory_mb, max_output_kb, and max_cpu are forwarded exactly as in execute_code.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: executing code while streaming progress and partial output. It distinguishes itself from execute_code/run_submit/session_run in the first sentence, so an agent can immediately grasp what makes this tool unique.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells the agent when to choose this over alternatives: use this for the same run when output is wanted while it runs, with a 300s cap. This is concrete routing guidance with named sibling tools and a clear condition.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract a named function (with its imports + referenced helpers) into a standalone program and run it in the sandbox.
python3 gets exact ast extraction; other languages best-effort block
extraction (pass call to execute non-python). Returns the extracted
program and per-input runs.
| Name | Required | Description | Default |
|---|---|---|---|
| call | No | Call expression to invoke the extracted function; required for non-python3 languages | |
| code | Yes | Source containing the function to extract, plus its imports and helpers | |
| language | Yes | Language `code` is written in; python3 gets exact ast extraction, others best-effort block extraction | |
| test_inputs | No | Inputs to run the extracted program with, one run per input | |
| function_name | Yes | Name of the function within `code` to extract into a standalone program |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that the tool runs code in a sandbox, handles python3 exactly but other languages best-effort, and returns the extracted program plus per-input runs. This adds meaningful behavioral context beyond the annotations, which only cover mutability and idempotence. No contradiction with annotations is present.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three focused sentences, each earning its place: the first states the core action, the second covers language-specific behavior, and the third states the return value. The most important scoping information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with five parameters, an output schema, and detailed per-parameter schema descriptions, the description covers all essential behavior: extraction scope, sandbox execution, language-dependent semantics, the `call` requirement, and return contents. Nothing critical is missing for an agent to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the input schema already documents all parameters and their descriptions. The description reinforces that `call` is needed for non-python languages and that language affects extraction behavior, but it does not add substantial new meaning beyond what the schema already states.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Extract') with a clear resource ('a named function ... into a standalone program') and clarifies the scope ('with its imports + referenced helpers'). It also says it runs the program and returns results, which clearly distinguishes it from generic code execution tools like execute_code.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit context for language-specific usage: python3 uses exact AST extraction, while other languages use best-effort block extraction and require the `call` parameter. It does not name alternative tools or exclusion conditions, but it provides clear operational guidance for when to call this tool and how to configure it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
What binary64 actually stores for X: exact value, raw bits, ULP, both
neighbours, and whether the literal is representable. float_repr(0.1)
shows 0.1000000000000000055511151231257827...; float_repr(0.25) says
EXACT. Above 2^53 warns consecutive integers are indistinguishable.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | Value to inspect as binary64: exact stored value, raw bits, ULP, neighbours, and representability |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds valuable behavioral context: it reports ULP, neighbors, representability, and warns about the ≥2^53 limit where consecutive integers become indistinguishable. This goes beyond the annotation flags.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded with the core purpose, followed by concrete examples and an edge-case warning. Every sentence adds value, and the structure is easy for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With a single parameter and an output schema present, the description covers the purpose, behavior, and edge cases. It lacks an explicit note on error handling (e.g., non-finite values), but is otherwise complete for the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter x is fully described in the schema (coverage 100%), with the same explanation in the tool description. The description re-emphasizes the meaning and provides examples, but does not add new parameter semantics beyond what the schema already states.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states precisely what the tool does: it shows the exact binary64 stored value, raw bits, ULP, neighboring values, and representability. It is clearly distinguishable from siblings like bits, radix_convert, or calc_exact by focusing on float representation. The example cases reinforce the purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a usage example but does not explicitly state when to use this tool versus alternatives such as bits or radix_convert. It implies the tool is for inspecting float representation, but there is no guidance on when not to use it or which sibling to prefer in other cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Convert a SPAN of elapsed seconds into a humanised duration (e.g.
'2d 3h 4m 5s') plus per-day and per-30d rates. For an epoch timestamp
to a calendar date, use epoch_time instead. For byte counts, not
seconds, use data_sizes. Returns human, per_day, per_30d, and the
echoed seconds.
| Name | Required | Description | Default |
|---|---|---|---|
| seconds | Yes | Elapsed span in seconds (not a point-in-time timestamp) to humanize, e.g. into '2d 3h 4m 5s' |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already signal read-only, idempotent, non-destructive behavior. The description supplements this by naming returned fields (`human`, `per_day`, `per_30d`) and emphasizing that input is an elapsed span, not a point-in-time timestamp. That is useful but not exhaustive behavior detail; annotations carry much of the safety profile.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences, none redundant; purpose first, sibling routing second/third, return shape last.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Complete enough for a simple conversion tool with one parameter heavily documented in schema den output schema present. Could mention precision/rounding edge cases but these are not essential for agent use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents seconds at 100% coverage Conclusion description about 'elapsed span' reinforces the point-in-time/timestamp distinction but adds no new parameter constraints. Baseline 3 is appropriate because structured coverage handles the parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action: convert an elapsed span of seconds into a human-readable duration plus per-day and per-30d rates. It also explicitly contrasts itself with epoch_time and data_sizes, so an agent can immediately distinguish this tool from siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit routing guidance: use epoch_time for timestamps and data_sizes for byte counts. This is clear when-to-use/when-not-to-use information, not just a vague description.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Install a package for a language (uv pip / npm / gem / go get / cargo add...).
Asks the caller to confirm before installing (a protocol-level gate, not
just the anthropic/requiresUserInteraction _meta hint — see
codecalc/confirmation.py); a declined or malformed confirmation refuses
with no install attempted.
With session_id, executed code in that session can import the result.
NETWORK: yes, always. The package manager fetches from its registry (PyPI, npm, rubygems, crates.io). codecalc opens no socket itself; the child process does.
NOT SANDBOXED: the installer runs as a direct subprocess of the server, so install-time hooks (npm postinstall, Python build backends, Cargo build scripts) execute with the server user's filesystem access. The environment is still restricted to the allowlist, so secrets do not leak, but the filesystem is not confined. Do not point this at untrusted input. See SECURITY.md.
| Name | Required | Description | Default |
|---|---|---|---|
| package | Yes | Package name to install via that language's manager (uv pip/npm/gem/go get/cargo add) | |
| version | No | Exact version to install; omit to install the manager's default/latest | |
| language | Yes | Language whose package manager installs the package, e.g. 'python3', 'node' | |
| session_id | No | Install into this session's workspace instead of the shared cache; omit for the shared cache |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes beyond the destructiveHint annotation by explaining precisely why this is dangerous: it requires user confirmation, runs as a direct subprocess, triggers install-time hooks, and executes with the server user's filesystem access. It also flags network egress. This is exactly the kind of behavioral context a model agent needs before invoking a mutating tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact, scannable, and every sentence adds value: the package-manager list clarifies scope, the confirmation note sets expectations, and the sandbox warning is essential. No filler or redundant restatement of the schema is present.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive, network-touching installation tool, the description covers the critical operational context: languages supported, session behavior, confirmation requirement, network use, and lack of sandboxing. Given the schema already documents parameters and there is an output schema, nothing important is left for the agent to infer.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is already 100%, with each parameter described in the input schema. The description adds useful examples for `language`, but the core parameter meanings remain in the schema. It does not materially clarify `package`, `version`, or `session_id` beyond what the schema already states.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description opens with a precise verb-object statement: 'Install a package for a language' and enumerates concrete package managers (uv pip, npm, gem, go get, cargo add). This makes the tool's role immediately unambiguous, even in a large sibling list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly frames when to use the tool—when a dependency needs to be installed into a language environment, with optional session scoping. It does not name a specific alternative to prefer over, but the package-manager examples make the intended context concrete. The security warning 'Do not point this at untrusted input' is a meaningful usage guardrail.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List execution providers (execution BACKENDS — local subprocess, gVisor-strict, remote) and their machine-readable capabilities.
This is about which BACKEND runs your code, not which LANGUAGE it runs —
a provider's ready/strict fields are resolution facts about the
backend itself. Per-language reliability (how much codecalc's CI has
actually verified a given language's toolchain, vs merely resolved it) is
a separate axis reported by list_languages/runtimes_status/codecalc doctor as tier; a ready provider says nothing about whether a
specific language running through it has ever been execution-tested.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and non-destructive behavior. The description adds valuable context on the meaning of ready/strict fields as resolution facts and explicitly warns that a ready provider does not imply language execution testing. This goes beyond the annotations without contradicting them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is slightly longer than necessary but each sentence adds value: it names the resource, gives examples, and clarifies a common misconception. The key purpose is front-loaded, and the additional explanation is relevant for correct usage.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no parameters, an output schema present, and annotations covering safety, the description is complete. It provides the necessary distinction from sibling tools and clarifies the semantics of the returned data, leaving no critical gaps for an agent to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline is 4. The description does not need to explain parameters, but it does mention 'machine-readable capabilities' which hints at the output structure, though the output schema itself covers details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear action (list) and resource (execution providers/backends) with concrete examples (local subprocess, gVisor-strict, remote). It explicitly distinguishes from list_languages by clarifying the backend-vs-language axis, which prevents agent confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states that this tool is about backend resolution, not language reliability, and names the alternatives (list_languages, runtimes_status, codecalc doctor) that report the separate language tier axis. This gives clear when-to-use versus when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List every language codecalc can execute, with extension, compile flag, and what this machine resolved. status is installed (its command was found on the sandbox PATH) or supported (nothing for it here); status_basis is resolved, meaning nothing was executed to check. Run codecalc doctor --deep to promote a runtime to available by actually running it.
tier is a DIFFERENT axis from status: status says whether THIS
machine resolved the command (resolution); tier says whether codecalc's
own CI has actually executed this language and asserted on its output
(reliability) — tested, best_effort (declared, plausibly works, never
CI-checked), or plan_only (never validated anywhere). A language can be
installed here and still be best_effort or worse — that combination is
exactly "the toolchain resolved and may still be broken".
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, but the description adds crucial behavioral nuance: it states that status_basis is 'resolved' meaning nothing was executed, and distinguishes this from CI-tested reliability via the tier axis. It also discloses that a language can be installed yet still broken, which is a key limitation an agent must know. This goes well beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the primary purpose in the first sentence, then dives into clarifying the status and tier fields. Every sentence adds value—explaining the status values, the resolution basis, the doctor command, and the key distinction between status and tier. It is detailed but not wasteful; the length is justified by the two-axis complexity it must convey.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a list tool with an output schema, the description fully explains the semantics of the fields it returns (extension, compile flag, status, status_basis, tier) and all their possible values. It also provides guidance on how to get more thorough checks via `codecalc doctor --deep`. Nothing an agent needs to correctly interpret the output is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the description has no parameter explanations to add. Per the rubric, 0 params yields a baseline of 4. The description does not reference parameters, which is appropriate since none exist.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource: 'List every language codecalc can execute, with extension, compile flag, and what this machine resolved.' It clearly distinguishes itself from siblings like list_execution_providers and runtimes_status by focusing on languages and their resolution status. The added detail about status and tier further clarifies the scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by stating it lists languages and explains the status semantics, but it does not explicitly name alternatives or exclusion conditions. It does hint that `codecalc doctor --deep` is the way to promote a runtime, which suggests this tool is for a quick resolution check. No contradictory guidance, but no explicit 'when not to use' either.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List every supported unit alias — all aliases and spellings — for convert_units.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint false, covering safety. The description adds the detail that output includes aliases and spellings, which is useful but not extensive. For a trivial read-only listing tool, this is acceptable but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence with zero waste, front-loading the core action and resource. It is appropriately sized for a tool with no parameters.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter read-only listing tool with an output schema, the description fully covers what an agent needs to know: it lists all unit aliases for convert_units. No missing information is required to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline is 4. The description does not need to explain parameters, and it correctly omits any mention since there are none.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (list), the resource (every supported unit alias), and the scope (all aliases and spellings for convert_units). This distinguishes it from sibling tools like list_languages and list_execution_providers without ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage: it is the go-to tool for discovering unit aliases before using convert_units. While it does not explicitly state when not to use it or name alternatives, the context makes its purpose clear, and no sibling tool overlaps in function.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Structured matrix operations: det, inverse, eigenvalues, transpose, rank, trace.
evaluate_expression refuses Matrix([[1,2],[3,4]]) on purpose — [/]
are denied there to block subscript-based RCE escapes, and a matrix
literal is collateral from that (correctly aimed) screen. This tool is
the structured replacement: rows is a JSON array of arrays (row-major),
never a string to parse. Each entry is either a JSON number, used
directly, or a scalar expression string ('1/2', 'sqrt(2)', 'x+1'),
screened per-entry the same way evaluate_expression screens its input
before anything reaches SymPy. Example: rows=[[1,2],[3,4]], op='det' -> -2.
| Name | Required | Description | Default |
|---|---|---|---|
| op | Yes | Operation to apply: one of det, inverse, eigenvalues, transpose, rank, trace | |
| rows | Yes | Row-major matrix as a JSON array of arrays; each entry is a number or a scalar expression string like 'sqrt(2)' |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnlyHint and idempotentHint annotations, the description discloses how entries are handled: JSON numbers are used directly, string entries are treated as scalar expressions, and each entry is screened per-entry like evaluate_expression. It also explains the security rationale for rejecting matrix literals in evaluate_expression, adding meaningful behavioral context not available in annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the operation list and follows with a clear example, but the RCE/security explanation is more verbose than necessary. Phrases like 'collateral from that (correctly aimed) screen' add color but not essential guidance, so the description could be tightened without losing meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, the description does not need to explain return values. It covers the input format, per-entry expression behavior, an example, and the relationship to evaluate_expression. It could still note shape constraints (e.g., square matrices for det/inverse/eigenvalues), but the core invocation details are sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description still adds value by specifying row-major ordering, the exact meaning of string entries ('1/2', 'sqrt(2)', 'x+1'), and that rows is never a string to parse. The op parameter is straightforwardly listed in both the first line and the schema, so no additional interpretation is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The first line, 'Structured matrix operations: det, inverse, eigenvalues, transpose, rank, trace,' names the resource and the exact operations, and the example with op='det' makes the behavior concrete. It also explicitly positions this tool against evaluate_expression, so an agent can distinguish it from the closely related sibling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description directly says evaluate_expression refuses Matrix([[...]]) and that this tool is 'the structured replacement.' This gives an explicit when-to-use-this vs. when-not-to-use-the-alternative rule. It also clarifies that rows must be JSON, not a string, which is crucial for correct invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Exact share and percentage of PART / TOTAL. Use calc_exact for a single arithmetic expression, or compare_threshold to check the result against a threshold rather than just compute it.
| Name | Required | Description | Default |
|---|---|---|---|
| part | Yes | Numerator expression (rationals accepted), evaluated exactly | |
| total | Yes | Denominator expression (rationals accepted), evaluated exactly |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already communicate read-only, idempotent, and non-destructive behavior. The description adds the key behavioral detail that computation is exact, which is useful beyond the annotations. The presence of an output schema reduces the need to describe return-value details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no filler. The core function is front-loaded, and the sibling alternatives are mentioned in a compact second sentence.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple pure-calculation tool with fully described required parameters, rich read-only/idempotent annotations, and an output schema, the description is complete. It also names the relevant sibling tools to prevent misrouting.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with part and total already described as numerator and denominator expressions evaluated exactly. The description adds no additional parameter-specific meaning beyond what the schema provides, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool computes the exact share and percentage of PART / TOTAL, and it names calc_exact and compare_threshold as distinct siblings. This gives an agent a precise resource and operation, so it can tell this tool apart from related calculation tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use calc_exact for a single arithmetic expression and compare_threshold for threshold checking instead. This gives clear routing guidance and leaves no ambiguity about when percentage is the right tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
p50/p90/p95/p99 (the 50th/90th/95th/99th percentile cutoffs) by nearest-rank AND linear interpolation. Pairs with calc_stats, which gives mean/median/stdev/CV on the same sample instead of these distribution points.
Warns when n < 100 that p99 is just the maximum wearing a label.
| Name | Required | Description | Default |
|---|---|---|---|
| nums | Yes | Sample of numbers to compute p50/p90/p95/p99 for, by nearest-rank and linear interpolation |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already provide readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the agent knows this is a safe, non-mutating operation. The description adds valuable behavioral context: it discloses the two computation methods (nearest-rank and linear interpolation) and warns about the n < 100 caveat for p99, which is not implied by annotations. However, it does not describe the return format or whether the output is a map or array, but given the annotations cover safety and idempotency, a slight deduction is fair.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact, using three focused sentences. It front-loads the core purpose and method, then immediately provides sibling differentiationassador, and ends with a crucial caveat. Every sentence adds substantive information without redundancy. The structure is logical: purpose first, comparison second, warning last.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a single parameter fully documented in the schema, an output schema exists to describe return values, and annotations cover safety and idempotency, the description is nearly complete. The main missing piece is a clear note on the output format (e.g., a dictionary with both interpolation methods), but the output schema presumably handles that. The description effectively covers the tool's scope, usage, and a key limitation, making it well-rounded.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, so the parameter 'nums' is already well-described in the schema as 'Sample of numbers to compute p50/p90/p95/p99 for, by nearest-rank and linear interpolation.' The description reiterates the same information without adding new details like accepted ranges (e.g., must be non-empty, can contain floats) or edge cases. Thus, it adds minimal value beyond the schema, earning the baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool computes p50/p90/p95/p99 percentiles using both nearest-rank and linear interpolation methods for a given sample. It explicitly names the resource ('percentile cutoffs') and the verb ('computes'), making the purpose unambiguous. The inclusion of the specific percentiles (50th/90th/95th/99th) further distinguishes it from generic statistical tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly pairs the tool with calc_stats, clarifying when to use each: use percentiles for distribution points, use calc_stats for mean/median/stdev/CV. This direct comparison gives clear usage guidance and prevents confusion among siblings. It also warns about the n < 100 case, advising caution when interpreting p99 as the maximum, which helps the agent decide if this tool is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Look up a physical constant (speed_of_light, planck, avogadro, gravity, electron_mass, gas_constant, ...) or list all 22 with values.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | Constant to look up, e.g. 'speed_of_light', 'planck', 'avogadro'; omit to list all 22 |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already establish read-only, idempotent, and non-destructive behavior. The description adds the return expectation by saying 'with values', and it doesn't hide any side effects or contradictions. With annotations present, the bar for additional transparency is reasonably met.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The whole description is one rich, front-loaded sentence. It says the main action first, lists examples, and gives the alternate mode without any filler or redundant explanations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter, read-only lookup with a fully documented parameter, clear annotations, and a known fixed set of constants, the description covers everything needed. The availability of an output schema further reduces the need to describe return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already provides 100% description coverage for the single parameter, including the key examples and the omit-to-list-all behavior. The description repeats examples but adds no substantive semantic value beyond the schema, matching the baseline for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action ('Look up a physical constant') and resource, provides concrete examples, and explicitly adds the alternative behavior of listing all 22 constants with values. This clearly differentiates it from sibling tools like convert_units or list_units.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear usage context: pass a specific constant name for a lookup, or omit the name to get the full list of 22. It doesn't name alternatives explicitly, but no sibling appears to overlap with this domain, so the guidance is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Convert a value between ANY bases 2..36, fractions included; bases that
cannot represent the fraction (e.g. 0.1 in base 2) are flagged
non-terminating. radix_convert('zz', 36, 7) is one call.
| Name | Required | Description | Default |
|---|---|---|---|
| value | Yes | Digit string to convert (fractions with '.' accepted), in `from_base` | |
| to_base | No | Base to convert `value` into; valid range 2..36, default 10 | |
| from_base | No | Base `value` is written in; valid range 2..36, default 10 |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already declare readOnlyHint, idempotentHint, and destructiveHint, covering the safety profile. The description adds genuinely useful behavior: fractions are accepted and non-terminating results are flagged, with an example showing the call shape. It does not contradict any annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two tight sentences that immediately state scope and a key edge-case behavior, then give a compact example. Every clause earns its place with no fluff or repetition of schema details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the base range, fraction handling, non-terminating behavior, and an example, which is sufficient for a moderate-complexity read-only tool. An output schema exists, so the absence of return-format details is acceptable. It is complete enough for an agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3, and the schema already explains each parameter and ranges. The description adds an example call, but the positional order in `radix_convert('zz', 36, 7)` is ambiguous against the schema's property order (value, to_base, from_base), which may confuse an agent. Thus it adds some value but also introduces uncertainty.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Convert'), a precise resource ('a value between ANY bases 2..36'), and a distinctive scope ('fractions included'). This clearly differentiates it from sibling tools like convert_units or float_repr, and the example reinforces the purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description conveys clear usage context: any base between 2 and 36, with fraction support and non-terminating detection. It does not explicitly name alternatives or exclusions, but for a general-purpose conversion tool the domain is unambiguous. The sibling list contains no close competitor that would require an explicit 'use X instead' note.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Cancel a background run started with run_submit.
Idempotent: calling this on a run that is already finished/cleaned
reports cancelled: false, state: <its actual terminal state> rather
than erroring — matching execute_code's own "no partial result" rule,
there is nothing partial to hand back either way.
Propagation depends on the SELECTED PROVIDER (see
list_execution_providers' cancel capability). The built-in local
provider does not support stopping a run once it has started; that is
reported honestly here rather than silently pretended to have worked —
the computation keeps running to completion and its result stays
available via run_inspect, so bound it in advance with run_submit's own
timeout instead. A provider that DOES advertise cancel: true reaches
the full spawned process tree the same way execute_code's own
cancellation does — RunSupervisor already owns that; this tool only
calls it.
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | Id of a background run, as returned by run_submit |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations (idempotent, destructive), the description reveals concrete behavior: response shape for finished runs, provider-dependent cancellation, honest reporting when the local provider cannot propagate cancellation, and that results remain available via run_inspect. This substantially exceeds what annotations alone communicate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficiently structured with the core purpose first, why it is worthwhile, followed by idempotency semantics and provider caveats. It is somewhat long, but every sentence contributes meaningful behavior or usage guidance; no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the single parameter, presence of an output schema, and existing annotations, the description covers all decision-relevant behavior: cancellation semantics by provider, idempotency guarantees, and interaction with run_submit/run_inspect. No important agent-facing gap remains.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the run_id parameter is adequately documented ('Id of a background run, as returned by run_submit'). The description adds no further param detail, so the schema carries the load. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The opening sentence 'Cancel a background run started with run_submit' states a specific verb and resource, clearly distinguishing it from siblings like run_submit and run_inspect. It leaves no ambiguity about the tool's role.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit when-not guidance: the local provider cannot stop a run, so you should use run_submit's timeout instead. It also tells the agent to check list_execution_providers for cancel: true to know when cancellation is effective, and describes the idempotent behavior for already-finished runs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Poll a background run started with run_submit.
While running: {"ok": True, "state": "running"|"cancelling", "run_id", "provider_id", "started_at", "deadline"}.
Once terminal (state "finished"/"cleaned"/"recovered"), this returns
the SAME result shape execute_code returns — stdout/stderr/exit_code/
verdict/unenforced/provider (the interface_version/provider_id/limits
receipt)/... — merged with a small set of run_* extras (run_id,
provider_id, started_at, deadline, state, cleaned; see server.py's
_RUN_EXTRA_KEYS). This terminal reply carries the same
anthropic/maxResultSizeChars _meta execute_code advertises (see
server.py's _LARGE_RESULT_TOOLS) — it is the same envelope, once the
run started with run_submit has finished, and run_submit's own
max_output_kb is clamped the same way execute_code's is so that value
stays true here too. Read ok and verdict on a terminal result to tell
a clean finish from a failure; a run stopped by run_cancel is only
reflected there for a provider that actually supports cancellation (see
run_cancel's own docstring) — check the result the same way you would
any other run.
Retention: a finished run's result stays inspectable for the life of this server process — call this as many times as you like; nothing is consumed by reading it. What IS released on the first terminal read is the PROVIDER's own resources for that run (RunSupervisor.cleanup(), idempotent on repeat calls) — the in-memory record of the run itself is not evicted; there is no cap or TTL on it here, deliberately: the durable state machine, leases and TTL-based eviction are out of this residual's scope (see run_supervisor.py's own docstring). A long-lived server that calls run_submit very many times will grow this table; the on-disk crash-recovery journal underneath it is already bounded (RunSupervisor.max_completed), independent of this.
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | Id of a background run, as returned by run_submit |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite the annotations declaring readOnlyHint, idempotentHint, and destructiveHint=false, the description goes beyond them by detailing critical behaviors: the run's in-memory record is not evicted, there is no TTL, but provider resources are released on the first terminal read (idempotent cleanup). It also discusses memory growth in long-lived servers and the bounded on-disk journal, which is valuable operational context not captured by annotations. The description contradicts no annotations; in fact, it enriches them by explaining the lifecycle and resource implications in detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is lengthy and dense, with detailed technical information about state transitions, resource cleanup, and caveats. It is structured with paragraphs for running vs. terminal states, and then moves to retention behavior. While every sentence adds value, the length could be seen as verbose for a simple polling tool. However, the description is front-loaded with the core purpose and state information, and the extra details are necessary for correct usage. It is not wasteful, but it could be more concise by trimming some redundant references to server files. Score 4.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity—handling polling, terminal state detection, resource cleanup, and result formatting—the description covers all necessary aspects: what to poll, how to interpret states, the result shape, and the retention semantics. It also addresses edge cases like cancellation support and memory growth. The presence of an output schema helps, but the description explicitly explains the result envelope and the `_meta` information, which is not fully covered by the schema. The only minor gap is not detailing pagination or exhaustively listing output fields, but those are covered by the output schema and the reference to run_submit's response. Overall, it's comprehensive for the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already describes the run_id as a string with '[i]d of a background run, as returned by run_submit', giving it high coverage (100%). The description adds further value by explaining that the run_id is used to identify the run and that the result can be polled or retrieved; it clarifies the run_id's role in accessing a specific run's state and result, and mentions the run_id is echoed in the response. This is more than the schema alone provides, so the description enhances parameter understanding beyond the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear verb ('Poll') and a specific resource ('a background run started with run_submit'). It distinguishes itself from siblings by focusing on inspecting a run's status and result, while run_cancel is for stopping and run_submit is for starting. The state machine transitions (running, cancelling, terminal states) are explicitly defined, making it immediately clear what this tool does and how it differs from other run_* tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains that it is used to poll or retrieve the final result of a run submitted by run_submit, and implies it should be called after run_submit and possibly alongside run_cancel. It does not explicitly say 'use run_submit instead when...' or 'use run_cancel for...', but the context is clear from the sibling tool names and the description of run_cancel's behavior. It also mentions the result can be read multiple times, guiding the agent on when to read (retrieval) vs. when to avoid consuming resources (first read releases provider resources). The only omission is an explicit exclusion for cases where the run might still be running and the agent should wait, but the polling implication covers that.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Submit code for BACKGROUND execution; returns a run_id immediately. Use this, not execute_code/execute_code_stream/session_run, when you do not want to hold the call open — poll run_inspect(run_id), and run_cancel(run_id) to stop it early.
Same request shape as execute_code minus session_id (a run is a
standalone process, not a session workspace). timeout bounds the WORK
itself, not how long you wait to collect it.
This call's own reply carries no output — a small run_id handle — so
the anthropic/maxResultSizeChars hint lives on run_inspect instead,
which returns the full envelope, same shape execute_code returns,
once the run lands.
Admission is capped (CODECALC_MAX_ACTIVE_RUNS, default 64): past that many runs still running/cancelling at once, this returns a resource_exhausted error rather than growing without bound — call run_inspect/run_cancel to make room, or wait for one to finish.
Retention: see run_inspect.
dependencies: same semantics as execute_code's own. A refusal is
returned directly with no run created. Otherwise this call still
returns immediately: the install itself runs on the background worker,
ahead of the code, and a failed install becomes the run's own terminal
error — readable via run_inspect(run_id) like any other outcome.
| Name | Required | Description | Default |
|---|---|---|---|
| code | Yes | Source code to run in `language` | |
| stdin | No | Text piped to the program's standard input; empty means no input | |
| no_net | No | Block outbound network access for this run; best-effort on platforms without seccomp | |
| max_cpu | No | Per-call CPU-time ceiling in seconds; 0 means no explicit limit is set | |
| timeout | No | Wall-clock seconds before the run is killed; clamped to a 120s ceiling, same as execute_code | |
| language | Yes | Runtime to execute in, e.g. 'python3', 'node'; see list_languages for the full catalog | |
| provider | No | Execution backend id to use (see list_execution_providers); default picks automatically | |
| dependencies | No | Packages to install before running, e.g. ['requests==2.31.0']; merged with any PEP 723 block | |
| max_memory_mb | No | Per-call memory ceiling in MiB; 0 means no explicit limit is set | |
| max_output_kb | No | Stdout/stderr capture cap in KiB per stream; 0 uses the 64 KiB default, hard-clamped to 240 |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses much more than the annotations: background execution, immediate return, timeout semantics, admission cap, dependency installation behavior, and the fact that the initial reply carries no output. It also explains that a resource_exhausted error can occur and how to resolve it. No contradictions with the extended coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is organized in clear, front-loaded paragraphs that show the causal narrative. It starts with the core purpose, then usage guidance, then execution/response nuances, then capacity, and then dependencies. Each sentence carries meaningful information, with no repetitive filler or redundancy. The length is justified by the complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the complexity of the tool (10 parameters, asynchronous execution, related reserve), the description covers all essential aspects: what the tool does, when to use it, how to process the result, how to cancel, what happens on failure (install fails on worker), capacity limits, and the relationship to output via run_inspect. The output schema provided handles the return shape documentation; the description does not need to restate it. An agent has everything needed to invoke and reason about this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers all 10 parameters (100% coverage), so the baseline is 3. The description adds value by explaining that 'timeout' bounds the work itself not the collection time, that 'dependencies' follows execute_code semantics and triggers runtime installs, and that the call shape is the same as execute_code minus session_id. These semantic details go beyond the schema descriptions, so a 4 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The opening sentence states a clear specific action: 'Submit code for BACKGROUND execution; returns a run_id immediately.' It names the exact verb, resource, and the key return value. It also explicitly contrasts itself with execute_code, execute_code_stream, and session_run, making the distinction obvious without needing to open other schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit when-to-use guidance: 'Use this, not execute_code/execute_code_stream/session_run, when you do not want to hold the call open.' It also tells the agent exactly what to do next (poll run_inspect, run_cancel) and describes the request-shape relationship with execute_code. Nothing is left to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Check every language runtime for available updates (NON-MUTATING).
Reports current vs latest version per language, which package manager owns it (mise/rustup/swiftly/apt/npm/uv), and the exact command that would run.
Each entry also carries tier (registry.RELIABILITY_TIERS) — see
list_languages for what tested/best_effort/plan_only mean. A
current, up-to-date toolchain can still be best_effort: tier is
orthogonal to whether the update check below found a newer version.
NETWORK: yes. Non-mutating refers to this machine's runtimes, not to traffic — each package manager is asked what the latest version is, and they answer by contacting their own remote index.
| Name | Required | Description | Default |
|---|---|---|---|
| languages | No | Comma-separated languages to check, e.g. 'python3,node,rust'; empty checks all |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark read-only/idempotent/non-destructive, and the description goes well beyond by disclosing that the tool makes network calls, that each package manager contacts its remote index, and that non-mutating refers to local runtimes, not traffic. It also clarifies that `tier` is unrelated to update availability.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The most important fact (non-mutating) is front-loaded in the first sentence nested in parentheses. Every subsequent sentence adds distinct value: output contents, tier semantics, network behavior, and clarification of what non-mutating means. Nothing is redundant.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema and a single self-describing parameter, the description provides all necessary operational context: output fields, tier cross-reference, network behavior, and non-mutating semantics. No critical behavioral or selection information is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema already covers the single parameter with a default and example, so the baseline is high. The description reinforces meaning by saying output is per language but adds no new parameter-level detail, which is acceptable because the schema is sufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description leads with a specific verb and resource: 'Check every language runtime for available updates'. It enumerates the output contents (current vs latest version, owning package manager, exact command, tier), which makes the tool's function unmistakable and distinguishes it from a mutating update tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly frames the tool as a non-mutating check of all language runtimes for available updates, which implicitly tells an agent when to use it. It does not explicitly name the alternative update_runtimes, though the phrase 'NON-MUTATING' and 'exact command that would run' strongly imply the comparison.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List files created by executed code in a session (excluding runner internals like main.py/run.out).
| Name | Required | Description | Default |
|---|---|---|---|
| session_id | Yes | Id of the session whose executed-code output files to list |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds useful content-filter context by excluding runner internals like main.py/run.out, but it does not describe output shape or pagination; the output schema helps fill that gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, front-loaded with the action and resource, and the exclusion note is compact and useful. There is no wasted text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a simple, read-only, one-parameter listing tool with rich annotations and an output schema. The description states what is included and what is excluded, which is enough for an agent to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the single session_id parameter is already described in the schema. The description does not add extra parameter-level meaning beyond tying the listing to a session.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('List') and names the resource ('files created by executed code in a session') while also noting exclusions like runner internals. This is clear and actionable, but it does not explicitly differentiate from the sibling session_files tool, so it stops short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool: after code execution, when you need files produced by the code. However, it gives no explicit when-not-to-use guidance or alternatives such as session_files for a broader file listing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List workspace files, optionally using a bounded cursor page. Use session_artifacts, not this, for only the files executed code produced; use session_read_file for one file's contents.
| Name | Required | Description | Default |
|---|---|---|---|
| path | No | Subdirectory to list, relative to the workspace root; empty lists the root | |
| cursor | No | Opaque page cursor from a previous session_files call's response, to fetch the next page | |
| page_size | No | Max entries per page; omit for one unpaginated listing | |
| session_id | Yes | Id of the session whose workspace files to list |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds useful behavioral context: it mentions the optional bounded cursor page and clarifies that omitting page_size yields one unpaginated listing. It doesn't describe pagination details or return format, but the output schema exists and the annotations carry the safety burden, so a 4 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with zero waste. The core action is front-loaded, and the sibling routing is compact and clear. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only listing tool with full schema coverage and an output schema, the description is nearly complete. It covers the main use case, pagination behavior, and sibling distinctions. The only minor gap is that it doesn't describe the shape of the returned listing, but the output schema likely covers that, so this is not a significant omission.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all four parameters. The description adds a little context by mentioning the cursor page and unpaginated listing, but it doesn't add meaning beyond what the schema provides. Baseline 3 is correct.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('List') and resource ('workspace files'), and immediately distinguishes itself from sibling tools: 'Use session_artifacts, not this, for only the files executed code produced; use session_read_file for one file's contents.' This makes the tool's purpose and boundaries clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells the agent when NOT to use this tool and names the alternatives: session_artifacts for code-produced files and session_read_file for single-file contents. This is direct routing guidance that leaves no ambiguity about selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List active sessions and their languages/state.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is fully covered. The description adds no behavioral details beyond the output fields, which is acceptable given the annotations but does not go further.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no redundant wording. It efficiently conveys the purpose and key output characteristics.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter read-only listing tool, the description is sufficiently complete. It mentions the output fields 'languages/state' and clearly identifies the resource. A more detailed output schema is not included, but the description covers the essential context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, so there are no parameter semantics to document. The schema coverage is trivially complete, and the description does not need to add parameter-level detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists active sessions and includes the key output attributes (languages/state), making the purpose unambiguous. The verb 'list' and resource 'active sessions' precisely define the action and target.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool versus related session tools such as session_read_file or session_run. However, the simple action of listing active sessions is self-explanatory, so some guidance is implied but not stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Read a file from a session workspace.
Text files return content. With as_image=True (or for image files), the file is returned as an inline image the model can see. Use session_files to discover paths; session_artifacts lists what executed code produced.
| Name | Required | Description | Default |
|---|---|---|---|
| path | Yes | Relative path inside the workspace to read | |
| as_image | No | Return the file as an inline image the model can see, instead of text | |
| max_bytes | No | Max bytes to read from the file; default 65536 (64 KiB) | |
| session_id | Yes | Id of the session whose workspace file to read |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is covered. The description adds behavioral detail beyond annotations by explaining that text files return content and that image files or as_image=True produce an inline image the model can see. This is useful context not present in the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short, dense sentences. The purpose is front-loaded, then the image behavior, then sibling references. No redundant phrasing or fluff—every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the annotations and full schema coverage, the description is complete for an agent to call the tool correctly. It covers what returns (text or image), how to find paths (session_files), and what artifacts are (session_artifacts). No critical information is missing for a read-only tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and all four parameters have descriptions. The description adds context by explaining the as_image parameter ('returned as an inline image the model can see') and implicitly ties the path parameter to session_files discovery. It doesn't over-explain, but the schema already does the heavy lifting, so a baseline of 3 plus this added clarity justifies a 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Read a file from a session workspace') and distinguishes itself from sibling tools by explicitly referencing session_files for path discovery and session_artifacts for executed-code output. It also clarifies the two return modes (text vs inline image), making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit routing guidance: 'Use session_files to discover paths; session_artifacts lists what executed code produced.' It also specifies when as_image=True is appropriate (for image files or when the model needs to see the content as an image). This directly tells the agent when to use this tool vs its siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run a multi-file program already written into a session workspace
(via session_write_file). Use this, not
execute_code/execute_code_stream/run_submit, when entry_file may
import other files already in that workspace (helper.py, data/...).
Runs as a fresh process in the session workdir (not the REPL worker), so
relative imports and data files resolve. Returns stdout/stderr/verdict
plus the entry file's path. Oversized output spills into the session
workspace the same way execute_code's does — see its docstring for
stdout_spill/stderr_spill.
Reports artifacts_created and inlines small ones as extra content
blocks (image/text/link), capped at 8 blocks / 4 MiB encoded;
truncated_inline: true past either cap. dependencies installs
packages before running, same rule as execute_code's — see its docstring.
This tool takes no max_output_kb (its inline stdout/stderr stay at the
64 KiB default and spill past that, same as execute_code's session
branch) but the anthropic/maxResultSizeChars _meta it advertises
covers only that text envelope — the JSON result serialized as the
reply's content text block. The inlined artifact blocks above
(image/text/link, up to 8 of them within the 4 MiB encoded budget) are
SEPARATE MCP content blocks, outside the text block this hint bounds.
Every run copies entry_file's own source into the runner's private
scratch subdirectory before executing it — never into a root-level
main.<ext> file a session's own files could collide with. A session's
own main.py (or the equivalent for another language) at the session
root is never touched by running a different entry file.
| Name | Required | Description | Default |
|---|---|---|---|
| stdin | No | Text piped to the program's standard input; empty means no input | |
| timeout | No | Wall-clock seconds before the run is killed | |
| language | No | Language to run `entry_file` as; omit to infer it from the session/file | |
| entry_file | Yes | Relative path of the file to execute; may import other files already in the workspace | |
| session_id | Yes | Id of the session workspace to run in | |
| dependencies | No | Packages to install before running, e.g. ['requests==2.31.0'] |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes well beyond annotations, explaining that execution happens in a fresh process in the session workdir, not the REPL worker, and that the entry file is copied into a private scratch subdirectory to avoid collisions with root-level main files. It also details output spill behavior, inline artifact caps, and the exact scope of the maxResultSizeChars hint, all non-obvious behavioral facts an agent needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long, but almost every sentence carries a distinct behavioral fact that is not visible from the schema. It is front-loaded with purpose and usage, then covers execution semantics, output behavior, and file-safety guarantees. It could be tightened, but for a complex tool the density is justified.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description still covers what the tool returns (stdout/stderr/verdict, entry file path, artifacts_created), how truncation is signaled, how dependencies behave, and how output can appear outside the main text envelope. Given the tool's complexity and the absence of an output schema, this is remarkably complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters. The description adds meaningful context beyond that by clarifying that entry_file may import other workspace files, and that dependencies installs packages before running with the same rule as execute_code, enriching the agent's understanding without repeating the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Run a multi-file program already written into a session workspace.' It also names the sibling tools it should not be confused with (execute_code/execute_code_stream/run_submit), making the tool's identity and scope immediately clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is explicitly conditioned: 'Use this, not execute_code/execute_code_stream/run_submit, when `entry_file` may import other files already in that workspace.' This gives the agent a concrete decision rule and names the alternatives, which is exactly the guidance needed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Archive or restore a session's workspace files. action:
"save": tar.gz the session's current files (same rules as session_artifacts: .codecalc-run/ excluded, symlinks/hardlinks refused) into a snapshot stored OUTSIDE the workspace, so sandboxed code can never read or tamper with it. Returns snapshot_id.
"restore": extract snapshot_id's files into a brand-new session (default) or, with replace=True, wipe and recreate session_id's OWN workspace first. Only files are restored — a python3/node session's REPL variables/imports are never part of a snapshot.
"list": snapshots saved for session_id, oldest first.
"delete": remove one snapshot (snapshot_id required).
Snapshots are deleted when their session is stopped (session_stop(keep_snapshots=True) to keep them).
| Name | Required | Description | Default |
|---|---|---|---|
| label | No | Optional human-readable label to store with a new snapshot; only used by action='save' | |
| action | No | One of 'save', 'restore', 'list', 'delete'; default 'save' archives the workspace | save |
| replace | No | For action='restore', wipe and reuse session_id's own workspace instead of creating a new session | |
| session_id | Yes | Id of the session the snapshot belongs to or is restored into | |
| snapshot_id | No | Id of an existing snapshot; required for action='restore' or action='delete' |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses key side-effects and constraints beyond annotations: snapshots are stored outside the workspace (so sandboxed code cannot read/tamper), restore never includes REPL variables/imports, and snapshots are deleted when their session stops. This adds context to the destructiveHint annotation and clarifies scope of restore.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact yet precise, using a bulleted action list to organize behaviors without redundancy. It packs essential details (exclusions, storage location, lifecycle) into minimal prose, making it easy for an agent to parse and apply.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the full lifecycle, integration with sibling tools (session_artifacts, session_stop), and edge cases (restore into new vs existing session). With an output schema present and high schema coverage, no information an agent needs to call this tool correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All five parameters are fully described in the schema, including conditional requirements (snapshot_id for restore/delete), defaults (action='save', replace=False), and scope (label only for save). The description reinforces these by tying each parameter to specific actions, ensuring no ambiguity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the tool archives or restores session workspace files and enumerates four actions (save, restore, list, delete) with distinct behaviors. It clearly distinguishes from sibling tools like session_artifacts by referencing its rules, and from session_stop by referencing snapshot lifecycle.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit action-specific guidance: when to use save (with exclusions), restore (default new session vs replace=True), list (snapshots for session), and delete (requires snapshot_id). It also notes the alternative session_stop(keep_snapshots=True) for preserving snapshots, giving clear routing between actions and related tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Start a persistent session. python3/node get a stateful REPL worker (variables/imports persist across execute_code calls); other languages get a persistent workspace directory. Returns session_id.
| Name | Required | Description | Default |
|---|---|---|---|
| language | No | Language for the new session's worker/workspace; default 'python3' gets a stateful REPL | python3 |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=false (not read-only) and destructiveHint=false; the description adds concrete behavior: persistence across execute_code calls and language-specific workspace creation. It does not explicitly warn about resource lifecycle, but the persistent aspect is transparently stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no filler. The primary effect (persistent session) is front-loaded, and language-specific details and return value are packed efficiently. Every clause adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With one optional parameter, a generated output schema, and no nested objects, the description is complete for calling this tool correctly. It explains what is returned (session_id) and the exact behavior for the parameter, leaving nothing essential missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already provides 100% coverage for the single 'language' parameter, including its default. The description enriches semantics by explicitly stating that python3/node get a stateful REPL while other languages get a persistent workspace, adding meaning beyond the schema's generic wording.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Start a persistent session.' It clarifies language-specific behavior (python3/node get stateful REPL worker; other languages get a persistent workspace directory) and states it returns session_id, fully distinguishing it from siblings like session_stop or execute_code.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It implies when to use this tool by noting variables/imports persist across execute_code calls and mentions persistent workspace, indicating it should be used when cross-call state is needed. It does not explicitly state when not to use it (e.g., one-off code runs), but the distinction from execute_code is reasonably clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Stop a session: kill its REPL worker (if any) and delete its workspace. Also deletes every session_snapshot saved for it, unless keep_snapshots=True.
| Name | Required | Description | Default |
|---|---|---|---|
| session_id | Yes | Id of the session to stop, as returned by session_start | |
| keep_snapshots | No | Keep this session's saved session_snapshot archives instead of deleting them |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already mark the tool as destructive and idempotent, and the description adds concrete behavioral detail: it kills the REPL worker, deletes the workspace, and removes every session_snapshot unless keep_snapshots=True. This gives the agent a clear model of the irreversible side effects and the exception to them. There is no contradiction with the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences with no filler: the main action is front-loaded, and every subsequent clause adds operational information about scope or exceptions. The conditional 'if any' and the keep_snapshots caveat are both essential to safe invocation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a simple two-parameter tool with full schema coverage and annotations already declaring destructiveness and idempotence, the description covers the remaining operational essentials: what gets deleted, when deletion can be avoided, and the terminal nature of the operation. Nothing an agent needs to decide whether and how to call it is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents both parameters at 100% coverage, so the description does not need to re-explain them. It adds value by clarifying the behavioral consequence of keep_snapshots—that snapshots are otherwise deleted—and by tying session_id to the session whose worker and workspace are destroyed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description leads with a specific verb and resource: 'Stop a session' and then enumerates exactly what stopping entails—killing the REPL worker and deleting the workspace. This clearly distinguishes it from sibling tools like session_list or session_start, so an agent can identify the right operation without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description makes the core use case obvious but does not explicitly state when to use it versus alternatives, such as listing sessions or starting a new one. The destructive language strongly implies this is the terminal cleanup action, but the agent must infer that rather than being told. No when-not-to-use or exclusion guidance is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Write a file into a session workspace (relative path, no escapes). Use this to seed input data for executed code.
| Name | Required | Description | Default |
|---|---|---|---|
| path | Yes | Relative destination path inside the workspace; path escapes (e.g. '../') are refused | |
| content | Yes | Text content to write to `path`, overwriting any existing file | |
| session_id | Yes | Id of the session workspace to write into |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true and idempotentHint=true. The description adds useful behavioral context: it overwrites existing files (via the schema's content description) and refuses path escapes. It doesn't contradict annotations. It could mention that overwriting is destructive, but the annotation covers that.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with zero waste. The core action and constraint are front-loaded, and the use case is stated in the second sentence. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple write tool with full schema coverage and an output schema, the description is complete enough. It covers the action, the constraint, and the use case. It doesn't explain return values, but the output schema exists, so that's not required.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all three parameters. The description adds the 'relative path, no escapes' constraint, which reinforces the path parameter's schema description. Baseline 3 is appropriate since the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Write'), a specific resource ('a file into a session workspace'), and a key constraint ('relative path, no escapes'). It clearly distinguishes this from sibling tools like session_read_file and session_files by focusing on the write action.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a clear use case: 'Use this to seed input data for executed code.' This tells the agent when to use it. It doesn't explicitly name alternatives or exclusions, but the context is clear enough given the sibling list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Symbolic algebra, selected by op — replaces the four former
standalone tools solve_expression, solve_linear, simplify_expression
and limit_expression, retired in 0.12.0 (CHANGELOG.md). Every op
returns exactly its former tool's own result, plus op (additive).
op="solve" (was solve_expression) — the roots of one equation:
'x**2 - 4 = 0', '2*x + 1 = 7'. For a system of several equations, use
op="solve_linear". For general constraint satisfiability (inequalities,
boolean constraints, multiple solvers), use z3_check. Returns
solutions as a list of strings alongside the parsed equation and
variable. Used by this op: expr (required), var (optional).
op="solve_linear" (was solve_linear) — a system of equations sharing
variables. Example: system='x + y = 10; x - y = 2', variables='x, y'.
Used by this op: system, variables (both required).
op="simplify" (was simplify_expression) — simplify, factor, and expand
an expression — algebraic forms, not solving (use op="solve") and not a
numeric value (use calc_exact). Returns simplified, factored, and
expanded as strings alongside the parsed original. Used by this op:
expr (required).
op="limit" (was limit_expression) — asymptotic behaviour: limit of
expr as var -> point. 'symbolic("limit", "n*log(n)/n**2", "n")'
returns 0 — settles complexity arguments faster than arguing. Used by
this op: expr (required), var (optional), point (optional).
| Name | Required | Description | Default |
|---|---|---|---|
| op | Yes | Which symbolic operation to run: 'solve', 'solve_linear', 'simplify', or 'limit' (each has its own required params) | |
| var | No | Variable to solve for or take the limit over; optional, default 'x'; used by op='solve'/'limit' | |
| expr | No | Expression or equation to solve/simplify/take the limit of; required by op='solve'/'simplify'/'limit' | |
| point | No | Point `var` approaches for op='limit'; optional, default 'oo' (infinity) | |
| system | No | ';'-separated equations for op='solve_linear', e.g. 'x + y = 10; x - y = 2'; required by that op | |
| variables | No | Comma-separated variable names for op='solve_linear', e.g. 'x, y'; required by that op |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly=true, destructive=false, and idempotent=true, so the safety profile is covered. The description adds meaningful behavioral detail beyond that: each op returns the same result as the retired standalone tool plus an additive op field, solve accepts equations/expressions, and limit has a default 'oo'. It stops short of documenting error cases (e.g., no real roots, malformed expression, unsupported syntax), so it is not a 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is information-dense and its per-op parallel structure makes it scannable, but it is long and partially restates what the schema already says for each parameter. For a 6-parameter tool this length is mostly justified; it earns a middle score rather than a higher one because some sentences repeat schema content rather than adding new guidance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex op-dispatch tool, the description covers operation selection, parameter requirements per op, input formats, worked examples, and migration from the former tools. Remaining gaps are minor: no mention of multiple solutions, no solution, or parse-error behavior. Together with the 100% schema coverage and output schema, this is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, providing a baseline of 3; the description adds real value on top: concrete equation strings ('2*x + 1 = 0'), the exact system syntax for solve_linear ('x + y = 10; x - y = 2'), which params each op requires vs. treats as optional, and the default variable/point behavior. This reduces ambiguity beyond what the schema alone offers.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens by identifying the tool as a symbolic-math dispatcher with four named operations (solve, simplify, factor, limit), each paired with a concrete verb and resource. It also situates itself against the retired standalone tools and names sibling tools like z3_check, so an agent can immediately tell what the tool does and how it differs from alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives per-op selection guidance: use solve for equations, simplify/factor for expressions, limit for limits, and explicitly redirects non-symbolic numeric work to calc_exact and constraint solving to z3_check. It also names the four former standalone tools that this dispatch tool replaces, making the migration path and op-selection criteria unambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Debug WHY, line by line, for the ONE input you actually ran it on: which statements fired, in what order, with what variable values at each step, and which if/elif/while/for/try branch was taken versus never taken. Want just the printed output instead? Use execute_code.
Returns events: ordered {step, line, event, func, locals}, one entry
per traced line/call/return/exception in YOUR code only (library
internals excluded). locals on each entry is only the names that
changed since the previous step in that same call — not a full dump
every line. A return entry also carries return_value; an
exception entry carries exception_type/exception_message.
Also returns branches (hit count per if/elif/while/for/try line),
lines_executed / lines_never_executed (coverage from a static parse),
and truncated/truncated_reason when max_events or an internal
size ceiling stopped RECORDING early (the underlying stdout/exit code
are unaffected either way).
TRUST: the trace is produced BY the traced program at its OWN privilege
— a debugging aid, not an attestation of behaviour, exactly as
trustworthy as that program's own stdout. discarded_events /
events_consistent are a best-effort tamper/corruption signal (never a
guarantee) computed independently of the file's own content.
unenforced may additionally note "only the main thread is traced"
(sys.settrace is per-thread) or, fallback backend only, an OLE
exit_code race.
For a structural Big-O guess with nothing executed, use analyze_complexity.
| Name | Required | Description | Default |
|---|---|---|---|
| code | Yes | Source code to trace line by line | |
| stdin | No | Text piped to the program's standard input; empty means no input | |
| no_net | No | Block outbound network access for this run; best-effort on platforms without seccomp | |
| max_cpu | No | Per-call CPU-time ceiling in seconds; 0 means no explicit limit is set | |
| timeout | No | Wall-clock seconds before the run is killed; clamped to a 120s ceiling | |
| language | Yes | Runtime to trace; only 'python3' is supported, any other value is refused | |
| provider | No | Execution backend id; only 'local' (the default) is supported here | |
| max_events | No | Max trace events to record before truncating; the run's own stdout/exit code are unaffected | |
| max_memory_mb | No | Per-call memory ceiling in MiB; 0 means no explicit limit is set | |
| max_output_kb | No | Stdout/stderr capture cap in KiB per stream; 0 uses the 64 KiB default, hard-clamped to 240 |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Even with no annotations carrying a safety profile, the description goes deep: it discloses that only user code is traced (library internals excluded), that `locals` only includes changed names, that `truncated` stops recording but leaves stdout/exit code unaffected, and the TRUST section warns the trace is not an attestation and that only the main thread may be traced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is substantial but well-organized into paragraphs: the core behavior, return details, then TRUST limitations. No redundant sentences exist, but the TRUST section is long and could be tightened without sacrificing value. It's slightly verbose, but every sentence does carry content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex tool with ten parameters and an output schema, the description covers the outcome shape (`events`, `branches`, `lines_executed`, etc.), the semantics of truncation, and honest limitations like thread-only tracing and fallback backend races. An agent has enough to call the tool correctly and interpret its output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema alone defines the parameters. The description adds behavior context for `max_events` (recording truncates, stdout/exit unaffected) and mentions `max_events`/`max_cpu`/`max_memory_mb` semantics implicitly. It doesn't fully explain every property (e.g., `provider`) but those are already described in the schema, so the increment over the schema is meaningful though not exhaustive.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool 'Debug WHY, line by line' and specifies it traces the one input you ran it on, showing statements fired, order, variable values, and branches taken. It explicitly contrasts with execute_code (printed output) and analyze_complexity (Big-O guess), so an agent can distinguish it from relevant siblings without opening schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes direct 'when-not' guidance: 'Want just the printed output instead? Use execute_code' and 'For a structural Big-O guess with nothing executed, use analyze_complexity.' It also implies when to use (debugging line-by-line behavior) and clearly separates from passive analysis.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Build the truth table for a boolean expression over and/or/not/xor/ implies/iff (plus true/false constants and variables): 'a and b or not c', 'p xor q', 'a implies b'. Use z3_check, not this, for satisfiability over inequalities or non-boolean variables; use evaluate_expression for symbolic (non-boolean) math.
Returns variables (sorted names) and rows (one dict per assignment,
each variable name -> bool plus result), plus row_count,
satisfiable (any row true), and tautology (every row true).
| Name | Required | Description | Default |
|---|---|---|---|
| expression | Yes | Boolean logic expression to tabulate, e.g. 'a and b or not c', 'p xor q', 'a implies b' |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds value by specifying the output shape (variables, rows with each variable plus result, row_count, satisfiable, tautology) and the meaning of the summary booleans, which are not stated in the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences: a clear definition with examples, an explicit when-not-to-use, and a compact rundown of the return fields. Every sentence earns its place; there is no filler or redundancy with the schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool that only accepts one expression string and has an output schema, the description fully covers what an agent needs: domain of valid input, the output object's semantics, and routing to similarly named tools. The return value and the meaning of satisfiable/tautology are explicitly explained, so nothing essential is left out.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 100% coverage and describes the expression parameter with examples, so the baseline is 3. The description goes further by defining the grammar of accepted expressions (allowed operators, true/false constants, variables) and reinforcing with examples, which adds meaning beyond the schema's one-line description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb-resource pairing, 'Build the truth table for a boolean expression', and enumerates the supported operators (and/or/not/xor/implies/iff) with three concrete examples. It also explicitly names sibling tools it is not, differentiating itself from z3_check and evaluate_expression at a glance.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit usage exclusions: 'Use z3_check, not this, for satisfiability over inequalities or non-boolean variables; use evaluate_expression for symbolic (non-boolean) math.' This directly instructs when to avoid the tool and directs the agent to the correct sibling in those cases, going beyond vague contextual hints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Update language runtimes. SAFE BY DEFAULT: with apply=False this is a dry run — it returns the update commands that WOULD run without changing anything. Pass apply=True to actually execute them (mise up, rustup update, swiftly update, apt-get upgrade of language packages, npm -g update, uv tool upgrade).
apply=True asks the caller to confirm first (a protocol-level gate, not
just the anthropic/requiresUserInteraction _meta hint — see
codecalc/confirmation.py); apply=False is never gated, since nothing runs.
PRIVILEGE: the apt manager updates system packages and its command begins
with sudo. Those commands do NOT run unless the HOST has set
CODECALC_ALLOW_RUNTIME_APPLY=1; without it they are reported as skipped
with ok: false and the variable named, and the rest still run. Every
entry carries an elevated flag either way. mise/rustup/swiftly/npm/uv
touch user-owned toolchains and are never gated.
NETWORK: yes, on both paths. apply=False still asks each manager what the latest version is, which is a remote lookup; apply=True additionally downloads and installs. "Dry run" bounds what changes on disk, not what is sent.
| Name | Required | Description | Default |
|---|---|---|---|
| apply | No | False (default) is a dry run reporting commands only; True actually runs them and asks for confirmation first | |
| timeout | No | Wall-clock seconds allowed for the update commands to complete | |
| languages | No | Comma-separated languages to update, e.g. 'python3,node,rust'; empty updates all |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations (destructiveHint=true, readOnlyHint=false), the description discloses materially more: the CODECALC_ALLOW_RUNTIME_APPLY=1 gate for sudo commands, the protocol-level confirmation on apply=True, and the critical network caveat that dry-run still performs remote lookups. It flags that failed privileged commands return 'ok: false' and an 'elevated' flag. This context substantially exceeds what annotations alone convey, and it is consistent with them — no contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but the length is earned: it is a privileged, mutating, network-active tool whose complexity demands the SAFE BY DEFAULT, PRIVILEGE, and NETWORK sections. It is well structured with clear section headers and front-loads the safety default. It borders on verbose but no sentence is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool of this complexity — mutation, elevation, gating, network side effects, and dual modes — the description covers every operational concern an agent needs: what commands run, when confirmation is required, when commands are skipped (including the variable name), and what 'dry run' does and does not bound. The presence of an output schema relieves it of describing return values, and nothing critical is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents all three parameters (apply, timeout, languages) with defaults and types. The description reinforces apply's dry-run semantics and adds color about which managers map to which languages, but it does not meaningfully extend what the schema provides. Baseline 3 is appropriate since the schema carries the load.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a precise verb+resource statement, 'Update language runtimes,' then enumerates the exact underlying managers (mise, rustup, swiftly, apt, npm, uv). It distinguishes the two execution modes (dry-run vs apply) and is clearly separable from the sibling runtimes_status, which reports status rather than mutating. The purpose is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives strong internal usage guidance: when to use apply=False (safe dry run), when to use apply=True (actual execution with a confirmation gate), and the privilege conditions under which sudo commands do or do not run. It does not explicitly name sibling tools to avoid (e.g., install_package for single packages or runtimes_status for read-only checks), leaving those exclusions implied rather than stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
PROVE an optimisation: same outputs, measurably AND SIGNIFICANTLY faster.
Two gates, in order. Correctness: runs candidate against original on
shared inputs — a faster-but-wrong candidate fails here and is never
timed. Speed: times both at increasing sizes; accepts only when the
median ratio clears min_speedup AND a one-sided Mann-Whitney U test
rejects "not faster" at every counted size (2-3), or a
Bonferroni-corrected majority above that — one size never accepts alone.
See inference for the per-size U statistic, p-value, effect size.
A rejection names which gate failed and by how much, e.g. "correct, 1.3x median, but only 1/4 sizes significant."
Accepted grades cross_checked; any rejection — wrong, not faster
enough, not significant — grades ungraded: correctness alone earns no
grade for the speed claim this tool answers.
| Name | Required | Description | Default |
|---|---|---|---|
| sizes | No | Input sizes to time both programs at (2-3+ sizes needed for significance); omit for defaults | |
| language | Yes | Language both `original` and `candidate` are written in | |
| original | Yes | Baseline program to compare against | |
| candidate | Yes | Optimised version of `original`, to prove correct and measurably faster | |
| min_speedup | No | Minimum median speedup ratio required to accept the optimisation; default 1.15 (15% faster) | |
| test_inputs | No | Inputs to confirm both programs still agree on; omit to use the default set |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses far more than the sparse annotations provide (readOnlyHint: false, openWorldHint: true). It details the two-gate process, the Mann-Whitney U test, the Bonferroni correction, the acceptance threshold, and the grading consequences. It also gives a concrete rejection example. This is thorough behavioral disclosure that exceeds what annotations imply.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is relatively long but well-structured: a bold statement of purpose, then two gates, a rejection example, and grading outcomes. Every sentence adds operational detail; there is no fluff. The length is justified by the complexity of the statistical verification, though it could be tightened slightly without losing meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 6 parameters, an output schema, and a complex decision process, the description covers all necessary aspects: correctness gate, speed gate, statistical significance, grading behavior, and an example rejection message. The output schema handles the return format, so the description does not need to. Nothing an agent needs to call it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds significant meaning by explaining how parameters work together: 'runs candidate against original on shared inputs', 'times both at increasing sizes', and 'accepts only when the median ratio clears min_speedup'. It also clarifies the role of sizes and test_inputs, enriching the bare schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'PROVE an optimisation: same outputs, measurably AND SIGNIFICANTLY faster,' which names a specific verb (prove), resource (optimisation), and precise success criteria. This clearly separates it from siblings like benchmark (which only measures speed) or compare_execution (which likely checks equivalence without the statistical significance requirement).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool implicitly (when you have an original and an optimised candidate to verify) and details the process, but it does not explicitly name alternative tools or state when not to use this one. An agent could infer the usage, but there is no explicit guidance like 'use benchmark for speed-only comparisons'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
PROVE that a port is equivalent: run both programs, compare their output.
You write the translation — you are the language model. This runs your source and your port on the same inputs and reports, per input, whether they matched, diverged, or could not be compared (a runtime that is missing or a program that failed on both sides is INCONCLUSIVE, never a pass).
Use it after porting anything: python3 -> go, node -> rust, a rewritten function against the original. Pair with compare_edge_cases to find the inputs worth testing.
Matching tolerates only line-ending/trailing-whitespace noise; stdout_raw carries what actually ran.
A pass is graded cross_checked (two independent implementations, run
and agreeing — see grade_basis for which runtimes). A non-pass is
graded ungraded: never a softer positive grade.
| Name | Required | Description | Default |
|---|---|---|---|
| source_code | Yes | Original program, in `source_language` | |
| target_code | Yes | Ported program, in `target_language`, to check against `source_code` | |
| test_inputs | No | Inputs to run both programs on and compare; omit to use the default edge-case set | |
| source_language | Yes | Language of `source_code` | |
| target_language | Yes | Language of `target_code` |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations (openWorldHint, no readOnlyHint), the description discloses real behavioral detail: it runs both programs, reports per input as matched/diverged/inconclusive, and clarifies that a missing runtime or failure on both sides is inconclusive. It defines output tolerance and grading semantics—concrete behavior the agent wouldn't otherwise know.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Every sentence earns its place. The purpose is front-loaded in the first sentence, followed by a fundamental fairness that the agent writes the program, the execution model, when to use it, the tolerance rules, and grading. The all-caps emphasis on key terms is clean and helps scanning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists so return values don't need to be explained. The description covers the critical usage context (when, what input to provide, how output behaves, how grading works, and a companion tool for edge cases). It does not explicitly mention potential side effects of executing user code, but the shape of the tree, the instructions, and the open flow, the behavior is surrounded by enough care.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions already cover 100% of parameters. The description adds the meta note 'You write the translation — you are the language model', which clarifies the agent's responsibility for generating the target code but provides no extra semantic detail about the parameter format or ranges. This is the baseline for high schema-coverage tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States 'PROVE that a port is equivalent: run both programs, compare output' which is a specific verb and resource, and it emphasizes the translation scenario. It also gives concrete example pairs (python->go, node->rust), making the purpose unambiguous against other verification tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Use it after porting anything' and suggests pairing with compare_edge_cases for input selection. It clearly specifies the intended use-case, but it does not name the sibling 'verify_optimization' as an exclusion, leaving the differentiation between these two verify tools implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Use z3_check, not symbolic(op="solve"), for satisfiability over inequalities, boolean combinations, or several variables at once: sat/ unsat/unknown plus a model. Example: '(declare-const x Int)(assert (> x 5))(check-sat)'.
unsat is graded solver_proven — see grade_basis for the engine
version and timeout bound it was decided within. sat is graded
ungraded: it's a real decided answer, just not a proof — reserving
solver_proven for unsat means a counterexample can never wear a
proof grade. unknown carries no proof either way and is also graded
ungraded.
| Name | Required | Description | Default |
|---|---|---|---|
| smt2 | Yes | SMT-LIB2 script to check for satisfiability, e.g. '(declare-const x Int)(assert (> x 5))(check-sat)' |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes beyond the readOnly/idempotent annotations by explaining output semantics: sat/unsat/unknown plus a model, and the important grading distinction (unsat is a proof; sat and unknown are not). It also points to grade_basis for timeout/engine details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is slightly longer than strictly necessary but is front-loaded with the core purpose and example, then adds grading semantics. No wasted sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only, idempotent SMT check tool, the description fully covers when to use it, what it returns, how results are graded, and where to find engine details. An agent has enough context to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the smt2 parameter is already described with an example. The description restates this example but adds no substantial parameter-level meaning beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb and resource ('z3_check'), a concrete task (satisfiability over SMT-LIB inequalities/boolean combinations), and explicitly contrasts itself with symbolic(op='solve'). An agent can immediately tell what this tool does and what it is not for.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It says to use z3_check instead of symbolic(op='solve') for satisfiability checking and lists qualifying cases. It does not fully describe when symbolic should be used instead, but the key alternative and decision trigger are explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.12.0algebraic_equiv2 fields changedInput schema / properties / a / descriptionAdded value: +"First symbolic expression to compare for algebraic identity"Input schema / properties / b / descriptionAdded value: +"Second symbolic expression to compare for algebraic identity"analyze_complexity2 fields changedInput schema / properties / code / descriptionAdded value: +"Source code snippet to analyze structurally for its asymptotic time complexity"Input schema / properties / language / descriptionAdded value: +"Language `code` is written in; default 'python3'"base_reprbenchmark4 fields changedInput schema / properties / code / descriptionAdded value: +"Program that reads integer N from stdin's first line and does work sized by N"Input schema / properties / language / descriptionAdded value: +"Language `code` is written in; default 'python3'"Input schema / properties / sizes / descriptionAdded value: +"Comma-separated input sizes to run at, e.g. '100,1000,10000,100000'"Input schema / properties / timeout / descriptionAdded value: +"Wall-clock seconds allowed per size before that run is killed"bit_analysisbitopbits7 fields changedInput schema / properties / a / descriptionAdded value: +"First operand for mode='op'; required by that mode"Input schema / properties / align / descriptionAdded value: +"Alignment boundary for mode='analysis'; reports padding needed to reach it"Input schema / properties / b / descriptionAdded value: +"Second operand for mode='op'; required unless op='not'"Input schema / properties / mode / descriptionAdded value: +"Which fact/operation to compute: 'analysis', 'op', 'widths', or 'repr' (each has its own required params)"Input schema / properties / n / descriptionAdded value: +"The integer to inspect; required by modes 'analysis', 'widths', and 'repr'"Input schema / properties / op / descriptionAdded value: +"Bit operation for mode='op': and/or/xor/nand/nor/xnor/not/shl/shr/sar/rol/ror"Input schema / properties / width / descriptionAdded value: +"Bit width for mode='op' (8/16/32/64, default 64) or mode='repr' (omit to skip width analysis)"branch_reachability5 fields changedInput schema / properties / code / descriptionAdded value: +"Python3 function source to analyze for reachable/dead branches"Input schema / properties / inputs / descriptionAdded value: +"Parameter name -> 'int'/'bool'/'str', to narrow or override an unannotated parameter's inferred type"Input schema / properties / language / descriptionAdded value: +"Source language of `code`; only python3 functions are analyzed"Input schema / properties / max_branches / descriptionAdded value: +"Max branches to analyze before stopping; default 64"Input schema / properties / timeout / descriptionAdded value: +"Wall-clock seconds before the z3 solver call is abandoned"calc_exact1 field changedInput schema / properties / expr / descriptionAdded value: +"Literal arithmetic expression with no symbols, e.g. '2**64 - 1', 'comb(52,5)', '0.1+0.2 == 0.3'"calc_stats1 field changedInput schema / properties / nums / descriptionAdded value: +"Sample of numbers to summarize (mean, median, sample stdev, coefficient of variation)"collision_probability2 fields changedInput schema / properties / bits / descriptionAdded value: +"Width of the hash in bits, e.g. 32, 64, 128"Input schema / properties / items / descriptionAdded value: +"Number of items being hashed"compare_edge_cases2 fields changedInput schema / properties / inputs / descriptionAdded value: +"Inputs to run every snippet on; omit for the default set covering empty/zero/negative/float cases"Input schema / properties / snippets / descriptionAdded value: +"Language name -> code; provide one correct snippet per language, implementing the same logic"compare_execution4 fields changedInput schema / properties / dependencies / descriptionAdded value: +"Not supported here; any truthy value is refused — install packages beforehand instead"Input schema / properties / snippets / descriptionAdded value: +"Language name -> code; each snippet must be a complete, valid program in its own language"Input schema / properties / stdin / descriptionAdded value: +"Text piped to every snippet's standard input; empty means no input"Input schema / properties / timeout / descriptionAdded value: +"Wall-clock seconds allowed per language before that run is killed"compare_threshold3 fields changedInput schema / properties / a / descriptionAdded value: +"Left-hand numeric expression, evaluated exactly"Input schema / properties / b / descriptionAdded value: +"Right-hand numeric expression, evaluated exactly"Input schema / properties / op / descriptionAdded value: +"Comparison operator: one of ==, !=, >, >=, <, <= ('=' also accepted for ==)"convert_units3 fields changedInput schema / properties / from_unit / descriptionAdded value: +"Source unit alias, e.g. 'mph', 'celsius', 'gb'; see list_units for all aliases"Input schema / properties / to_unit / descriptionAdded value: +"Target unit alias, e.g. 'km/h', 'fahrenheit', 'mib'; see list_units for all aliases"Input schema / properties / value / descriptionAdded value: +"Numeric quantity to convert, in `from_unit`"data_sizes1 field changedInput schema / properties / n / descriptionAdded value: +"Byte count to express in both binary (KiB/MiB/GiB/TiB, /1024) and decimal (KB/MB/GB/TB, /1000) units"epoch_time1 field changedInput schema / properties / n / descriptionAdded value: +"Epoch timestamp to convert to ISO 8601 UTC; units (seconds/millis/micros/nanos) are inferred from magnitude"evaluate_expression1 field changedInput schema / properties / expression / descriptionAdded value: +"Symbolic math expression to evaluate via SymPy, e.g. 'integrate(x**2, x)', 'sqrt(144) + 2**10'"execute_code12 fields changedInput schema / properties / code / descriptionAdded value: +"Source code to run in `language`"Input schema / properties / compact / descriptionAdded value: +"Drop diagnostic fields (timings, workdir, platform) from the result; safety disclosures are always kept"Input schema / properties / dependencies / descriptionAdded value: +"Packages to install before running, e.g. ['requests==2.31.0']; merged with any PEP 723 block"Input schema / properties / language / descriptionAdded value: +"Runtime to execute in, e.g. 'python3', 'node'; see list_languages for the full catalog"Input schema / properties / max_cpu / descriptionAdded value: +"Per-call CPU-time ceiling in seconds; 0 means no explicit limit is set"Input schema / properties / max_memory_mb / descriptionAdded value: +"Per-call memory ceiling in MiB; 0 means no explicit limit is set"Input schema / properties / max_output_kb / descriptionAdded value: +"Stdout/stderr capture cap in KiB per stream; 0 uses the 64 KiB default, hard-clamped to 240"Input schema / properties / no_net / descriptionAdded value: +"Block outbound network access for this run; best-effort on platforms without seccomp"Input schema / properties / provider / descriptionAdded value: +"Execution backend id to use (see list_execution_providers); default picks automatically"Input schema / properties / session_id / descriptionAdded value: +"Run inside this session's workspace (from session_start) instead of a throwaway sandbox"Input schema / properties / stdin / descriptionAdded value: +"Text piped to the program's standard input; empty means no input"Input schema / properties / timeout / descriptionAdded value: +"Wall-clock seconds before the run is killed as TLE; clamped to a 120s ceiling"execute_code_stream10 fields changedInput schema / properties / code / descriptionAdded value: +"Source code to run in `language`"Input schema / properties / dependencies / descriptionAdded value: +"Packages to install before running, e.g. ['requests==2.31.0']; merged with any PEP 723 block"Input schema / properties / language / descriptionAdded value: +"Runtime to execute in, e.g. 'python3', 'node'; see list_languages for the full catalog"Input schema / properties / max_cpu / descriptionAdded value: +"Per-call CPU-time ceiling in seconds; 0 means no explicit limit is set"Input schema / properties / max_memory_mb / descriptionAdded value: +"Per-call memory ceiling in MiB; 0 means no explicit limit is set"Input schema / properties / max_output_kb / descriptionAdded value: +"Stdout/stderr capture cap in KiB per stream; 0 uses the 64 KiB default, hard-clamped to 240"Input schema / properties / no_net / descriptionAdded value: +"Block outbound network access for this run; best-effort on platforms without seccomp"Input schema / properties / provider / descriptionAdded value: +"Execution backend id to use (see list_execution_providers); default picks automatically"Input schema / properties / stdin / descriptionAdded value: +"Text piped to the program's standard input; empty means no input"Input schema / properties / timeout / descriptionAdded value: +"Wall-clock seconds before the run is killed; clamped to a 300s ceiling (longer than execute_code's)"extract_function5 fields changedInput schema / properties / call / descriptionAdded value: +"Call expression to invoke the extracted function; required for non-python3 languages"Input schema / properties / code / descriptionAdded value: +"Source containing the function to extract, plus its imports and helpers"Input schema / properties / function_name / descriptionAdded value: +"Name of the function within `code` to extract into a standalone program"Input schema / properties / language / descriptionAdded value: +"Language `code` is written in; python3 gets exact ast extraction, others best-effort block extraction"Input schema / properties / test_inputs / descriptionAdded value: +"Inputs to run the extracted program with, one run per input"float_repr1 field changedInput schema / properties / x / descriptionAdded value: +"Value to inspect as binary64: exact stored value, raw bits, ULP, neighbours, and representability"human_duration1 field changedInput schema / properties / seconds / descriptionAdded value: +"Elapsed span in seconds (not a point-in-time timestamp) to humanize, e.g. into '2d 3h 4m 5s'"install_package4 fields changedInput schema / properties / language / descriptionAdded value: +"Language whose package manager installs the package, e.g. 'python3', 'node'"Input schema / properties / package / descriptionAdded value: +"Package name to install via that language's manager (uv pip/npm/gem/go get/cargo add)"Input schema / properties / session_id / descriptionAdded value: +"Install into this session's workspace instead of the shared cache; omit for the shared cache"Input schema / properties / version / descriptionAdded value: +"Exact version to install; omit to install the manager's default/latest"int_widthslimit_expressionmatrix2 fields changedInput schema / properties / op / descriptionAdded value: +"Operation to apply: one of det, inverse, eigenvalues, transpose, rank, trace"Input schema / properties / rows / descriptionAdded value: +"Row-major matrix as a JSON array of arrays; each entry is a number or a scalar expression string like 'sqrt(2)'"percentage2 fields changedInput schema / properties / part / descriptionAdded value: +"Numerator expression (rationals accepted), evaluated exactly"Input schema / properties / total / descriptionAdded value: +"Denominator expression (rationals accepted), evaluated exactly"percentiles1 field changedInput schema / properties / nums / descriptionAdded value: +"Sample of numbers to compute p50/p90/p95/p99 for, by nearest-rank and linear interpolation"physical_constants1 field changedInput schema / properties / name / descriptionAdded value: +"Constant to look up, e.g. 'speed_of_light', 'planck', 'avogadro'; omit to list all 22"radix_convert3 fields changedInput schema / properties / from_base / descriptionAdded value: +"Base `value` is written in; valid range 2..36, default 10"Input schema / properties / to_base / descriptionAdded value: +"Base to convert `value` into; valid range 2..36, default 10"Input schema / properties / value / descriptionAdded value: +"Digit string to convert (fractions with '.' accepted), in `from_base`"run_cancel1 field changedInput schema / properties / run_id / descriptionAdded value: +"Id of a background run, as returned by run_submit"run_inspect1 field changedInput schema / properties / run_id / descriptionAdded value: +"Id of a background run, as returned by run_submit"run_submit10 fields changedInput schema / properties / code / descriptionAdded value: +"Source code to run in `language`"Input schema / properties / dependencies / descriptionAdded value: +"Packages to install before running, e.g. ['requests==2.31.0']; merged with any PEP 723 block"Input schema / properties / language / descriptionAdded value: +"Runtime to execute in, e.g. 'python3', 'node'; see list_languages for the full catalog"Input schema / properties / max_cpu / descriptionAdded value: +"Per-call CPU-time ceiling in seconds; 0 means no explicit limit is set"Input schema / properties / max_memory_mb / descriptionAdded value: +"Per-call memory ceiling in MiB; 0 means no explicit limit is set"Input schema / properties / max_output_kb / descriptionAdded value: +"Stdout/stderr capture cap in KiB per stream; 0 uses the 64 KiB default, hard-clamped to 240"Input schema / properties / no_net / descriptionAdded value: +"Block outbound network access for this run; best-effort on platforms without seccomp"Input schema / properties / provider / descriptionAdded value: +"Execution backend id to use (see list_execution_providers); default picks automatically"Input schema / properties / stdin / descriptionAdded value: +"Text piped to the program's standard input; empty means no input"Input schema / properties / timeout / descriptionAdded value: +"Wall-clock seconds before the run is killed; clamped to a 120s ceiling, same as execute_code"runtimes_status1 field changedInput schema / properties / languages / descriptionAdded value: +"Comma-separated languages to check, e.g. 'python3,node,rust'; empty checks all"session_artifacts1 field changedInput schema / properties / session_id / descriptionAdded value: +"Id of the session whose executed-code output files to list"session_files4 fields changedInput schema / properties / cursor / descriptionAdded value: +"Opaque page cursor from a previous session_files call's response, to fetch the next page"Input schema / properties / page_size / descriptionAdded value: +"Max entries per page; omit for one unpaginated listing"Input schema / properties / path / descriptionAdded value: +"Subdirectory to list, relative to the workspace root; empty lists the root"Input schema / properties / session_id / descriptionAdded value: +"Id of the session whose workspace files to list"session_read_file4 fields changedInput schema / properties / as_image / descriptionAdded value: +"Return the file as an inline image the model can see, instead of text"Input schema / properties / max_bytes / descriptionAdded value: +"Max bytes to read from the file; default 65536 (64 KiB)"Input schema / properties / path / descriptionAdded value: +"Relative path inside the workspace to read"Input schema / properties / session_id / descriptionAdded value: +"Id of the session whose workspace file to read"session_run6 fields changedInput schema / properties / dependencies / descriptionAdded value: +"Packages to install before running, e.g. ['requests==2.31.0']"Input schema / properties / entry_file / descriptionAdded value: +"Relative path of the file to execute; may import other files already in the workspace"Input schema / properties / language / descriptionAdded value: +"Language to run `entry_file` as; omit to infer it from the session/file"Input schema / properties / session_id / descriptionAdded value: +"Id of the session workspace to run in"Input schema / properties / stdin / descriptionAdded value: +"Text piped to the program's standard input; empty means no input"Input schema / properties / timeout / descriptionAdded value: +"Wall-clock seconds before the run is killed"session_snapshot5 fields changedInput schema / properties / action / descriptionAdded value: +"One of 'save', 'restore', 'list', 'delete'; default 'save' archives the workspace"Input schema / properties / label / descriptionAdded value: +"Optional human-readable label to store with a new snapshot; only used by action='save'"Input schema / properties / replace / descriptionAdded value: +"For action='restore', wipe and reuse session_id's own workspace instead of creating a new session"Input schema / properties / session_id / descriptionAdded value: +"Id of the session the snapshot belongs to or is restored into"Input schema / properties / snapshot_id / descriptionAdded value: +"Id of an existing snapshot; required for action='restore' or action='delete'"session_start1 field changedInput schema / properties / language / descriptionAdded value: +"Language for the new session's worker/workspace; default 'python3' gets a stateful REPL"session_stop2 fields changedInput schema / properties / keep_snapshots / descriptionAdded value: +"Keep this session's saved session_snapshot archives instead of deleting them"Input schema / properties / session_id / descriptionAdded value: +"Id of the session to stop, as returned by session_start"session_write_file3 fields changedInput schema / properties / content / descriptionAdded value: +"Text content to write to `path`, overwriting any existing file"Input schema / properties / path / descriptionAdded value: +"Relative destination path inside the workspace; path escapes (e.g. '../') are refused"Input schema / properties / session_id / descriptionAdded value: +"Id of the session workspace to write into"simplify_expressionsolve_expressionsolve_linearsymbolic6 fields changedInput schema / properties / expr / descriptionAdded value: +"Expression or equation to solve/simplify/take the limit of; required by op='solve'/'simplify'/'limit'"Input schema / properties / op / descriptionAdded value: +"Which symbolic operation to run: 'solve', 'solve_linear', 'simplify', or 'limit' (each has its own required params)"Input schema / properties / point / descriptionAdded value: +"Point `var` approaches for op='limit'; optional, default 'oo' (infinity)"Input schema / properties / system / descriptionAdded value: +"';'-separated equations for op='solve_linear', e.g. 'x + y = 10; x - y = 2'; required by that op"Input schema / properties / var / descriptionAdded value: +"Variable to solve for or take the limit over; optional, default 'x'; used by op='solve'/'limit'"Input schema / properties / variables / descriptionAdded value: +"Comma-separated variable names for op='solve_linear', e.g. 'x, y'; required by that op"trace_execution10 fields changedInput schema / properties / code / descriptionAdded value: +"Source code to trace line by line"Input schema / properties / language / descriptionAdded value: +"Runtime to trace; only 'python3' is supported, any other value is refused"Input schema / properties / max_cpu / descriptionAdded value: +"Per-call CPU-time ceiling in seconds; 0 means no explicit limit is set"Input schema / properties / max_events / descriptionAdded value: +"Max trace events to record before truncating; the run's own stdout/exit code are unaffected"Input schema / properties / max_memory_mb / descriptionAdded value: +"Per-call memory ceiling in MiB; 0 means no explicit limit is set"Input schema / properties / max_output_kb / descriptionAdded value: +"Stdout/stderr capture cap in KiB per stream; 0 uses the 64 KiB default, hard-clamped to 240"Input schema / properties / no_net / descriptionAdded value: +"Block outbound network access for this run; best-effort on platforms without seccomp"Input schema / properties / provider / descriptionAdded value: +"Execution backend id; only 'local' (the default) is supported here"Input schema / properties / stdin / descriptionAdded value: +"Text piped to the program's standard input; empty means no input"Input schema / properties / timeout / descriptionAdded value: +"Wall-clock seconds before the run is killed; clamped to a 120s ceiling"truth_table1 field changedInput schema / properties / expression / descriptionAdded value: +"Boolean logic expression to tabulate, e.g. 'a and b or not c', 'p xor q', 'a implies b'"update_runtimes3 fields changedInput schema / properties / apply / descriptionAdded value: +"False (default) is a dry run reporting commands only; True actually runs them and asks for confirmation first"Input schema / properties / languages / descriptionAdded value: +"Comma-separated languages to update, e.g. 'python3,node,rust'; empty updates all"Input schema / properties / timeout / descriptionAdded value: +"Wall-clock seconds allowed for the update commands to complete"verify_optimization6 fields changedInput schema / properties / candidate / descriptionAdded value: +"Optimised version of `original`, to prove correct and measurably faster"Input schema / properties / language / descriptionAdded value: +"Language both `original` and `candidate` are written in"Input schema / properties / min_speedup / descriptionAdded value: +"Minimum median speedup ratio required to accept the optimisation; default 1.15 (15% faster)"Input schema / properties / original / descriptionAdded value: +"Baseline program to compare against"Input schema / properties / sizes / descriptionAdded value: +"Input sizes to time both programs at (2-3+ sizes needed for significance); omit for defaults"Input schema / properties / test_inputs / descriptionAdded value: +"Inputs to confirm both programs still agree on; omit to use the default set"verify_translation5 fields changedInput schema / properties / source_code / descriptionAdded value: +"Original program, in `source_language`"Input schema / properties / source_language / descriptionAdded value: +"Language of `source_code`"Input schema / properties / target_code / descriptionAdded value: +"Ported program, in `target_language`, to check against `source_code`"Input schema / properties / target_language / descriptionAdded value: +"Language of `target_code`"Input schema / properties / test_inputs / descriptionAdded value: +"Inputs to run both programs on and compare; omit to use the default edge-case set"z3_check1 field changedInput schema / properties / smt2 / descriptionAdded value: +"SMT-LIB2 script to check for satisfiability, e.g. '(declare-const x Int)(assert (> x 5))(check-sat)'"v0.11.0algebraic_equiv1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "algebraic_equivDictOutput",
+ "type": "object"
+}analyze_complexity1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "analyze_complexityDictOutput",
+ "type": "object"
+}base_repr1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "base_reprDictOutput",
+ "type": "object"
+}benchmark1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "benchmarkDictOutput",
+ "type": "object"
+}bit_analysis1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "bit_analysisDictOutput",
+ "type": "object"
+}bitop1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "bitopDictOutput",
+ "type": "object"
+}bitsbranch_reachabilitycalc_exact1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "calc_exactDictOutput",
+ "type": "object"
+}calc_stats1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "calc_statsDictOutput",
+ "type": "object"
+}collision_probability1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "collision_probabilityDictOutput",
+ "type": "object"
+}compare_edge_cases1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "compare_edge_casesDictOutput",
+ "type": "object"
+}compare_execution2 fields changedInput schema / properties / dependenciesAdded value: +{
+ "anyOf": [
+ {
+ "additionalProperties": {
+ "items": {
+ "type": "string"
+ },
+ "type": "array"
+ },
+ "type": "object"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": null,
+ "title": "Dependencies"
+}Output schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "compare_executionDictOutput",
+ "type": "object"
+}compare_threshold1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "compare_thresholdDictOutput",
+ "type": "object"
+}convert_units1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "convert_unitsDictOutput",
+ "type": "object"
+}data_sizes1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "data_sizesDictOutput",
+ "type": "object"
+}epoch_time1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "epoch_timeDictOutput",
+ "type": "object"
+}evaluate_expression1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "evaluate_expressionDictOutput",
+ "type": "object"
+}execute_code2 fields changedInput schema / properties / dependenciesAdded value: +{
+ "anyOf": [
+ {
+ "items": {
+ "type": "string"
+ },
+ "type": "array"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": null,
+ "title": "Dependencies"
+}Output schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "execute_codeDictOutput",
+ "type": "object"
+}execute_code_stream2 fields changedInput schema / properties / dependenciesAdded value: +{
+ "anyOf": [
+ {
+ "items": {
+ "type": "string"
+ },
+ "type": "array"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": null,
+ "title": "Dependencies"
+}Output schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "execute_code_streamDictOutput",
+ "type": "object"
+}extract_function1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "extract_functionDictOutput",
+ "type": "object"
+}float_repr1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "float_reprDictOutput",
+ "type": "object"
+}human_duration1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "human_durationDictOutput",
+ "type": "object"
+}install_package1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "install_packageDictOutput",
+ "type": "object"
+}int_widths1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "int_widthsDictOutput",
+ "type": "object"
+}limit_expression1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "limit_expressionDictOutput",
+ "type": "object"
+}list_units1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "list_unitsDictOutput",
+ "type": "object"
+}matrix1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "matrixDictOutput",
+ "type": "object"
+}percentage1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "percentageDictOutput",
+ "type": "object"
+}percentiles1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "percentilesDictOutput",
+ "type": "object"
+}physical_constants1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "physical_constantsDictOutput",
+ "type": "object"
+}radix_convert1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "radix_convertDictOutput",
+ "type": "object"
+}run_cancel1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "run_cancelDictOutput",
+ "type": "object"
+}run_inspect1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "run_inspectDictOutput",
+ "type": "object"
+}run_submit2 fields changedInput schema / properties / dependenciesAdded value: +{
+ "anyOf": [
+ {
+ "items": {
+ "type": "string"
+ },
+ "type": "array"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": null,
+ "title": "Dependencies"
+}Output schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "run_submitDictOutput",
+ "type": "object"
+}runtimes_status1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "runtimes_statusDictOutput",
+ "type": "object"
+}session_artifacts1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "session_artifactsDictOutput",
+ "type": "object"
+}session_files1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "session_filesDictOutput",
+ "type": "object"
+}session_list1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "session_listDictOutput",
+ "type": "object"
+}session_run1 field changedInput schema / properties / dependenciesAdded value: +{
+ "anyOf": [
+ {
+ "items": {
+ "type": "string"
+ },
+ "type": "array"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": null,
+ "title": "Dependencies"
+}session_snapshotsession_start1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "session_startDictOutput",
+ "type": "object"
+}session_stop2 fields changedInput schema / properties / keep_snapshotsAdded value: +{
+ "default": false,
+ "title": "Keep Snapshots",
+ "type": "boolean"
+}Output schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "session_stopDictOutput",
+ "type": "object"
+}session_write_file1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "session_write_fileDictOutput",
+ "type": "object"
+}simplify_expression1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "simplify_expressionDictOutput",
+ "type": "object"
+}solve_expression1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "solve_expressionDictOutput",
+ "type": "object"
+}solve_linear1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "solve_linearDictOutput",
+ "type": "object"
+}symbolictrace_executiontruth_table1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "truth_tableDictOutput",
+ "type": "object"
+}update_runtimes1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "update_runtimesDictOutput",
+ "type": "object"
+}verify_optimization1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "verify_optimizationDictOutput",
+ "type": "object"
+}verify_translation1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "verify_translationDictOutput",
+ "type": "object"
+}z3_check1 field changedOutput schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "title": "z3_checkDictOutput",
+ "type": "object"
+}v0.4.0matrixv0.2.0algebraic_equiv5 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / a / titleAdded value: +"A"Input schema / properties / b / titleAdded value: +"B"Input schema / titleAdded value: +"algebraic_equivArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullanalyze_complexity5 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / code / titleAdded value: +"Code"Input schema / properties / language / titleAdded value: +"Language"Input schema / titleAdded value: +"analyze_complexityArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullbase_repr5 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / n / titleAdded value: +"N"Input schema / properties / width / titleAdded value: +"Width"Input schema / titleAdded value: +"base_reprArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullbenchmark7 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / code / titleAdded value: +"Code"Input schema / properties / language / titleAdded value: +"Language"Input schema / properties / sizes / titleAdded value: +"Sizes"Input schema / properties / timeout / titleAdded value: +"Timeout"Input schema / titleAdded value: +"benchmarkArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullbit_analysis5 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / align / titleAdded value: +"Align"Input schema / properties / n / titleAdded value: +"N"Input schema / titleAdded value: +"bit_analysisArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullbitop7 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / a / titleAdded value: +"A"Input schema / properties / b / titleAdded value: +"B"Input schema / properties / op / titleAdded value: +"Op"Input schema / properties / width / titleAdded value: +"Width"Input schema / titleAdded value: +"bitopArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullcalc_exact4 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / expr / titleAdded value: +"Expr"Input schema / titleAdded value: +"calc_exactArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullcalc_stats4 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / nums / titleAdded value: +"Nums"Input schema / titleAdded value: +"calc_statsArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullcollision_probability5 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / bits / titleAdded value: +"Bits"Input schema / properties / items / titleAdded value: +"Items"Input schema / titleAdded value: +"collision_probabilityArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullcompare_edge_cases5 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / inputs / titleAdded value: +"Inputs"Input schema / properties / snippets / titleAdded value: +"Snippets"Input schema / titleAdded value: +"compare_edge_casesArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullcompare_execution6 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / snippets / titleAdded value: +"Snippets"Input schema / properties / stdin / titleAdded value: +"Stdin"Input schema / properties / timeout / titleAdded value: +"Timeout"Input schema / titleAdded value: +"compare_executionArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullcompare_threshold6 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / a / titleAdded value: +"A"Input schema / properties / b / titleAdded value: +"B"Input schema / properties / op / titleAdded value: +"Op"Input schema / titleAdded value: +"compare_thresholdArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullcontext7_docsconvert_units6 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / from_unit / titleAdded value: +"From Unit"Input schema / properties / to_unit / titleAdded value: +"To Unit"Input schema / properties / value / titleAdded value: +"Value"Input schema / titleAdded value: +"convert_unitsArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nulldata_sizes4 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / n / titleAdded value: +"N"Input schema / titleAdded value: +"data_sizesArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullepoch_time4 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / n / titleAdded value: +"N"Input schema / titleAdded value: +"epoch_timeArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullevaluate_expression4 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / expression / titleAdded value: +"Expression"Input schema / titleAdded value: +"evaluate_expressionArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullexecute_code14 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / code / titleAdded value: +"Code"Input schema / properties / compact / titleAdded value: +"Compact"Input schema / properties / language / titleAdded value: +"Language"Input schema / properties / max_cpu / titleAdded value: +"Max Cpu"Input schema / properties / max_memory_mb / titleAdded value: +"Max Memory Mb"Input schema / properties / max_output_kb / titleAdded value: +"Max Output Kb"Input schema / properties / no_net / titleAdded value: +"No Net"Input schema / properties / providerAdded value: +{
+ "anyOf": [
+ {
+ "type": "string"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": null,
+ "title": "Provider"
+}Input schema / properties / session_id / titleAdded value: +"Session Id"Input schema / properties / stdin / titleAdded value: +"Stdin"Input schema / properties / timeout / titleAdded value: +"Timeout"Input schema / titleAdded value: +"execute_codeArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullexecute_code_stream12 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / code / titleAdded value: +"Code"Input schema / properties / language / titleAdded value: +"Language"Input schema / properties / max_cpuAdded value: +{
+ "default": 0,
+ "title": "Max Cpu",
+ "type": "integer"
+}Input schema / properties / max_memory_mbAdded value: +{
+ "default": 0,
+ "title": "Max Memory Mb",
+ "type": "integer"
+}Input schema / properties / max_output_kb / titleAdded value: +"Max Output Kb"Input schema / properties / no_net / titleAdded value: +"No Net"Input schema / properties / providerAdded value: +{
+ "anyOf": [
+ {
+ "type": "string"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": null,
+ "title": "Provider"
+}Input schema / properties / stdin / titleAdded value: +"Stdin"Input schema / properties / timeout / titleAdded value: +"Timeout"Input schema / titleAdded value: +"execute_code_streamArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullextract_function8 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / call / titleAdded value: +"Call"Input schema / properties / code / titleAdded value: +"Code"Input schema / properties / function_name / titleAdded value: +"Function Name"Input schema / properties / language / titleAdded value: +"Language"Input schema / properties / test_inputs / titleAdded value: +"Test Inputs"Input schema / titleAdded value: +"extract_functionArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullfloat_repr4 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / x / titleAdded value: +"X"Input schema / titleAdded value: +"float_reprArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullhuman_duration4 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / seconds / titleAdded value: +"Seconds"Input schema / titleAdded value: +"human_durationArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullinstall_package7 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / language / titleAdded value: +"Language"Input schema / properties / package / titleAdded value: +"Package"Input schema / properties / session_id / titleAdded value: +"Session Id"Input schema / properties / version / titleAdded value: +"Version"Input schema / titleAdded value: +"install_packageArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullint_widths4 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / n / titleAdded value: +"N"Input schema / titleAdded value: +"int_widthsArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nulllimit_expression6 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / expr / titleAdded value: +"Expr"Input schema / properties / point / titleAdded value: +"Point"Input schema / properties / var / titleAdded value: +"Var"Input schema / titleAdded value: +"limit_expressionArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nulllist_execution_providerslist_languages5 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / titleAdded value: +"list_languagesArguments"Output schema / properties / result / titleAdded value: +"Result"Output schema / titleAdded value: +"list_languagesOutput"Output schema / x-fastmcp-wrap-resultRemoved value: -truelist_units3 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / titleAdded value: +"list_unitsArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nulloptimize_codepercentage5 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / part / titleAdded value: +"Part"Input schema / properties / total / titleAdded value: +"Total"Input schema / titleAdded value: +"percentageArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullpercentiles4 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / nums / titleAdded value: +"Nums"Input schema / titleAdded value: +"percentilesArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullphysical_constants4 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / name / titleAdded value: +"Name"Input schema / titleAdded value: +"physical_constantsArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullradix_convert6 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / from_base / titleAdded value: +"From Base"Input schema / properties / to_base / titleAdded value: +"To Base"Input schema / properties / value / titleAdded value: +"Value"Input schema / titleAdded value: +"radix_convertArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullrun_cancelrun_inspectrun_submitruntimes_status4 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / languages / titleAdded value: +"Languages"Input schema / titleAdded value: +"runtimes_statusArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullsession_artifacts4 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / session_id / titleAdded value: +"Session Id"Input schema / titleAdded value: +"session_artifactsArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullsession_files7 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / cursorAdded value: +{
+ "anyOf": [
+ {
+ "type": "string"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": null,
+ "title": "Cursor"
+}Input schema / properties / page_sizeAdded value: +{
+ "anyOf": [
+ {
+ "type": "integer"
+ },
+ {
+ "type": "null"
+ }
+ ],
+ "default": null,
+ "title": "Page Size"
+}Input schema / properties / path / titleAdded value: +"Path"Input schema / properties / session_id / titleAdded value: +"Session Id"Input schema / titleAdded value: +"session_filesArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullsession_list3 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / titleAdded value: +"session_listArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullsession_read_file6 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / as_image / titleAdded value: +"As Image"Input schema / properties / max_bytes / titleAdded value: +"Max Bytes"Input schema / properties / path / titleAdded value: +"Path"Input schema / properties / session_id / titleAdded value: +"Session Id"Input schema / titleAdded value: +"session_read_fileArguments"session_run8 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / entry_file / titleAdded value: +"Entry File"Input schema / properties / language / titleAdded value: +"Language"Input schema / properties / session_id / titleAdded value: +"Session Id"Input schema / properties / stdin / titleAdded value: +"Stdin"Input schema / properties / timeout / titleAdded value: +"Timeout"Input schema / titleAdded value: +"session_runArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullsession_start4 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / language / titleAdded value: +"Language"Input schema / titleAdded value: +"session_startArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullsession_stop4 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / session_id / titleAdded value: +"Session Id"Input schema / titleAdded value: +"session_stopArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullsession_write_file6 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / content / titleAdded value: +"Content"Input schema / properties / path / titleAdded value: +"Path"Input schema / properties / session_id / titleAdded value: +"Session Id"Input schema / titleAdded value: +"session_write_fileArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullsimplify_expression4 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / expr / titleAdded value: +"Expr"Input schema / titleAdded value: +"simplify_expressionArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullsolve_expression5 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / expr / titleAdded value: +"Expr"Input schema / properties / var / titleAdded value: +"Var"Input schema / titleAdded value: +"solve_expressionArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullsolve_linear5 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / system / titleAdded value: +"System"Input schema / properties / variables / titleAdded value: +"Variables"Input schema / titleAdded value: +"solve_linearArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nulltranslate_codetruth_table4 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / expression / titleAdded value: +"Expression"Input schema / titleAdded value: +"truth_tableArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullupdate_runtimes6 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / apply / titleAdded value: +"Apply"Input schema / properties / languages / titleAdded value: +"Languages"Input schema / properties / timeout / titleAdded value: +"Timeout"Input schema / titleAdded value: +"update_runtimesArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullverify_optimizationverify_translationz3_check4 fields changedInput schema / additionalPropertiesRemoved value: -falseInput schema / properties / smt2 / titleAdded value: +"Smt2"Input schema / titleAdded value: +"z3_checkArguments"Output schema / (root)Previous value: -{
- "additionalProperties": true,
- "type": "object"
-}New value: +nullv0.1.0algebraic_equivanalyze_complexitybase_reprbenchmarkbit_analysisbitopcalc_exactcalc_statscollision_probabilitycompare_edge_casescompare_executioncompare_thresholdcontext7_docsconvert_unitsdata_sizesepoch_timeevaluate_expressionexecute_codeexecute_code_streamextract_functionfloat_reprhuman_durationinstall_packageint_widthslimit_expressionlist_languageslist_unitsoptimize_codepercentagepercentilesphysical_constantsradix_convertruntimes_statussession_artifactssession_filessession_listsession_read_filesession_runsession_startsession_stopsession_write_filesimplify_expressionsolve_expressionsolve_lineartranslate_codetruth_tableupdate_runtimesz3_checkScored across 49 tools
Many tools are clearly distinct (e.g., session_*, run_*, symbolic vs calc_exact), but there is notable overlap among execution tools: execute_code, execute_code_stream, run_submit, session_run, and compare_execution all run code with different modes, and several math tools (calc_exact, evaluate_expression, symbolic, calc_stats, percentiles) have adjacent purposes. The descriptions are detailed enough to disambiguate with careful reading, but the sheer number of similar execution/math tools creates real selection risk.
Most tools follow a clear verb_noun pattern (execute_code, session_start, run_inspect, convert_units, verify_optimization). There are minor deviations: 'bits' and 'symbolic' are noun/adjective-only names, and 'matrix' is a single noun, but these are documented as mode-based consolidated tools. Overall the pattern is consistent and predictable.
49 tools is a very large surface for a code calculation/execution server. While the server covers many domains (execution, sessions, math, units, verification, runtimes), the count is heavy and includes several consolidated mode-based tools that could reduce the count further. It exceeds the typical well-scoped range and will burden agent tool selection.
The server covers its apparent domains thoroughly: code execution (sync, stream, background, session), file/session management, math/units/constants, verification (optimization, translation, edge cases), and runtime management. Minor gaps exist (e.g., no explicit session_run cancellation, no direct file deletion tool), but the core workflows are well covered and there are no obvious dead ends.
MCP server for progressive tool usage at any scale (see https://klavis.ai)
Nifty's MCP server — exposes tasks, projects, messages, and files as tools for AI agents.
Hosted MCP server to manage a restaurant menu from AI agents - 39 tools over the DuckHub API.
MCP server for building and testing AI agents with multi-model experimentation and insights.
