remote_exec

Run a shell command on one or more remote hosts over SSH and get the result immediately (exit code + SEPARATE stdout/stderr). This is the default way to execute ANYTHING on a remote machine — reach for it instead of hand-rolling ssh user@host …; it reuses a warm connection pool and fans out across hosts. Stateless: need cwd/exports to persist → remote_shell; a long task to background and poll → remote_job.

★ sudo: to run a command as root, call this with use_sudo=True — NEVER put a bare sudo … in the command or in remote_shell (there is no TTY and the password can't be fed). The password is supplied out-of-band; you never pass it. (For root on the server's OWN machine, sudo is not available — see local_exec.)

★ credentials: when a command needs a secret (API token, deploy key, …), do NOT have the user paste it into the chat. They run portal secret set <name> in their own terminal; you pass secrets=[""] and reference it as the uppercased env var ($NAME).

Targets (pick one): host="web01" | host=["web01","web02"] | group_tag="prod" (all registered hosts carrying that tag).

Commands (pick one): command="uptime" — ONE command per call is the DEFAULT and preferred shape. Run it, read the real output, check it against what you expected (an exit-0 step can still be wrong), THEN decide the next call — that keeps you in the loop and able to course-correct. commands=["a","b"] batches a sequence into a SINGLE call (each with its own exit code; stops at the first failure when stop_on_error=True, and with use_sudo each entry runs as its own sudo command) — reach for it ONLY for a fixed, independent sequence you do NOT need to inspect between steps. Don't bury a long or branching pipeline in one a && b && c string: you lose per-step visibility and can't intervene. A multi-line command string runs as one bash script.

Multi-host fan-out is parallel by default; serialize=True (+ delay_s, stop_on_error) does a rolling, stop-on-first-failure rollout.

timeout (seconds, REQUIRED — no default; consciously choose a cut-off): the call is held open until the command finishes or timeout elapses. Keepalive pings stop the client aborting a hung call, so timeout is your real cut-off — pass a SMALL one for an exploratory / re-runnable command (e.g. 10–30) to fail fast, raising it only for a genuinely slow but bounded command. Hard ceiling (PORTAL_MAX_TIMEOUT, default 300s): a request above it is REFUSED — background a long task with remote_job instead of pinning it on a blocking call.

login (default on): run the command in a LOGIN shell (bash -lc) so the remote user's ~/.profile / ~/.bashrc PATH + env (conda / nvm / pyenv, ~/.local/bin, …) is loaded; pass login=False for a bare non-login shell. Silently degrades to a plain exec on a host without bash. Affects only this plain path (not sudo / secrets); operators set the default via PORTAL_LOGIN_SHELL or a per-host login_shell: in hosts.yaml.

use_sudo: run via sudo -S, with the password resolved out-of-band per host (the per-user credential agent from portal sudo set <host>, or the host's sudo_password_command) — never from you. Refused with guidance if no source is set. Consumes the command's stdin (tools that read their own stdin aren't supported under sudo). May be combined with secrets (their values ride the same stdin channel, after the password).

secrets: a list of NAMES, not values (e.g. ["github_token"]); each is resolved server-side, fed over SSH stdin (never on argv/audit), exported as its uppercased env var ($GITHUB_TOKEN), and redacted to *** in the output. May be combined with use_sudo (values are read+exported inside the sudo'd shell, so sudo's env_reset can't strip them).

Returns one dict {host, command, exit_code, stdout, stderr, elapsed_s} for a single host + command, else a JSON list (a multi-command host carries {host, results:[…]}).

★ When use_sudo or secrets is used the result is flagged "high_risk": briefly tell the user you ran a privileged / credentialed command with their stored sudo password / secret, or only do so with their explicit prior permission.