Skip to main content
Glama

VMware Pilot

Author: Wei Zhou, VMware by Broadcom — wei-wz.zhou@broadcom.com This is a community-driven project by a VMware engineer, not an official VMware product. For official VMware developer tools see developer.broadcom.com.

English | 中文

Multi-step workflow orchestration for VMware MCP skills — state machine, approval gates, audit trail.

Companion skills handle everything else:

Skill

Scope

Install

vmware-aiops

VM lifecycle, deployment, guest ops, cluster

uv tool install vmware-aiops

vmware-monitor

Read-only: inventory, health, alarms, events

uv tool install vmware-monitor

vmware-storage

Datastores, iSCSI, vSAN management

uv tool install vmware-storage

vmware-vks

Tanzu Namespaces, TKC cluster lifecycle

uv tool install vmware-vks

vmware-nsx

NSX networking: segments, gateways, NAT

uv tool install vmware-nsx-mgmt

vmware-nsx-security

DFW firewall rules, security groups

uv tool install vmware-nsx-security

vmware-aria

Aria Ops: metrics, alerts, capacity

uv tool install vmware-aria

vmware-avi

AVI load balancing, pool management, AKO K8s ops

uv tool install vmware-avi

Install

uv tool install vmware-pilot
vmware-pilot mcp          # start the MCP server (stdio)

Offline / Air-Gapped Install (from source)

This project uses the modern PEP 517 build system (hatchling), so there is no setup.py by design — that is expected, not a missing file. If you cloned the source and hit ERROR: File "setup.py" or "setup.cfg" not found ... editable mode currently requires a setuptools-based build, your pip is older than 21.3 and cannot do an editable (-e) install with a non-setuptools backend. Editable mode is a developer convenience, not needed to run the tool — do one of:

# From the source tree — a normal (non-editable) install builds a wheel:
pip install .              # NOT  pip install -e .

# ...or upgrade pip first, and editable works too:
pip install --upgrade pip && pip install -e .

For a truly air-gapped host, build the wheels on a connected machine and copy them over — the target then needs no network:

# On a connected machine, collect this package + its dependencies as wheels:
pip wheel . -w dist        # → dist/*.whl   (or: uv build, for just this package)

# Copy dist/ to the air-gapped host, then install offline:
pip install --no-index --find-links dist vmware-pilot

Related MCP server: vmware-harden

MCP Tools (13 — 4 read, 9 write)

Tool

Description

get_skill_catalog

List all available skills and tools for workflow design

list_workflows

List built-in and custom templates

review_workflow

Sanity-check a planned workflow before execution

design_workflow

Natural language goal → draft workflow

update_draft

Edit draft workflow steps

confirm_draft

Finalize draft → ready to execute

create_workflow_from_template

Create and save a workflow from a template (not a dry run), returns workflow_id

create_workflow

Create custom workflow from step list (refused if a destructive step has no approval gate before it)

run_workflow

Execute workflow, pauses at approval gates

get_workflow_status

Query state + diff report + audit log

approve

Human approval, continue execution. Refused while steps before the gate are not_executed, until acknowledge_not_executed=True says you ran them

rollback

Explicit, best-effort undo of steps pilot recorded as succeeded — never automatic. Previews (blast_radius) unless confirm=True

cancel_workflow

Cancel a workflow — move it to the terminal CANCELLED state. Previews (blast_radius) unless confirm=True

Built-in Templates (15)

n is the number of VMs (or drift items); ranges depend on which optional parameters are set. clone_and_test needs snapshot_name (the snapshot to clone from), resizes staging while it is powered off, and has 8 steps when change_spec is a guest command (an extra gate before it runs in staging). investigate_alert's approvals are synthesis checkpoints; its steps are all reads. See skills/vmware-pilot/references/templates.md for parameters and steps.

Template

Steps

Approval

Skills Used

clone_and_test

7-8

Yes

aiops, monitor

incident_response

4

Yes

monitor, aiops

investigate_alert

4 (8 with deep_dive)

Yes

monitor, aria

plan_and_approve

3

Yes

aiops

compliance_scan

1-3

No

monitor, aria

network_segment_setup

3-6

Yes

nsx, nsx-security

vks_cluster_deploy

5

Yes

vks

rolling_restart

2+3n

Yes

aiops, monitor

capacity_expansion

5

Yes

aria, aiops, monitor

disaster_recovery

5

Yes

aiops, monitor, nsx

patch_deployment

1+3n

Yes

aiops, monitor

storage_expansion

6

Yes

storage

baseline_capture

1-5

No

monitor, nsx, storage

baseline_audit

1-5

No

monitor, nsx, storage, aria

baseline_remediate

3+n

Yes

varies

Upgrading to 2.0

2.0 renames one MCP tool: plan_workflow → create_workflow_from_template. It always created and saved a workflow from a template (it is a write, not a dry run); the new name says so and sets it apart from create_workflow, which saves your own step list. Parameters and results are unchanged, and so are the CLI commands. The skill catalog also follows vmware-vks's renames (get_supervisor_kubeconfig → issue_supervisor_kubeconfig, get_tkc_kubeconfig → issue_tkc_kubeconfig). Workflows saved with the old names are translated when loaded. If an agent prompt, script or ~/.vmware/rules.yaml names plan_workflow, update it. Requires vmware-policy ≥ 1.19.0. Details: RELEASE_NOTES.md.

MCP Configuration

{
  "mcpServers": {
    "vmware-pilot": {
      "command": "vmware-pilot",
      "args": ["mcp"]
    }
  }
}

Fallback: {"command": "uvx", "args": ["--from", "vmware-pilot", "vmware-pilot-mcp"]} still works, but uvx re-resolves against PyPI on every start and fails behind a TLS-inspecting corporate proxy (invalid peer certificate: UnknownIssuer). The installed entry point above touches the network zero times; set UV_NATIVE_TLS=true if you must use uvx.

Pilot runs the steps itself: --dispatch

By default pilot executes nothing: run_workflow returns each step in pending_dispatch and your agent performs it. Add --dispatch (or set VMWARE_PILOT_DISPATCH=stdio) and pilot runs each step itself, in that skill's own MCP server (vmware-<skill>-mcp, which must be installed), so the skill's policy rules, confirmation gate and audit apply as usual. Nothing after an approval gate runs before approve, and workflows reach completed.

{"mcpServers": {"vmware-pilot": {"command": "vmware-pilot", "args": ["mcp", "--dispatch"]}}}

stdio only: pilot refuses to start with --dispatch and --transport https/http (the skill servers it starts would have no caller identity, so roles could not apply). A step that takes longer than VMWARE_PILOT_DISPATCH_TIMEOUT seconds (default 1800) is recorded interrupted — its effect is unknown — and rollback waits for you to check it. run_workflow and approve take as long as the steps they run: set your MCP client's tool timeout above your longest step. If a call does time out, Pilot carries on and answers nothing else until the step ends; then get_workflow_status shows the outcome. Never perform those steps yourself.

Remote access (HTTPS) and limiting tools

By default vmware-pilot mcp speaks stdio to one local client and offers every tool. Two options change that (vmware-policy ≥ 1.21.0).

Serve over HTTPS so clients on other machines can connect. Every caller needs a bearer token:

vmware-audit token create alice          # prints the token once; give it to alice
vmware-audit token list                  # names only
vmware-audit token revoke alice          # refused from its next request, no restart
vmware-pilot mcp --transport https --host 0.0.0.0 --port 8443 \
    --cert /etc/vmware-mcp/server.pem --key /etc/vmware-mcp/server-key.pem \
    --public-name mcp.example.com

Connect a client (Claude Code shown; any MCP client that supports streamable HTTP and custom headers works):

claude mcp add --transport http vmware-pilot https://mcp.example.com:8443/mcp \
    --header "Authorization: Bearer $VMWARE_MCP_TOKEN"
{
  "mcpServers": {
    "vmware-pilot": {
      "type": "http",
      "url": "https://mcp.example.com:8443/mcp",
      "headers": { "Authorization": "Bearer ${VMWARE_MCP_TOKEN}" }
    }
  }
}
  • The server will not start without a token, refuses Host/Origin values outside --public-name, and checks before it binds that unauthenticated requests get 401. Use a certificate your clients trust for those names.

  • Behind a TLS-terminating reverse proxy on the same machine use --transport http --host 127.0.0.1 --public-name mcp.example.com and pass the Authorization header through.

  • Behind a load balancer or gateway on another machine, keep --transport https (a self-signed certificate is fine), list in --public-name the host name the balancer sends as Host, and have it forward Authorization. Its health probe can use GET /healthz, which needs no token and returns only {"status": "ok"} (vmware-policy ≥ 1.23.0).

  • Send traceparent or X-Request-ID with each request to have it stored on the audit row; vmware-audit log --correlation-id ID then finds the calls behind one platform request.

  • The audit log records each caller as token:<name>. Calls naming different targets run in parallel; calls that name no target run one at a time, so name the target when several people share a server.

  • Everyone who can reach the endpoint acts with this server's account. Tokens identify callers; they grant nothing on the platform. Run one endpoint per privilege level.

  • Several people or roles on one endpoint: run vmware-audit roles enable once, then give each token a role (vmware-audit token create NAME --role reader, or a role in roles.yaml naming tools and targets; vmware-policy ≥ 1.24.0). A role limits what the token may call on this endpoint; it cannot grant more than the account has. See vmware-policy's README, "Roles for network callers".

Offer only read tools, plus the writes you choose:

vmware-pilot mcp --expose read --allow-write cancel_workflow
# or: VMWARE_PILOT_MCP_EXPOSE=read VMWARE_PILOT_MCP_ALLOW_WRITE=cancel_workflow vmware-pilot mcp

Tools not offered do not appear in the client's tool list. A bad value or an unknown write tool stops the server (exit 2). This limits what clients are offered; it is not authorization, so the account itself should be least-privilege.

Details: skills/vmware-pilot/references/setup-guide.md ("Serving MCP over HTTPS", "Limiting which tools are offered").

License

MIT

Available Tools

13 tools
approveA

[WRITE] Approve a workflow that is waiting for human confirmation.

Use this only after showing the user the pending step and getting explicit human consent; use cancel_workflow instead when the approval is rejected. Only works when workflow state is 'awaiting_approval'. After approval, execution continues to the next steps.

Without --dispatch, the steps before the gate are 'not_executed': Pilot has no record that they ran, whatever the gate's message says. approve refuses (listing them in not_executed_before_gate) until acknowledge_not_executed=True — set it only after you performed each of those steps yourself and showed the user their results. They are then recorded as 'reported_done' and not handed out again.

Note: with --dispatch, approving runs the next steps (this call takes as long as they do; if it times out in your client, Pilot carries on — check get_workflow_status once it answers again, and never perform those steps yourself). Without it, the remaining steps are recorded as 'not_executed' and the result carries outcome='dispatch_required' with a 'pending_dispatch' list for you to perform (see run_workflow); the workflow then never reaches 'completed'.

Returns: Updated workflow state after resuming.

ParametersJSON Schema
NameRequiredDescriptionDefault
approverNoName of the person approving (for audit trail). Over an authenticated network transport leave it empty: the approver is the caller's token name, and a different name is refused.
workflow_idYesThe workflow ID to approve.
acknowledge_not_executedNoTrue says the steps before this gate were performed outside Pilot and their results were checked.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only cover the generic safety profile (write, non-idempotent, open-world), while the description discloses non-obvious behavior: refusal listing steps in not_executed_before_gate, the acknowledge_not_executed gate, the 'reported_done' vs 'not_executed' bookkeeping, timeout behavior ('Pilot carries on — check get_workflow_status'), and the outcome='dispatch_required'/'pending_dispatch' return semantics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Content is valuable and mostly front-loaded, but the not_executed behavior is explained twice (once in the gate paragraph and again in the dispatch paragraph), and the CLI-style '--dispatch' framing adds noise to a JSON-parameter tool. Trimming the duplication would sharpen it without losing information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the return burden and does reasonably well, stating 'Updated workflow state after resuming' and enumerating the dispatch_required outcome and pending_dispatch list. It is nearly complete for a complex stateful write; only the exact shape of the returned state is left unspecified.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3, but the description adds real depth beyond the schema for acknowledge_not_executed ('set it only after you performed each of those steps yourself and showed the user their results') and explains the approver/auth interaction. It loses a point for repeatedly referencing a '--dispatch' flag that does not exist as a parameter in the schema, which could mislead an agent about the call signature.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource and scope: '[WRITE] Approve a workflow that is waiting for human confirmation.' It also explicitly distinguishes itself from the sibling cancel_workflow and names the required precondition (state 'awaiting_approval'), so an agent can select it without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit when-to-use ('only after showing the user the pending step and getting explicit human consent'), when-not ('use cancel_workflow instead when the approval is rejected'), and a hard state prerequisite ('Only works when workflow state is awaiting_approval'). It also gives conditional guidance for the dispatch and acknowledge_not_executed paths.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

cancel_workflowA
Destructive

[WRITE] Cancel a workflow — move it to the terminal CANCELLED state.

Use this when an approval is REJECTED, a review flags the plan as unsafe, or an operator decides the workflow must never run. A cancelled workflow is dead: run_workflow and approve refuse to execute it. Without this, an approval-rejected PENDING workflow could still be picked up and run.

Without confirm=True this only previews: it returns blast_radius — the workflow's id, type and state, the executed steps that stay applied (left_in_place: cancelling part-way leaves a half-applied change), steps performed outside Pilot (performed_outside_pilot), steps already handed to you in pending_dispatch that cancel can neither stop nor undo (handed_out_to_caller), the steps that would be skipped (would_skip), steps with unknown effects, and blockers — and changes nothing. Show that to the user and get their explicit decision. Do not set confirm=True on your own because the user asked to cancel earlier — the user has not seen the preview yet.

Cancel only stops FUTURE steps. It does NOT undo already-completed steps — use rollback() to reverse those. Refused with confirm=True: an already completed/failed/cancelled workflow, a step whose status Pilot does not recognise, a workflow record that cannot be read, and — unless acknowledge_unknown_effects=True — a step left 'running' or 'interrupted' (listed in unknown_effects). The cancellation is written to the workflow audit log.

Returns: Preview: {"action": "preview", "blast_radius", "hint"}. Acting: the workflow state (state='cancelled', outcome='cancelled', action='cancelled') with blast_radius, or an error if refused.

ParametersJSON Schema
NameRequiredDescriptionDefault
reasonNoOptional human-readable reason (e.g. "approval rejected by on-call"), recorded in the audit log.
confirmNoFalse (default) returns the blast radius and changes nothing. True applies it.
workflow_idYesThe workflow ID to cancel.
acknowledge_unknown_effectsNoSet True only after the user has checked, in the target system, whether each unknown_effects step took effect. Covers only those steps; every other refusal still applies.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructive=true, but the description goes well beyond them: it discloses the preview/act gate, enumerates blast_radius fields (left_in_place, performed_outside_pilot, handed_out_to_caller, would_skip, unknown_effects), lists the exact refusal conditions, notes the acknowledge_unknown_effects escape hatch, and states the cancellation is written to the audit log.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Dense but well front-loaded: the write marker, verb, and terminal-state outcome come first, followed by when-to-use, then the preview contract, then refusals. It is longer than typical, and the blast_radius field enumeration is verbose, but each block carries distinct operational information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, and the description compensates by specifying both return shapes (preview with blast_radius/hint vs. acting with state/outcome/action) plus the refusal error case. Combined with the refusal list and the partial-application caveat, an agent has everything needed to call and interpret this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real meaning: it explains that confirm=False is a preview-only mode whose purpose is user review, and that acknowledge_unknown_effects covers only the unknown_effects steps while every other refusal still applies. The reason parameter's audit-log role is also reinforced.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Cancel a workflow — move it to the terminal CANCELLED state') and immediately differentiates from the closest sibling by clarifying it does not undo completed steps ('use rollback() to reverse those'). An agent can distinguish this from run_workflow, approve, and rollback without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit triggers (approval REJECTED, review flags unsafe, operator decision) and an explicit exclusion with the alternative named (cancelling does not undo completed steps — use rollback). It also prescribes the correct workflow for the preview step: show blast_radius to the user, do not set confirm=True unprompted.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

confirm_draftA

[WRITE] Confirm a draft workflow — changes state from DRAFT to PENDING.

Use this once the user has approved the draft's steps; call update_draft instead if anything still needs changing. After confirmation, the workflow can be executed via run_workflow(). Optionally saves as a YAML template for future reuse. Refused — the draft stays a draft and nothing is saved — while any destructive or unclassifiable step, or any step passing confirm=True, lacks a require_approval gate before it.

Returns: Confirmed workflow summary. Call run_workflow() to execute.

ParametersJSON Schema
NameRequiredDescriptionDefault
workflow_idYesThe draft workflow ID to confirm.
save_as_templateNoIf True, save to ~/.vmware/workflows/ for reuse. Refused when the draft's name is a built-in template's name.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, it discloses the DRAFT-to-PENDING state change, optional template persistence, and important refusal conditions for destructive or unclassifiable steps lacking require_approval gates. It also states that nothing is saved when confirmation is refused, which is genuinely useful behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded, covering action, usage, alternatives, refusal, and return value in a short space. It loses a point for the awkwardly worded 'Refused — ... while ...' sentence, which is grammatically tangled but still decipherable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a write operation with no output schema, it explains the return value ('Confirmed workflow summary'), the required next step, the alternative tool, and the failure/refusal semantics. The remaining details, such as built-in template conflict behavior, are covered by the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents workflow_id and save_as_template well. The description adds small extra meaning by mentioning that the template is YAML and placing the save within the confirmation flow, which is slightly beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Confirm a draft workflow — changes state from DRAFT to PENDING.' This clearly names the action, the object, and the state transition, and it is immediately distinguishable from siblings like update_draft and run_workflow.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly states the trigger condition ('once the user has approved the draft's steps'), the alternative ('call update_draft instead if anything still needs changing'), and the next step ('run_workflow()'). This gives the agent concrete routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_workflowA

[WRITE] Create a custom workflow dynamically from a step list.

Use this when you already know the steps and no built-in template matches; prefer create_workflow_from_template when one does, and design_workflow when the user gave a goal rather than steps. Call get_skill_catalog first for the skill and tool names a step may target.

Each step dict must have: action, skill, tool, params. Optional: rollback_tool, rollback_params. action="require_approval" (skill "pilot", tool "approve") inserts a human approval gate. A workflow whose destructive or unclassifiable step, or a step passing confirm=True, has no gate before it is REFUSED — nothing is saved — and the error names the step and the gate to insert.

Returns: dict with workflow_id and plan summary. Next call review_workflow to check the plan, then run_workflow to execute. Nothing is undone automatically on failure; see rollback.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesWorkflow name (used as workflow_type).
stepsYesList of step dicts, each with action/skill/tool/params.
descriptionYesHuman-readable description.
save_as_templateNoIf True, save as YAML to ~/.vmware/workflows/ for reuse. Refused when name is a built-in template's name, since the saved template would replace the built-in.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only supply readOnlyHint=false and destructiveHint=false; the description goes well beyond that by disclosing the required step schema, the approval-gate mechanism (action='require_approval', skill 'pilot', tool 'approve'), and the refusal behavior that names the offending step when a destructive/confirm=True step lacks a gate. It also states the return value, the follow-up sequence, and that nothing is undone automatically on failure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the [WRITE] tag and one-line purpose, then usage routing, then mechanics. The refusal paragraph is dense but load-bearing; the Returns block is slightly more verbose than necessary, keeping it just short of maximum.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the return contract (workflow_id + plan summary) and the recommended next calls (review_workflow, then run_workflow), plus rollback expectations. For a mutation tool with a validation-refusal path, this is complete enough to drive correct use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3, but the description adds real semantics the schema lacks: the exact step keys (action, skill, tool, params plus optional rollback_tool/rollback_params) and the special require_approval action values. It does not elaborate on save_as_template beyond what the schema already states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with '[WRITE] Create a custom workflow dynamically from a step list,' naming a specific verb, resource, and input mode. It explicitly distinguishes itself from two siblings (create_workflow_from_template and design_workflow), so an agent can route without reading schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-to-use ('you already know the steps and no built-in template matches'), names the two alternatives with the exact conditions that select them, and adds a prerequisite ('Call get_skill_catalog first for the skill and tool names'). Nothing is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_workflow_from_templateA

[WRITE] Create and save a workflow from a template, ready for run_workflow.

Takes a template name (workflow_type) plus its params, builds the steps the template defines, and persists the workflow in state "pending" — it is not a dry run. Templates are the built-in types below plus custom YAML templates (list_workflows shows both). Use this when the goal matches a template; use create_workflow instead to save your own step list when none of them fit. A custom YAML template whose destructive or unclassifiable step (or step passing confirm=True) has no require_approval gate before it is refused, naming the step and the file to fix.

Available workflow types:

  • clone_and_test: Clone VM → apply changes → monitor → approve → commit

  • incident_response: Diagnose alert → collect info → approve → remediate

  • plan_and_approve: Wrap aiops batch operations with approval gate

  • compliance_scan: Read-only health/capacity/anomaly check (no approval)

Returns: dict with workflow_id, steps summary, and plan details.

ParametersJSON Schema
NameRequiredDescriptionDefault
paramsYesWorkflow-specific parameters.
workflow_typeYesOne of the available workflow types.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety posture (write, non-idempotent, non-destructive), yet the description adds substantial context beyond them: the workflow persists in 'pending', it is explicitly not a dry run, and custom YAML templates with an unguarded destructive/unclassifiable/confirm=True step are refused with the step and file named. That refusal semantics and persistence detail go well past what the annotations convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with a [WRITE] marker and a one-sentence purpose, followed by usage, refusal conditions, type list, and returns in a logical order. The four-entry template list is the longest block but each entry carries distinct routing value, so little is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, and the description compensates by describing the return (workflow_id, steps summary, plan details). With persistence state, refusal behavior, alternative routing, and template enumeration all covered, an agent has everything needed to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline would be 3, but the description adds genuine meaning: it explains that workflow_type is a template name and enumerates the four available built-in types with their step shapes, plus notes params are workflow-specific. That fills the gap left by the schema's vague 'One of the available workflow types.'

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Create and save a workflow from a template') and immediately clarifies the output state ('pending', 'not a dry run'). It explicitly distinguishes itself from the sibling create_workflow, which saves a custom step list instead.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit selection rule: 'Use this when the goal matches a template; use create_workflow instead to save your own step list when none of them fit.' It also names list_workflows as the discovery path for both built-in and custom templates, covering when-to-use, when-not, and alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

design_workflowA

[WRITE] Start designing a workflow from a natural language description.

Call this when the user describes a complex operation and you need to design a multi-step workflow. Returns a DRAFT workflow with proposed steps for the user to review and edit before execution.

Design flow: design_workflow → update_draft (add steps, then iterate on user feedback) → confirm_draft (state becomes PENDING) → run_workflow.

Use get_skill_catalog() first to see which tools the steps may target.

Returns: dict with workflow_id (state=DRAFT), proposed steps placeholder, and instructions for the AI to fill in steps via update_draft.

ParametersJSON Schema
NameRequiredDescriptionDefault
goalYesNatural language description of what the user wants to accomplish.
constraintsNoOptional constraints (e.g. "must have approval before any destructive step", "use NSX for networking", "target is vcenter-prod").

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses the key behavior: it creates a DRAFT workflow with state=DRAFT, returns a proposed-steps placeholder for review/edit, and pushes the agent toward update_draft and confirm_draft next. The annotations already indicate a non-read-only, non-destructive write, and the description adds lifecycle context without contradicting them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured and front-loaded with the action and trigger, followed by the flow and return contract. There is minor redundancy between 'Returns a DRAFT workflow...' and the later 'Returns:' block, but the latter is more precise about the output shape.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter tool with no output schema, the description is largely complete: it names the required input, the returned workflow state, the placeholder behavior, and the follow-up flow. It does not cover error conditions or constraints details, but the schema fills the constraints gap and the missing details are minor.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, with both `goal` and `constraints` already documented. The description adds little parameter-level detail beyond explaining that the input is a natural-language description; it does not go deeper on constraints, so the schema carries the semantic weight.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a clear verb and resource ('Start designing a workflow') and explains that it converts a natural-language goal into a DRAFT workflow with proposed steps. It does not explicitly call out how it differs from similar siblings like plan_workflow or create_workflow, though the DRAFT/lifecycle wording provides some differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit trigger: use this when the user describes a complex operation needing a multi-step workflow. It also provides a design flow and instructs the agent to call get_skill_catalog for candidate tools. However, it does not mention when not to use this tool or how it compares to sibling planning tools, so it lacks explicit exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_skill_catalogA
Read-onlyIdempotent

[READ] Get the complete catalog of available skills and tools for workflow design.

Use this to understand what building blocks are available when designing a custom workflow, then feed the skill and tool names into create_workflow or update_draft steps. Note this is a static curated catalog, not a live query of each skill's registry, so it may lag a skill's actual tool list; pilot cannot call these tools itself — the calling agent does.

Returns: dict mapping skill name → {description, tools: {tool_name: {risk, desc}}}.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint false. The description adds valuable context: the catalog is static and may lag actual skill tool lists, and that the calling agent cannot invoke these tools itself. It also specifies the return structure, going beyond the annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured and concise: it starts with the purpose, then usage context, a caveat, and the return format. Each sentence earns its place, with no redundancy or fluff. The front-loading of the read nature and purpose is effective.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a no-parameter read-only catalog tool, the description is complete. It explains what is returned (dict mapping), how to use the result, and a known limitation (static data, lag). It also clarifies an important behavioral constraint (pilot cannot call these tools). Nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are zero parameters, so schema coverage is effectively 100% and the description cannot add parameter-specific details. Per the baseline for no parameters, a score of 4 is appropriate; the description does not need to compensate for any missing parameter docs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific verb ('Get') and resource ('complete catalog of available skills and tools for workflow design'). It distinguishes itself from sibling tools by explicitly positioning itself as the source for building blocks used in workflow creation, not as a tool that designs or executes workflows.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides clear context for when to use it: when designing a custom workflow, to understand available building blocks, and then feed names into create_workflow or update_draft. It also notes it is not a live query, which implicitly advises against using it when real-time registry data is needed, though it doesn't explicitly name alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_workflow_statusA
Read-onlyIdempotent

[READ] Get current workflow state, diff report, and audit log.

Use this to poll a workflow after run_workflow and to find out why one stopped: steps with status 'not_executed' are yours to perform (the server runs without --dispatch), state 'awaiting_approval' means call approve. Returns a point-in-time snapshot and does not advance the workflow.

Returns: Full workflow state including steps, audit log, and diff report.

ParametersJSON Schema
NameRequiredDescriptionDefault
workflow_idYesThe workflow ID to query.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive, and the description still adds non-obvious behavior: it is a point-in-time snapshot that 'does not advance the workflow', and it decodes status semantics into next actions. That is exactly the kind of polling/idempotency context annotations cannot express.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the scoping '[READ]' marker and the primary action, then the guidance. The trailing 'Returns:' block largely restates the opening ('Full workflow state including steps, audit log, and diff report'), a small redundancy but not enough to hurt usability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description must cover return shape, and it does: snapshot semantics plus steps, audit log, and diff report. Combined with status-to-action guidance, an agent has everything needed to call and act on this tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter and schema coverage is 100% ('The workflow ID to query'), so the schema carries the meaning. The description adds nothing about ID format or origin, which is the correct baseline when the schema already documents the parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb+resource ('Get current workflow state') and enumerates the payload (state, diff report, audit log). It is clearly distinguishable from siblings like run_workflow, cancel_workflow, and approve, which are the mutation/transition counterparts.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says when to use it ('poll a workflow after run_workflow', 'find out why one stopped') and routes the agent to alternatives based on observed state: 'not_executed' steps are the caller's job, 'awaiting_approval' means call approve. That is when/when-not plus alternative routing in two sentences.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_workflowsA
Read-onlyIdempotent

[READ] List all available workflow templates (built-in + custom).

Use this first to see whether a template already covers the goal, then pass its name to create_workflow_from_template; if none fit, use create_workflow instead. Built-in templates are always available. Custom templates are loaded from ~/.vmware/workflows/*.yaml — drop a YAML file there to add your own workflows.

Returns: dict with builtin and custom workflow lists, each with name, description, steps count, plus active_workflows — the IDs of in-flight runs to pass to get_workflow_status.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive/openWorld, so the safety profile is covered. The description still adds real behavioral context the annotations cannot: that built-in templates are always present while customs come from ~/.vmware/workflows/*.yaml, which tells the agent why a template might be missing.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the [READ] marker and the core purpose, then routing, then source behavior, then a Returns block. Every sentence carries a distinct piece of information with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, and the description compensates by enumerating the return shape: builtin and custom lists with name, description, step count, plus the active_workflows IDs to feed into get_workflow_status. That is everything an agent needs to consume the result correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes no parameters, so there is nothing for the description to disambiguate; baseline 4 applies. No misleading parameter hints are present.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('List all available workflow templates') and immediately scopes it to built-in plus custom, which no sibling does. The [READ] tag and the named siblings (create_workflow_from_template, create_workflow) let an agent place this tool without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit conditional routing: use this first, then pass a name to create_workflow_from_template, otherwise fall back to create_workflow. Both the when-to-use and the alternatives are named outright.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

review_workflowA
Read-onlyIdempotent

[READ] Sanity-check a planned workflow before execution.

Performs structural validation only — does NOT call into other skills. Catches the common authoring errors before they hit production:

  • Delete-then-use: a step deletes resource X, a later step references X

  • Missing required params: a step has empty params or placeholder values

  • Cross-skill order issues: surfacing the cross-skill dispatch sequence

  • Risk profile: count of destructive / write / read-only / unclassified steps

  • Approval coverage: is every destructive OR unclassifiable step gated behind a preceding require_approval?

Each step is placed in a tier from the tool's entry in get_skill_catalog first, then from name patterns. A step that matches neither is reported as ungated_unclassified rather than assumed safe: pilot dispatches nothing itself and cannot inspect a sibling skill's annotations, so "this tool is unknown to me" is the honest finding, and it needs the same gate a known destructive step does. The remedy is in the message — add the tool to SKILL_CATALOG with the risk its own skill declares, or add a gate.

Medium-risk writes (create / scale / enable) are classified and counted but not gated: the family gates destructive work, and several built-in templates deliberately create in staging before asking for approval.

Returns: Dict with keys: - verdict: "approved" if no structural issues, otherwise "needs_revision" - findings: list of {severity, kind, message, step_index}. Kinds ungated_destructive, ungated_unclassified and destructive_in_parallel_group are what run_workflow refuses on; force=True overrides that only for a built-in template, never for a custom workflow's missing approval gate. - summary: counts — total/destructive/write/read_only/approval_gates, parallel_groups, est_duration_min, plus classified_steps and unclassified_steps so an "approved" verdict can be told apart from a workflow this review could not read.

ParametersJSON Schema
NameRequiredDescriptionDefault
workflow_idYesThe workflow ID returned by ``create_workflow_from_template``.

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnlyHint, idempotentHint, destructiveHint=false), and the description adds substantial behavior beyond them: no cross-skill calls, tier-based classification, the deliberate decision not to gate medium-risk writes, the honest 'ungated_unclassified' fallback, and the force=True limitation for custom workflows. This is exactly the kind of non-obvious behavior an agent needs and cannot infer from structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core purpose, then organized as a scannable bullet list of the exact error classes caught, followed by a returns contract. It is longer than most descriptions, but nearly every line carries decision-relevant content; only minor prose (e.g. the medium-risk write rationale) could be tightened.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite having no output schema, the description fully specifies the return dict (verdict, findings kinds, summary counts) and explains which finding kinds cause run_workflow to refuse. Combined with the classification and gating rules, an agent has everything needed to call this and interpret its result.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is a single required parameter with 100% schema description coverage, so the schema already documents workflow_id (the ID returned by create_workflow_from_template). The description adds no format, provenance, or constraint detail beyond that, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource and an explicit scope qualifier ('[READ] Sanity-check a planned workflow before execution') and immediately bounds what it is not ('structural validation only — does NOT call into other skills'). An agent can distinguish it from design_workflow, run_workflow, and get_workflow_status without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The timing condition is stated clearly ('before execution') and the description ties findings to run_workflow's refusal behavior plus the force=True escape hatch, which effectively tells the agent when a review is load-bearing. It stops short of naming sibling alternatives (e.g. design_workflow / create_workflow_from_template) as the things to do first, so it is strong context rather than a full when/when-not routing statement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

rollbackA
Destructive

[WRITE] Abort a workflow and rollback completed steps in reverse order.

Without confirm=True this only previews: it returns blast_radius — the workflow's id, type and state, the executed steps whose rollback_tool would run with its rollback_params, secrets redacted (would_roll_back — these often carry confirm=True, and this call's confirm=True is the decision to run them), the executed steps that have none and stay applied (left_in_place), steps you performed from pending_dispatch that Pilot will not reverse (not_reversed_by_pilot: they have an undo you must call; not_reversible_outside_pilot: they have none — a clone, a production resize), undos that would leave something behind (leaves_behind on a would_roll_back entry), steps with unknown effects, and blockers — and changes nothing. Show that to the user and get their explicit decision. Do not set confirm=True on your own because the user asked for a rollback earlier — the user has not seen the preview yet.

Nothing rolls back automatically: a failed step leaves the workflow 'failed' and stops. This tool is the explicit, best-effort undo, and it only reverses steps pilot itself recorded as 'success' — without --dispatch, approval gates only. Steps YOU performed from pending_dispatch are 'not_executed' (or 'reported_done' once acknowledged at a gate) here, so to undo them call each one's rollback_tool with its rollback_params yourself, last step first. Embedders that pass a dispatcher to WorkflowExecutor get those calls made for them. Steps without a rollback_tool are reported 'not_reversible' and stay applied; an undo that does not remove what the step created is 'partial' (a clone powered off stays). A failed undo does not stop the rest. The workflow state is set to 'failed' afterwards.

Refused with confirm=True: a workflow in draft, completed, rolling_back or cancelled state, a step whose status Pilot does not recognise, a workflow record that cannot be read, and — unless acknowledge_unknown_effects=True — a step left 'running' or 'interrupted' by a Pilot process that stopped mid-dispatch (listed in unknown_effects).

Returns: Preview: {"action": "preview", "blast_radius", "hint"}. Acting: the workflow state with "action": "rolled_back" ("rollback_incomplete", with outcome "failed" and an error naming them, when an undo failed or its effect is unknown), rollback_results for each step pilot recorded as succeeded, and blast_radius. Check get_workflow_status afterwards to see which steps were actually reversed and which were skipped.

ParametersJSON Schema
NameRequiredDescriptionDefault
confirmNoFalse (default) returns the blast radius and changes nothing. True applies it.
workflow_idYesThe workflow ID to rollback.
acknowledge_unknown_effectsNoSet True only after the user has checked, in the target system, whether each unknown_effects step took effect. Covers only those steps; every other refusal still applies.

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already flag destructive/openWorld/non-idempotent, but the description goes well beyond: secrets are redacted, some steps stay applied (left_in_place), some are not_reversible or partial, a failed undo does not halt the rest, and the workflow is left in 'failed'. It also enumerates the exact refusal set and the acknowledge_unknown_effects escape hatch.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The preview/act distinction and the user-decision warning are front-loaded, and nearly every sentence carries operational weight. It is nevertheless a dense wall of parenthetical clauses that could be tightened without losing meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite no output schema, the description documents both return shapes (preview vs acting, including rollback_incomplete and rollback_results), the terminal state, and the follow-up step of checking get_workflow_status. Nothing an agent needs to call this destructive tool safely is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% so the baseline is 3, but the description adds real meaning: confirm=True is framed as the user's explicit decision (not a re-confirmation of an earlier request), and acknowledge_unknown_effects is scoped to only the unknown_effects steps and only after the user has checked the target system.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource+scope: '[WRITE] Abort a workflow and rollback completed steps in reverse order,' and defines itself as the 'explicit, best-effort undo' distinct from an automatic abort. It never names a sibling (e.g. cancel_workflow) or contrasts itself against one, so the reader must infer the boundary rather than being told it.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states the when-not: without confirm=True it only previews, the agent must show blast_radius to the user and get an explicit decision, and must not set confirm=True on its own. It also names the alternative path for pending_dispatch steps (call each step's own rollback_tool, last step first) and lists every refusal condition.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_workflowA

[WRITE] Advance a planned workflow. Pauses at approval gates.

Who runs the steps depends on how this server was started:

  • with --dispatch (or VMWARE_PILOT_DISPATCH=stdio): Pilot runs each step itself in that skill's own MCP server, stops at each approval gate, and the workflow can reach 'completed'. This call takes as long as the steps it runs. If it times out in your client, Pilot carries on (and answers nothing else until the step ends); afterwards get_workflow_status shows the outcome. NEVER perform the steps yourself.

  • without it (the default): Pilot runs nothing. Steps are recorded as 'not_executed' and the run ends with outcome='dispatch_required' (NOT 'completed'), listing each step in 'pending_dispatch'. YOU (the calling agent) must perform those skill/tool calls in order. The result says which: a 'pending_dispatch' list means you run them.

Safety: the workflow is structurally reviewed before each run. Runs are REFUSED if review finds ungated destructive or unclassifiable steps, or destructive steps inside a parallel group. For a built-in template, force=True overrides that (forced runs are written to the workflow audit log). For a custom workflow (create_workflow, design_workflow, or a YAML template) an ungated step is refused even with force=True: add a require_approval step before it instead.

When an approval gate is reached, the workflow pauses with state 'awaiting_approval'. Call approve() to continue.

Returns: Current workflow state with 'outcome' (completed | awaiting_approval | dispatch_required | failed) and, when dispatch is required, a 'pending_dispatch' list of steps for the agent to perform.

ParametersJSON Schema
NameRequiredDescriptionDefault
forceNoBypass blocking review findings on a built-in template. Use only with explicit human consent; the bypass is audited. Has no effect on a custom workflow's missing approval gate.
workflow_idYesThe workflow ID from create_workflow_from_template.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare the safety profile (not read-only, not idempotent, not destructive, open-world); the description goes far beyond by disclosing refusal conditions, the structural review gate, audit logging of forced runs, the distinction between dispatch_required and completed, and the 'NEVER perform the steps yourself' constraint. This is exactly the kind of context annotations cannot carry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The [WRITE] tag and lead sentence front-load the operation, and the dispatch branches and safety rules are clearly delineated. It is long, but nearly every sentence carries decision-relevant information; the Returns section is slightly redundant given the outcome values are already stated.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex, stateful tool with no output schema and only two parameters, the description supplies the missing return semantics ('outcome' values and 'pending_dispatch'), the caller obligations, and the failure/refusal model. An agent has everything needed to invoke and interpret this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real semantic depth to force: it overrides review findings only on built-in templates, requires human consent, is audited, and has no effect on a custom workflow's missing gate. It does not extend workflow_id beyond the schema, which is already documented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('[WRITE] Advance a planned workflow') plus a behavioral trait ('Pauses at approval gates'). It distinguishes itself from siblings by naming approve() for continuation and get_workflow_status for outcomes, so the agent can separate it from cancel/rollback/status tools without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly covers the two dispatch modes, states what happens on timeout, when force=True is permitted (and when it is not), and when to call approve(). It effectively tells the agent whether it must perform steps itself or whether Pilot will, which is the key selection/usage question.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_draftA

[WRITE] Update a DRAFT workflow's name, description, or steps.

Call this after design_workflow() to fill in the actual steps, or to modify steps based on user feedback. Use it only while the workflow is still DRAFT — after confirm_draft the steps are frozen and you must create_workflow a new one instead.

Each step dict: {action, skill, tool, params, rollback_tool?, rollback_params?} Use action="require_approval" for approval gates. A draft may be saved without them while it is being designed; the result lists every step that still needs one, and confirm_draft refuses until none are left.

Returns: Updated workflow summary for user review.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoWorkflow name (optional, updates workflow_type).
stepsNoComplete list of steps (replaces all existing steps).
descriptionNoHuman-readable description.
workflow_idYesThe draft workflow ID.

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish this is a write (readOnlyHint=false) and non-idempotent. The description adds useful behavioral detail beyond those markers: steps are replaced wholesale, approval gates are tracked, and the result lists steps still needing approval while confirm_draft will refuse until none remain. It also discloses the return value. It doesn't mention potential errors or side effects in depth, but the added lifecycle and validation context earns a solid 4.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded: the first sentence states the core purpose, then usage context, then step format, then behavioral detail. Every sentence adds necessary information for correct invocation, with no filler or redundancy. It is slightly longer than minimal but the additional context is all operational, so it earns a 4.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with 4 parameters and no output schema, the description is thorough: it covers when to call, when not to call, the step data structure, approval validation behavior, and the return format. It doesn't explain default parameter behavior or error cases, but the schema already covers parameter descriptions and the draft lifecycle constraint is fully addressed. Overall, an agent has enough context to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds value by specifying the exact step dict shape ({action, skill, tool, params, rollback_tool?, rollback_params?}) and the special action value 'require_approval' for approval gates. It also explains the consequence of omitting approval gates during design, clarifying semantics of the steps parameter beyond the schema's 'replaces all existing steps.'

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Update a DRAFT workflow's name, description, or steps,' which names a specific action, resource, and the fields it affects. It further distinguishes itself from siblings by explicitly restricting usage to the DRAFT state and pointing to create_workflow for post-confirmation workflows, so an agent can clearly tell it apart without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit when-to-use guidance: 'Call this after design_workflow() to fill in the actual steps, or to modify steps based on user feedback.' It also states a hard when-not-to-use rule: after confirm_draft the steps are frozen and one must create_workflow a new one instead. This covers both the appropriate trigger and the exclusion condition, leaving no ambiguity.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 1 tool updatev2.4.0
    • Changedapprove1 field changed
      • addedInput schema / properties / acknowledge_not_executed
        Added value: +{
        +  "default": false,
        +  "description": "True says the steps before this gate were performed outside Pilot and their results were checked.",
        +  "title": "Acknowledge Not Executed",
        +  "type": "boolean"
        +}
  2. 5 tool updatesv2.3.0
    • Changedapprove1 field changed
      • changedInput schema / properties / approver / description
        Previous value: -"Name of the person approving (for audit trail)."New value: +"Name of the person approving (for audit trail). Over an authenticated network transport leave it empty: the approver is the caller's token name, and a different name is refused."
    • Addedcreate_workflow_from_template
    • Removedplan_workflow
    • Changedreview_workflow1 field changed
      • changedInput schema / properties / workflow_id / description
        Previous value: -"The workflow ID returned by ``plan_workflow``."New value: +"The workflow ID returned by ``create_workflow_from_template``."
    • Changedrun_workflow1 field changed
      • changedInput schema / properties / workflow_id / description
        Previous value: -"The workflow ID from plan_workflow."New value: +"The workflow ID from create_workflow_from_template."
  3. 2 tool updatesv1.12.0
    • Changedcancel_workflow2 fields changed
      • addedInput schema / properties / acknowledge_unknown_effects
        Added value: +{
        +  "default": false,
        +  "description": "Set True only after the user has checked, in the target system, whether each unknown_effects step took effect. Covers only those steps; every other refusal still applies.",
        +  "title": "Acknowledge Unknown Effects",
        +  "type": "boolean"
        +}
      • addedInput schema / properties / confirm
        Added value: +{
        +  "default": false,
        +  "description": "False (default) returns the blast radius and changes nothing. True applies it.",
        +  "title": "Confirm",
        +  "type": "boolean"
        +}
    • Changedrollback2 fields changed
      • addedInput schema / properties / acknowledge_unknown_effects
        Added value: +{
        +  "default": false,
        +  "description": "Set True only after the user has checked, in the target system, whether each unknown_effects step took effect. Covers only those steps; every other refusal still applies.",
        +  "title": "Acknowledge Unknown Effects",
        +  "type": "boolean"
        +}
      • addedInput schema / properties / confirm
        Added value: +{
        +  "default": false,
        +  "description": "False (default) returns the blast radius and changes nothing. True applies it.",
        +  "title": "Confirm",
        +  "type": "boolean"
        +}
  4. 13 tool updatesv1.11.1
    • Changedapprove3 fields changed
      • addedInput schema / additionalProperties
        Added value: +false
      • addedInput schema / properties / approver / description
        Added value: +"Name of the person approving (for audit trail)."
      • addedInput schema / properties / workflow_id / description
        Added value: +"The workflow ID to approve."
    • Changedcancel_workflow3 fields changed
      • addedInput schema / additionalProperties
        Added value: +false
      • addedInput schema / properties / reason / description
        Added value: +"Optional human-readable reason (e.g. \"approval rejected by on-call\"), recorded in the audit log."
      • addedInput schema / properties / workflow_id / description
        Added value: +"The workflow ID to cancel."
    • Changedconfirm_draft3 fields changed
      • addedInput schema / additionalProperties
        Added value: +false
      • addedInput schema / properties / save_as_template / description
        Added value: +"If True, save to ~/.vmware/workflows/ for reuse. Refused when the draft's name is a built-in template's name."
      • addedInput schema / properties / workflow_id / description
        Added value: +"The draft workflow ID to confirm."
    • Changedcreate_workflow5 fields changed
      • addedInput schema / additionalProperties
        Added value: +false
      • addedInput schema / properties / description / description
        Added value: +"Human-readable description."
      • addedInput schema / properties / name / description
        Added value: +"Workflow name (used as workflow_type)."
      • addedInput schema / properties / save_as_template / description
        Added value: +"If True, save as YAML to ~/.vmware/workflows/ for reuse. Refused when name is a built-in template's name, since the saved template would replace the built-in."
      • addedInput schema / properties / steps / description
        Added value: +"List of step dicts, each with action/skill/tool/params."
    • Changeddesign_workflow3 fields changed
      • addedInput schema / additionalProperties
        Added value: +false
      • addedInput schema / properties / constraints / description
        Added value: +"Optional constraints (e.g. \"must have approval before any destructive step\", \"use NSX for networking\", \"target is vcenter-prod\")."
      • addedInput schema / properties / goal / description
        Added value: +"Natural language description of what the user wants to accomplish."
    • Changedget_skill_catalog1 field changed
      • addedInput schema / additionalProperties
        Added value: +false
    • Changedget_workflow_status2 fields changed
      • addedInput schema / additionalProperties
        Added value: +false
      • addedInput schema / properties / workflow_id / description
        Added value: +"The workflow ID to query."
    • Changedlist_workflows1 field changed
      • addedInput schema / additionalProperties
        Added value: +false
    • Changedplan_workflow3 fields changed
      • addedInput schema / additionalProperties
        Added value: +false
      • addedInput schema / properties / params / description
        Added value: +"Workflow-specific parameters."
      • addedInput schema / properties / workflow_type / description
        Added value: +"One of the available workflow types."
    • Changedreview_workflow2 fields changed
      • addedInput schema / additionalProperties
        Added value: +false
      • addedInput schema / properties / workflow_id / description
        Added value: +"The workflow ID returned by ``plan_workflow``."
    • Changedrollback2 fields changed
      • addedInput schema / additionalProperties
        Added value: +false
      • addedInput schema / properties / workflow_id / description
        Added value: +"The workflow ID to rollback."
    • Changedrun_workflow3 fields changed
      • addedInput schema / additionalProperties
        Added value: +false
      • addedInput schema / properties / force / description
        Added value: +"Bypass blocking review findings on a built-in template. Use only with explicit human consent; the bypass is audited. Has no effect on a custom workflow's missing approval gate."
      • addedInput schema / properties / workflow_id / description
        Added value: +"The workflow ID from plan_workflow."
    • Changedupdate_draft5 fields changed
      • addedInput schema / additionalProperties
        Added value: +false
      • addedInput schema / properties / description / description
        Added value: +"Human-readable description."
      • addedInput schema / properties / name / description
        Added value: +"Workflow name (optional, updates workflow_type)."
      • addedInput schema / properties / steps / description
        Added value: +"Complete list of steps (replaces all existing steps)."
      • addedInput schema / properties / workflow_id / description
        Added value: +"The draft workflow ID."
  5. 2 tool updatesv1.5.38
    • Addedcancel_workflow
    • Changedrun_workflow1 field changed
      • addedInput schema / properties / force
        Added value: +{
        +  "default": false,
        +  "title": "Force",
        +  "type": "boolean"
        +}
  6. 12 tool updatesv1.5.22
    • First observedapprove
    • First observedconfirm_draft
    • First observedcreate_workflow
    • First observeddesign_workflow
    • First observedget_skill_catalog
    • First observedget_workflow_status
    • First observedlist_workflows
    • First observedplan_workflow
    • First observedreview_workflow
    • First observedrollback
    • First observedrun_workflow
    • First observedupdate_draft

TDQS

A4.4/5.0

Scored across 13 tools

Disambiguation4/5

Most tools target distinct lifecycle phases (design, draft, confirm, run, approve, cancel, rollback) and descriptions explicitly cross-reference each other to guide selection, e.g. cancel only stops future steps while rollback reverses completed ones. The residual ambiguity is the three creation entry points (create_workflow, create_workflow_from_template, design_workflow), which all start a workflow but are differentiated by intent (known steps, matching template, or a goal).

Naming Consistency4/5

The set mostly follows a predictable verb_noun pattern (cancel_workflow, run_workflow, create_workflow, review_workflow, update_draft, list_workflows). Two runtime gates break the pattern with bare verbs (approve, rollback) and get_workflow_status is verb_noun_noun, but these are minor, still-readable deviations.

Tool Count5/5

13 tools is well-scoped for a workflow orchestration server; each maps to a distinct lifecycle action (create/design/draft/confirm/review/run/approve/cancel/rollback/status/list/catalog). No redundant or filler tools.

Completeness5/5

The surface covers a full workflow lifecycle: discovery (list_workflows, get_skill_catalog), authoring (design_workflow, create_workflow, create_workflow_from_template, update_draft), validation (review_workflow), gating/execution (confirm_draft, run_workflow, approve), and termination (cancel_workflow, rollback), plus status. The only arguable gap is a hard delete for abandoned drafts, which cancel/terminal-state handling largely covers.

Maintenance

ActivityActive
ResponsivenessResponsive

Related MCP Connectors

Related MCP Servers

  • A
    license
    A
    quality
    A
    maintenance
    VMware vSphere compliance and hardening — read-only baseline scanning plus drift detection across CIS, DISA STIG, vSphere SCG, China DJCP 2.0, and PCI-DSS frameworks. Includes LLM-powered remediation suggestions; apply-side gated through the vmware-pilot approval workflow.
    8
    696 PyPI
    4
    MIT