Skip to main content
Glama
Hardik-Singh

Invariance MCP

Official
by Hardik-Singh
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Remote

cortex_run_job

Run evaluation, counterfactual, experiment, and attribution jobs by specifying job kind, target, and criteria; returns a job ID and status for tracking.

Instructions

Enqueue a generic Cortex job (evals, counterfactuals, experiments, attributions). The actor is resolved server-side from the API key (a key bound to an agent_id runs as that agent; otherwise as the api_key actor). The platform ACL-filters input_refs and target access before prompt construction. Returns {job_id, status} plus, when a synchronous MVP runner completes the job inline, the validated result.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
optionsNoJSON object of execution options. Keys: use_llm (boolean), create_surface_item (boolean), timeout_ms (number), dedupe_key (string). Example: {"use_llm":true,"create_surface_item":false}
criteriaNoJSON object describing what the eval is optimizing for. Free-form per job_kind, but typical keys: optimize_for (string[]), constraints (string[]), pass_threshold (number 0..1). Example: {"optimize_for":["resolution_time"],"constraints":["do_not_expose_private_evidence"]}
job_kindYesWhat kind of Cortex job to run. workflow_eval = check workflow behavior against criteria. counterfactual_eval = estimate what would have happened under a changed assumption (HYPOTHESIS, not fact). workflow_experiment = compare variants. outcome_attribution = explain why something succeeded/failed. recommendation_impact_eval / prompt_variant_eval / policy_eval = specialized variants.
questionNoFree-form question the job should answer. Required for counterfactual_eval (e.g. "What if Alice had owned this escalation earlier?"). Optional for workflow_eval where criteria suffice.
input_refsNoJSON object of evidence references the runner may use. Optional keys: run_ids (string[]), case_ids (string[]), node_ids (string[]), chunk_ids (string[]), surface_item_ids (string[]). The platform ACL-filters these before prompt construction; refs the caller cannot access are dropped or the job is denied. Example: {"run_ids":["run_1"],"case_ids":["case_123"]}
project_idYesProject ID. The platform uses this for target access checks and evidence filtering.
target_refYesStable reference to the target. For internal target_types this is the platform ID (e.g. "case_123"); for "external" it's the caller's ID for the object described in input_payload.
target_typeYesType of object being evaluated. Use "external" with input_payload when the target lives outside Invariance.
input_payloadNoJSON object for inline target data. Required when target_type="external" (the target isn't a row in our DB). Example: {"workflow_name":"refund approval","steps":[]}

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.3.0

TDQS

A3.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes well beyond the annotations by disclosing that the actor is resolved server-side from the API key, that the platform ACL-filters input_refs and target access before prompt construction, and that the return value is {job_id, status} plus an optional inline result when a synchronous runner completes. This is meaningful behavioral context for auth, access control, and async execution that the annotations do not provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, all dense and purposeful: the tool's core action is front-loaded, followed by actor resolution, ACL filtering, and return behavior. Every sentence earns its place, and there is no redundant repetition of schema content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 9 parameters, no output schema, and async behavior, the description covers the essential behavioral context: actor resolution, ACL filtering, and the return shape. It does not explicitly point to follow-up tools like cortex_get_job or cortex_get_result for asynchronous jobs, but the schema's rich per-parameter descriptions fill most practical gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3 without any additional parameter explanation. The description adds a useful cross-cutting detail about input_refs and target access being ACL-filtered, but it does not meaningfully clarify individual parameters beyond what the schema already documents. This is adequate but not exceptional.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Enqueue a generic Cortex job.' It enumerates the job categories (evals, counterfactuals, experiments, attributions), so the agent can tell this is the umbrella runner rather than a specialized sibling. It does not explicitly name the specialized siblings like cortex_run_eval or cortex_run_counterfactual, so it stops just short of full sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description says this is the 'generic' Cortex job runner but gives no explicit when-to-use versus alternatives, no exclusions, and no pointer to specialized helpers such as cortex_run_eval or cortex_run_counterfactual. The word 'generic' implies breadth, but the agent is left to infer when a specialized sibling would be a better fit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools