batch_simulate_execution

Simulate multiple code examples in batch to validate documentation examples efficiently. Use this tool to test all examples in a documentation file at once with configurable simulation options.

Instructions

Simulate execution of multiple code examples in batch. Useful for validating all examples in a documentation file at once.

Input Schema

TableJSON Schema

Name	Required	Description	Default
`examples`	Yes	Array of examples to simulate
`globalOptions`	No	Options applied to all simulations

Implementation Reference

src/tools/simulate-execution.ts:473-553 (handler)

The primary handler function that implements the logic for the batch_simulate_execution tool. It processes an array of code examples by simulating each one's execution using the single simulate_execution handler, aggregates results, computes pass/fail statistics, and returns a batched response.

export async function handleBatchSimulateExecution(
  args: {
    examples: Array<{
      code: string;
      implementationPath?: string;
      expectedBehavior?: string;
    }>;
    globalOptions?: SimulationOptions;
  },
  context?: any,
): Promise<{
  success: boolean;
  results: SimulateExecutionResult[];
  summary: {
    total: number;
    passed: number;
    failed: number;
    averageConfidence: number;
  };
}> {
  await context?.info?.(
    `🔬 Starting batch simulation of ${args.examples.length} example(s)...`,
  );

  const results: SimulateExecutionResult[] = [];
  let totalConfidence = 0;

  for (let i = 0; i < args.examples.length; i++) {
    const example = args.examples[i];
    await context?.info?.(
      `📝 Simulating example ${i + 1}/${args.examples.length}...`,
    );

    let implCode: string | undefined;
    if (example.implementationPath) {
      try {
        implCode = await fs.readFile(example.implementationPath, "utf-8");
      } catch {
        // Use example as implementation
      }
    }

    const result = await handleSimulateExecution(
      {
        exampleCode: example.code,
        implementationCode: implCode,
        expectedBehavior: example.expectedBehavior,
        options: args.globalOptions,
      },
      context,
    );

    results.push(result);
    totalConfidence += result.trace.confidenceScore;
  }

  const passed = results.filter(
    (r) =>
      r.success &&
      r.trace.potentialIssues.filter((i) => i.severity === "error").length ===
        0,
  ).length;
  const failed = results.length - passed;
  const averageConfidence =
    results.length > 0 ? totalConfidence / results.length : 0;

  await context?.info?.(
    `✅ Batch simulation complete: ${passed} passed, ${failed} failed`,
  );

  return {
    success: failed === 0,
    results,
    summary: {
      total: results.length,
      passed,
      failed,
      averageConfidence,
    },
  };
}

src/tools/simulate-execution.ts:431-468 (registration)

The Tool object registration for batch_simulate_execution, defining the tool's name, description, and input schema for batch processing of code examples.

export const batchSimulateExecution: Tool = {
  name: "batch_simulate_execution",
  description:
    "Simulate execution of multiple code examples in batch. " +
    "Useful for validating all examples in a documentation file at once.",
  inputSchema: {
    type: "object",
    properties: {
      examples: {
        type: "array",
        items: {
          type: "object",
          properties: {
            code: {
              type: "string",
              description: "The code example",
            },
            implementationPath: {
              type: "string",
              description: "Path to implementation file",
            },
            expectedBehavior: {
              type: "string",
              description: "Expected behavior description",
            },
          },
          required: ["code"],
        },
        description: "Array of examples to simulate",
      },
      globalOptions: {
        type: "object",
        description: "Options applied to all simulations",
      },
    },
    required: ["examples"],
  },
};

documcp