act
Control Scrapybara instances using mouse/keyboard actions and bash commands to execute tasks like web browsing, data extraction, or system operations based on provided prompts.
Instructions
Take action on a Scrapybara instance through an agent. The agent can control the instance with mouse/keyboard and bash commands.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| instance_id | Yes | The ID of the instance to act on. | |
| prompt | Yes | The prompt to act on. <EXAMPLES> - Go to https://ycombinator.com/companies, set batch filter to W25, and extract all company names. - Find the best way to contact Scrapybara. - Order a Big Mac from McDonald's on Doordash. </EXAMPLES> | |
| schema | No | Optional schema if you want to extract structured output. |
Implementation Reference
- src/index.ts:206-242 (handler)Handler for the 'act' tool: parses arguments with ActSchema, gets the instance, sets up tools (computer, bash, edit), calls client.act with model, tools, system prompt, and returns the response.case "act": { const args = ActSchema.parse(request.params.arguments); const instance = await client.get(args.instance_id, { abortSignal: currentController.signal, }); const tools: Scrapybara.Tool[] = [computerTool(instance)]; if (instance instanceof UbuntuInstance) { tools.push(bashTool(instance)); tools.push(editTool(instance)); } const actResponse = await client.act({ model: actModel, tools, system: actSystem, prompt: args.prompt, schema: args.schema, requestOptions: { abortSignal: currentController.signal, }, }); return { content: [ { type: "text", text: JSON.stringify( { text: actResponse.text, output: actResponse.output }, null, 2 ), } as TextContent, ], }; }
- src/schemas.ts:25-38 (schema)Zod schema definition for the 'act' tool inputs: instance_id, prompt, optional schema.export const ActSchema = z.object({ instance_id: z.string().describe("The ID of the instance to act on."), prompt: z.string().describe(`The prompt to act on. <EXAMPLES> - Go to https://ycombinator.com/companies, set batch filter to W25, and extract all company names. - Find the best way to contact Scrapybara. - Order a Big Mac from McDonald's on Doordash. </EXAMPLES> `), schema: z .any() .optional() .describe("Optional schema if you want to extract structured output."), });
- src/index.ts:89-98 (registration)Registration of the 'act' tool in the list of tools returned by ListToolsRequestHandler.name: "bash", description: "Run a bash command in a Scrapybara instance.", inputSchema: zodToJsonSchema(BashSchema), }, { name: "act", description: "Take action on a Scrapybara instance through an agent. The agent can control the instance with mouse/keyboard and bash commands.", inputSchema: zodToJsonSchema(ActSchema), },
- src/index.ts:33-38 (helper)Configuration of the actModel based on environment variable ACT_MODEL, defaulting to Anthropic.let actModel = process.env.ACT_MODEL === "anthropic" ? anthropic() : process.env.ACT_MODEL === "openai" ? openai() : anthropic(); // Default to Anthropic
- src/index.ts:40-45 (helper)Configuration of the actSystem prompt based on environment variable ACT_MODEL.let actSystem = process.env.ACT_MODEL === "anthropic" ? ANTHROPIC_UBUNTU_SYSTEM_PROMPT : process.env.ACT_MODEL === "openai" ? OPENAI_UBUNTU_SYSTEM_PROMPT : ANTHROPIC_UBUNTU_SYSTEM_PROMPT; // Default to Anthropic's prompt