Skip to main content
Glama

browserbase_stagehand_observe

Identifies interactive elements like buttons, form fields, and links on web pages to prepare for automated actions such as clicking, typing, or form submission.

Instructions

Observes and identifies specific interactive elements on the current web page that can be used for subsequent actions. This tool is specifically designed for finding actionable (interactable) elements such as buttons, links, form fields, dropdowns, checkboxes, and other UI components that you can interact with. Use this tool when you need to locate elements before performing actions with the act tool. DO NOT use this tool for extracting text content or data - use the extract tool instead for that purpose. The observe tool returns detailed information about the identified elements including their properties, location, and interaction capabilities. This information can then be used to craft precise actions. The more specific your observation instruction, the more accurate the element identification will be. Think of this as your 'eyes' on the page to find exactly what you need to interact with.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
instructionYesDetailed instruction for what specific elements or components to observe on the web page. This instruction must be extremely specific and descriptive. For example: 'Find the red login button in the top right corner', 'Locate the search input field with placeholder text', or 'Identify all clickable product cards on the page'. The more specific and detailed your instruction, the better the observation results will be. Avoid generic instructions like 'find buttons' or 'see elements'. Instead, describe the visual characteristics, location, text content, or functionality of the elements you want to observe. This tool is designed to help you identify interactive elements that you can later use with the act tool for performing actions like clicking, typing, or form submission.
returnActionNoWhether to return the action to perform on the element. If true, the action will be returned as a string. If false, the action will not be returned.

Implementation Reference

  • The handler function that executes the core logic of the 'browserbase_stagehand_observe' tool. It retrieves the stagehand instance from context and calls observe on the page with the provided instruction and optional returnAction flag, returning the observations as a text content result.
    async function handleObserve( context: Context, params: ObserveInput, ): Promise<ToolResult> { const action = async (): Promise<ToolActionResult> => { try { const stagehand = await context.getStagehand(); const observations = await stagehand.page.observe({ instruction: params.instruction, returnAction: params.returnAction, }); return { content: [ { type: "text", text: `Observations: ${JSON.stringify(observations)}`, }, ], }; } catch (error) { const errorMsg = error instanceof Error ? error.message : String(error); throw new Error(`Failed to observe: ${errorMsg}`); } }; return { action, waitForNetwork: false, }; }
  • The tool schema defining the name 'browserbase_stagehand_observe', detailed description, and reference to the input schema.
    const observeSchema: ToolSchema<typeof ObserveInputSchema> = { name: "browserbase_stagehand_observe", description: "Observes and identifies specific interactive elements on the current web page that can be used for subsequent actions. " + "This tool is specifically designed for finding actionable (interactable) elements such as buttons, links, form fields, " + "dropdowns, checkboxes, and other UI components that you can interact with. Use this tool when you need to locate " + "elements before performing actions with the act tool. DO NOT use this tool for extracting text content or data - " + "use the extract tool instead for that purpose. The observe tool returns detailed information about the identified " + "elements including their properties, location, and interaction capabilities. This information can then be used " + "to craft precise actions. The more specific your observation instruction, the more accurate the element identification " + "will be. Think of this as your 'eyes' on the page to find exactly what you need to interact with.", inputSchema: ObserveInputSchema, };
  • The Zod input schema defining the parameters for the observe tool: instruction (required string) and optional returnAction (boolean).
    const ObserveInputSchema = z.object({ instruction: z .string() .describe( "Detailed instruction for what specific elements or components to observe on the web page. " + "This instruction must be extremely specific and descriptive. For example: 'Find the red login button " + "in the top right corner', 'Locate the search input field with placeholder text', or 'Identify all " + "clickable product cards on the page'. The more specific and detailed your instruction, the better " + "the observation results will be. Avoid generic instructions like 'find buttons' or 'see elements'. " + "Instead, describe the visual characteristics, location, text content, or functionality of the elements " + "you want to observe. This tool is designed to help you identify interactive elements that you can " + "later use with the act tool for performing actions like clicking, typing, or form submission.", ), returnAction: z .boolean() .optional() .describe( "Whether to return the action to perform on the element. If true, the action will be returned as a string. " + "If false, the action will not be returned.", ), });
  • Includes the observeTool in the TOOLS export array, which is imported and used for registering all tools to the MCP server.
    export const TOOLS = [ ...multiSessionTools, ...sessionTools, navigateTool, actTool, extractTool, observeTool, screenshotTool, getUrlTool, ];
  • src/index.ts:192-222 (registration)
    Registers all tools from the TOOLS array to the MCP server by iterating over them and calling server.tool with the tool's schema.name ('browserbase_stagehand_observe' for this tool), description, input schema shape, and a wrapper handler that delegates to context.run(tool, params). This is the final MCP tool registration point.
    const tools: MCPToolsArray = [...TOOLS]; // Register each tool with the Smithery server tools.forEach((tool) => { if (tool.schema.inputSchema instanceof z.ZodObject) { server.tool( tool.schema.name, tool.schema.description, tool.schema.inputSchema.shape, async (params: z.infer<typeof tool.schema.inputSchema>) => { try { const result = await context.run(tool, params); return result; } catch (error) { const errorMessage = error instanceof Error ? error.message : String(error); process.stderr.write( `[Smithery Error] ${new Date().toISOString()} Error running tool ${tool.schema.name}: ${errorMessage}\n`, ); throw new Error( `Failed to run tool '${tool.schema.name}': ${errorMessage}`, ); } }, ); } else { console.warn( `Tool "${tool.schema.name}" has an input schema that is not a ZodObject. Schema type: ${tool.schema.inputSchema.constructor.name}`, ); } });

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/vaibhavAtlys/mcp-server-browserbase'

If you have feedback or need assistance with the MCP directory API, please join our Discord server