multi_browserbase_stagehand_observe_session
Identifies interactive web page elements like buttons, links, and form fields for subsequent automation actions using specific visual or functional instructions.
Instructions
Observes and identifies specific interactive elements on the current web page that can be used for subsequent actions. This tool is specifically designed for finding actionable (interactable) elements such as buttons, links, form fields, dropdowns, checkboxes, and other UI components that you can interact with. Use this tool when you need to locate elements before performing actions with the act tool. DO NOT use this tool for extracting text content or data - use the extract tool instead for that purpose. The observe tool returns detailed information about the identified elements including their properties, location, and interaction capabilities. This information can then be used to craft precise actions. The more specific your observation instruction, the more accurate the element identification will be. Think of this as your 'eyes' on the page to find exactly what you need to interact with. (for a specific session)
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| sessionId | Yes | The session ID to use | |
| instruction | Yes | Detailed instruction for what specific elements or components to observe on the web page. This instruction must be extremely specific and descriptive. For example: 'Find the red login button in the top right corner', 'Locate the search input field with placeholder text', or 'Identify all clickable product cards on the page'. The more specific and detailed your instruction, the better the observation results will be. Avoid generic instructions like 'find buttons' or 'see elements'. Instead, describe the visual characteristics, location, text content, or functionality of the elements you want to observe. This tool is designed to help you identify interactive elements that you can later use with the act tool for performing actions like clicking, typing, or form submission. | |
| returnAction | No | Whether to return the action to perform on the element. If true, the action will be returned as a string. If false, the action will not be returned. |
Implementation Reference
- src/tools/multiSession.ts:257-260 (registration)Registers the 'multi_browserbase_stagehand_observe_session' tool by creating a session-aware wrapper around the base observeTool using the createMultiSessionAwareTool factory. The name is constructed as 'multi_' + 'browserbase_stagehand_observe' + '_session'.
export const observeWithSessionTool = createMultiSessionAwareTool(observeTool, { namePrefix: "multi_", nameSuffix: "_session", }); - src/tools/multiSession.ts:55-77 (handler)Handler implementation for all multi-session tools, including 'multi_browserbase_stagehand_observe_session'. Retrieves the specified session from the store, creates a session-specific context, and delegates execution to the original tool's handler.
handle: async ( context: Context, params: z.infer<typeof newInputSchema>, ): Promise<ToolResult> => { const { sessionId, ...originalParams } = params; // Get the session const session = stagehandStore.get(sessionId); if (!session) { throw new Error(`Session ${sessionId} not found`); } // Create a temporary context that points to the specific session const sessionContext = Object.create(context); sessionContext.currentSessionId = session.metadata?.bbSessionId || sessionId; sessionContext.getStagehand = async () => session.stagehand; sessionContext.getActivePage = async () => session.page; sessionContext.getActiveBrowser = async () => session.browser; // Call the original tool's handler with the session-specific context return originalTool.handle(sessionContext, originalParams); }, - src/tools/multiSession.ts:50-54 (schema)Dynamic schema generation for the multi-session tool, including construction of the exact name 'multi_browserbase_stagehand_observe_session' and input schema that adds 'sessionId' to the original schema.
schema: { name: `${namePrefix}${originalTool.schema.name}${nameSuffix}`, description: `${originalTool.schema.description} (for a specific session)`, inputSchema: newInputSchema, }, - src/tools/observe.ts:44-75 (handler)Core handler logic for the base 'browserbase_stagehand_observe' tool, which is delegated to by the multi-session wrapper. Performs the actual page observation using Stagehand's observe method.
async function handleObserve( context: Context, params: ObserveInput, ): Promise<ToolResult> { const action = async (): Promise<ToolActionResult> => { try { const stagehand = await context.getStagehand(); const observations = await stagehand.page.observe({ instruction: params.instruction, returnAction: params.returnAction, }); return { content: [ { type: "text", text: `Observations: ${JSON.stringify(observations)}`, }, ], }; } catch (error) { const errorMsg = error instanceof Error ? error.message : String(error); throw new Error(`Failed to observe: ${errorMsg}`); } }; return { action, waitForNetwork: false, }; } - src/tools/index.ts:33-33 (registration)Includes the 'multi_browserbase_stagehand_observe_session' tool (as observeWithSessionTool) in the multiSessionTools array, which is part of the central TOOLS export used for MCP tool registration.
observeWithSessionTool,