MCP Webcam Server

Instructions

Implementation Reference

• webcam-server-factory.ts:159-234 (registration)

  MCP tool registration for 'capture': defines the tool name, description, empty input schema, metadata, and inline asynchronous handler that coordinates with browser clients via SSE to capture webcam image and return it as base64 image content.

  mcpServer.tool(
    "capture",
    "Gets the latest picture from the webcam. You can use this " +
      " if the human asks questions about their immediate environment,  " +
      "if you want to see the human or to examine an object they may be " +
      "referring to or showing you.",
    {},
    {
      openWorldHint: true,
      readOnlyHint: true,
      title: "Take a Picture from the webcam",
    },
    async () => {
      const userClients = getUserClients(user);
      if (userClients.size === 0) {
        return {
          isError: true,
          content: [
            {
              type: "text",
              text: `Have you opened your web browser?. Direct the human to go to ${getMcpHost()}${user !== 'default' ? `?user=${user}` : ''}, switch on their webcam and try again.`,
            },
          ],
        };
      }
  
      const clientId = Array.from(userClients.keys())[0];
  
      if (!clientId) {
        throw new Error("No clients connected");
      }
  
      const userCallbacks = getUserCallbacks(user);
  
      // Modified promise to handle both success and error cases
      const result = await new Promise<string | { error: string }>(
        (resolve) => {
          Logger.info(`Capturing for ${clientId} (user: ${user}`);
          userCallbacks.set(clientId, resolve);
  
          userClients
            .get(clientId)
            ?.write(`data: ${JSON.stringify({ type: "capture" })}\n\n`);
        }
      );
  
      // Handle error case
      if (typeof result === "object" && "error" in result) {
        return {
          isError: true,
          content: [
            {
              type: "text",
              text: `Failed to capture: ${result.error}`,
            },
          ],
        };
      }
  
      const { mimeType, base64Data } = parseDataUrl(result);
  
      return {
        content: [
          {
            type: "text",
            text: "Here is the latest image from the Webcam",
          },
          {
            type: "image",
            data: base64Data,
            mimeType: mimeType,
          },
        ],
      };
    }
  );

• webcam-server-factory.ts:171-234 (handler)

  The core handler function for the 'capture' tool. It checks for connected clients, sends a Server-Sent Event (SSE) with type 'capture' to trigger webcam capture in the browser, awaits the result via callback, parses the data URL, and returns the image or error.

    async () => {
      const userClients = getUserClients(user);
      if (userClients.size === 0) {
        return {
          isError: true,
          content: [
            {
              type: "text",
              text: `Have you opened your web browser?. Direct the human to go to ${getMcpHost()}${user !== 'default' ? `?user=${user}` : ''}, switch on their webcam and try again.`,
            },
          ],
        };
      }
  
      const clientId = Array.from(userClients.keys())[0];
  
      if (!clientId) {
        throw new Error("No clients connected");
      }
  
      const userCallbacks = getUserCallbacks(user);
  
      // Modified promise to handle both success and error cases
      const result = await new Promise<string | { error: string }>(
        (resolve) => {
          Logger.info(`Capturing for ${clientId} (user: ${user}`);
          userCallbacks.set(clientId, resolve);
  
          userClients
            .get(clientId)
            ?.write(`data: ${JSON.stringify({ type: "capture" })}\n\n`);
        }
      );
  
      // Handle error case
      if (typeof result === "object" && "error" in result) {
        return {
          isError: true,
          content: [
            {
              type: "text",
              text: `Failed to capture: ${result.error}`,
            },
          ],
        };
      }
  
      const { mimeType, base64Data } = parseDataUrl(result);
  
      return {
        content: [
          {
            type: "text",
            text: "Here is the latest image from the Webcam",
          },
          {
            type: "image",
            data: base64Data,
            mimeType: mimeType,
          },
        ],
      };
    }
  );

• webcam-server-factory.ts:22-31 (helper)

  Helper function to parse data URLs received from browser captures into mimeType and base64 components, used in both tool handler and resource reader.

  function parseDataUrl(dataUrl: string): ParsedDataUrl {
    const matches = dataUrl.match(/^data:([^;]+);base64,(.+)$/);
    if (!matches) {
      throw new Error("Invalid data URL format");
    }
    return {
      mimeType: matches[1],
      base64Data: matches[2],
    };
  }

• webcam-server-factory.ts:70-75 (helper)

  Helper to retrieve or initialize per-user map of capture callbacks, used to await results from browser capture requests.

  export function getUserCallbacks(user: string): Map<string, (response: string | { error: string }) => void> {
    if (!captureCallbacks.has(user)) {
      captureCallbacks.set(user, new Map());
    }
    return captureCallbacks.get(user)!;
  }

• webcam-server-factory.ts:12-15 (helper)

  Global map storing capture result callbacks, scoped by user and clientId, essential for asynchronous coordination between server tool calls and browser responses.

  export let captureCallbacks = new Map<
    string,
    Map<string, (response: string | { error: string }) => void>
  >(); // user -> clientId -> callback