Scenic MCP

Instructions

Implementation Reference

• src/tools.ts:491-582 (handler)

  The handler function for 'inspect_viewport' tool. Checks connection, sends 'inspect_viewport' command to Elixir server, parses response, and formats a detailed text summary of the viewport including visual description, semantic elements, clickable items, etc.

  async function handleInspectViewport(args: any) {
    try {
      const isRunning = await conn.checkTCPServer();
      if (!isRunning) {
        return {
          content: [
            {
              type: 'text',
              text: 'Cannot inspect viewport: No Scenic application connected.\n\nStart your Scenic application first to inspect its interface.',
            },
          ],
          isError: false,
        };
      }
  
      const command = {
        action: 'inspect_viewport',
      };
  
      const response = await conn.sendToElixir(command);
      const data = JSON.parse(response);
  
      if (data.error) {
        return {
          content: [
            {
              type: 'text',
              text: `Error inspecting viewport: ${data.error}`,
            },
          ],
          isError: true,
        };
      }
  
      let inspectionText = `Viewport Inspection Results\n${'='.repeat(50)}\n\n`;
  
      if (data.visual_description) {
        inspectionText += `Visual Description: ${data.visual_description}\n`;
        inspectionText += `Script Count: ${data.script_count}\n\n`;
      }
  
      if (data.semantic_elements && data.semantic_elements.count > 0) {
        inspectionText += `Semantic DOM Summary\n${'-'.repeat(30)}\n`;
        inspectionText += `Total Elements: ${data.semantic_elements.count}\n`;
        inspectionText += `Clickable Elements: ${data.semantic_elements.clickable_count}\n`;
  
        if (data.semantic_elements.summary) {
          inspectionText += `Summary: ${data.semantic_elements.summary}\n`;
        }
  
        if (data.semantic_elements.by_type && Object.keys(data.semantic_elements.by_type).length > 0) {
          inspectionText += `\nElements by Type:\n`;
          for (const [type, count] of Object.entries(data.semantic_elements.by_type)) {
            inspectionText += `  - ${type}: ${count}\n`;
          }
        }
  
        const clickableElements = data.semantic_elements.elements?.filter((e: any) => e.clickable) || [];
        if (clickableElements.length > 0) {
          inspectionText += `\nClickable Elements:\n`;
          clickableElements.forEach((elem: any) => {
            const posStr = elem.position ? ` at (${elem.position.x}, ${elem.position.y})` : '';
            inspectionText += `  - ${elem.label || elem.type}${posStr}\n`;
            if (elem.description) {
              inspectionText += `    ${elem.description}\n`;
            }
          });
        }
      } else {
        inspectionText += `\nNo semantic DOM information available.\n`;
        inspectionText += `(Components need semantic annotations to appear here)\n`;
      }
  
      return {
        content: [
          {
            type: 'text',
            text: inspectionText,
          },
        ],
      };
    } catch (error) {
      return {
        content: [
          {
            type: 'text',
            text: `Error inspecting viewport: ${error instanceof Error ? error.message : 'Unknown error'}`,
          },
        ],
        isError: true,
      };
    }

• src/tools.ts:113-120 (schema)

  Tool schema definition in getToolDefinitions(), including name, description, and empty inputSchema (no parameters required).

  {
    name: 'inspect_viewport',
    description: 'UI ANALYSIS: Get a detailed text-based description of what\'s currently displayed in the Scenic application. Perfect for understanding UI structure, finding clickable elements, and programmatic interface analysis. Use when you need to understand what\'s on screen without taking a screenshot.',
    inputSchema: {
      type: 'object',
      properties: {},
    },
  },

• src/tools.ts:200-201 (registration)

  Registration in the handleToolCall switch statement that routes calls to the inspect_viewport handler function.

  case 'inspect_viewport':
    return await handleInspectViewport(args);