page.analyze
Analyze web pages to detect layout structures, extract motion patterns, and evaluate design quality in a unified parallel process.
Instructions
Analyze a web page URL with layout detection, motion pattern extraction, and quality evaluation. Executes layout.ingest, motion.detect, and quality.evaluate in parallel and returns unified results.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Target URL to analyze (required) | |
| sourceType | No | Source type: award_gallery or user_provided (default) | user_provided |
| usageScope | No | Usage scope: inspiration_only (default) or owned_asset | inspiration_only |
| features | No | Feature flags for analysis (default: all true) | |
| layoutOptions | No | Layout analysis options | |
| motionOptions | No | Motion detection options | |
| qualityOptions | No | Quality evaluation options | |
| summary | No | Return summary response (default: true). Set to false for full details. | |
| timeout | No | Overall timeout in ms (default: 60000) | |
| waitUntil | No | Page load completion criteria (default: load) | load |
| auto_timeout | No | Enable Pre-flight Probe for dynamic timeout calculation (v0.1.0). Analyzes page complexity (WebGL, SPA, heavy frameworks) before analysis and calculates optimal timeout. Results are included in preflightProbe response field. | |
| responsiveOptions | No | Responsive layout analysis options. Captures layouts at multiple viewport sizes (desktop/tablet/mobile) and detects differences in typography, spacing, navigation, and layout structure. |
Implementation Reference
- Main MCP tool handler for 'page.analyze'. Validates input, handles SSRF checks, manages job queue for async analysis (with Vision), and triggers sync processing with a hard timeout wrapper.
export async function pageAnalyzeHandler( input: unknown, progressContext?: ProgressContext ): Promise<PageAnalyzeOutput> { const overallStartTime = Date.now(); if (isDevelopment()) { logger.info("[MCP Tool] page.analyze called", { hasInput: input !== null && input !== undefined, }); } // 入力バリデーション let validated: PageAnalyzeInput; try { if (input === null || input === undefined) { return { success: false, error: { code: PAGE_ANALYZE_ERROR_CODES.VALIDATION_ERROR, message: "Input is required", }, }; } validated = pageAnalyzeInputSchema.parse(input); } catch (error) { if (isDevelopment()) { logger.error("[MCP Tool] page.analyze validation error", { error }); } return { success: false, error: { code: PAGE_ANALYZE_ERROR_CODES.VALIDATION_ERROR, message: error instanceof Error ? error.message : "Invalid input", }, }; } // SSRF対策: URL検証 const urlValidation = validateExternalUrl(validated.url); if (!urlValidation.valid) { if (isDevelopment()) { logger.warn("[MCP Tool] page.analyze SSRF blocked", { url: validated.url, error: urlValidation.error, }); } return { success: false, error: { code: PAGE_ANALYZE_ERROR_CODES.SSRF_BLOCKED, message: urlValidation.error ?? "URL is blocked for security reasons", }, }; } const normalizedUrl = urlValidation.normalizedUrl ?? normalizeUrlForValidation(validated.url); // robots.txt チェック（RFC 9309準拠）- 早期ブロック const robotsResult = await isUrlAllowedByRobotsTxt(validated.url, validated.respect_robots_txt); if (!robotsResult.allowed) { return { success: false, error: { code: PAGE_ANALYZE_ERROR_CODES.ROBOTS_TXT_BLOCKED, message: `Blocked by robots.txt: ${validated.url} (domain: ${robotsResult.domain}, reason: ${robotsResult.reason}). ` + `Use respect_robots_txt: false to override. ` + `Note: Overriding robots.txt may have legal implications depending on jurisdiction (e.g., EU DSM Directive Article 4).`, }, }; } // ===================================================== // Smart Defaults: Vision有効時の自動非同期モード（v0.1.0） // ===================================================== // Vision LLM (llama3.2-vision) はCPUモードで2-5分以上かかるため、 // MCPの600秒ハードタイムアウトを回避するために自動的にasyncモードを有効化 const useVisionEnabled = validated.layoutOptions?.useVision !== false; // デフォルトtrue const useNarrativeVisionEnabled = validated.narrativeOptions?.includeVision === true; const visionRequested = useVisionEnabled || useNarrativeVisionEnabled; // async が明示的に指定されていない場合のみ自動設定 // (ユーザーが async: false を明示指定した場合は尊重) let autoAsyncEnabled = false; if (visionRequested && validated.async === undefined) { const redisCheck = await isRedisAvailable(); if (redisCheck) { // Vision有効 + Redis利用可能 → 自動でasyncモードを有効化 validated = { ...validated, async: true }; autoAsyncEnabled = true; if (isDevelopment()) { logger.info("[page.analyze] Auto-async enabled for Vision analysis", { url: validated.url, useVision: useVisionEnabled, useNarrativeVision: useNarrativeVisionEnabled, }); } } else if (isDevelopment()) { logger.warn("[page.analyze] Vision requested but Redis unavailable, sync mode will be used", { url: validated.url, }); } } // ===================================================== // 非同期モード処理（Phase3-2） // ===================================================== // async=true の場合、ジョブをキューに投入して即座に返す if (validated.async === true) { if (isDevelopment()) { logger.info("[page.analyze] Async mode requested", { url: validated.url }); } // Redis可用性チェック const redisAvailable = await isRedisAvailable(); if (!redisAvailable) { if (isDevelopment()) { logger.warn("[page.analyze] Redis unavailable for async mode"); } return { success: false, error: { code: "REDIS_UNAVAILABLE", message: "Async mode requires Redis. Please start Redis or use sync mode (async=false).", }, }; } // ワーカープロセスが起動していなければ起動する getWorkerSupervisor().ensureWorkerRunning(); // ジョブIDとしてwebPageIdを事前生成 const webPageId = uuidv7(); // キューにジョブを追加 const queue = createPageAnalyzeQueue(); // バッチ投入前: orphaned/failed/stalledジョブを自動クリーンアップ const cleanupResult = await cleanupQueue(createQueueAdapter(queue)); if (cleanupResult.strategy !== "skipped" && isDevelopment()) { logger.info("[page.analyze] Queue cleanup before job submission", { strategy: cleanupResult.strategy, totalCleaned: cleanupResult.totalCleaned, }); } try { // ジョブオプションを構築（exactOptionalPropertyTypes対応） const jobOptions: PageAnalyzeJobOptions = { timeout: validated.timeout, features: { layout: validated.features?.layout, motion: validated.features?.motion, quality: validated.features?.quality, }, }; // layoutOptions（デフォルト値を常に設定 — undefinedの場合もデフォルトで構築） // Bug fix: デフォルトモード（layoutOptions未指定）でも useVision: true 等が適用されるように { const src = validated.layoutOptions; const layoutOpts: NonNullable<PageAnalyzeJobOptions["layoutOptions"]> = { useVision: src?.useVision ?? true, saveToDb: src?.saveToDb ?? true, autoAnalyze: src?.autoAnalyze ?? true, fullPage: src?.fullPage ?? true, scrollVision: src?.scrollVision ?? true, scrollVisionMaxCaptures: src?.scrollVisionMaxCaptures ?? 10, }; if (src?.viewport) { layoutOpts.viewport = src.viewport; } jobOptions.layoutOptions = layoutOpts; } // motionOptions if (validated.motionOptions) { jobOptions.motionOptions = { detectJsAnimations: validated.motionOptions.detect_js_animations ?? true, detectWebglAnimations: validated.motionOptions.detect_webgl_animations ?? true, enableFrameCapture: validated.motionOptions.enable_frame_capture ?? false, analyzeFrames: validated.motionOptions.analyze_frames ?? false, saveToDb: validated.motionOptions.saveToDb ?? true, maxPatterns: validated.motionOptions.maxPatterns ?? 500, // v0.1.0: Motion検出タイムアウト（asyncモードでは長時間検出可能） timeout: validated.motionOptions.timeout ?? 300000, }; } // qualityOptions（undefinedを明示的に除外） if (validated.qualityOptions) { const qualityOpts: NonNullable<PageAnalyzeJobOptions["qualityOptions"]> = { strict: validated.qualityOptions.strict ?? true, }; if (validated.qualityOptions.weights) { qualityOpts.weights = { originality: validated.qualityOptions.weights.originality ?? 0.35, craftsmanship: validated.qualityOptions.weights.craftsmanship ?? 0.4, contextuality: validated.qualityOptions.weights.contextuality ?? 0.25, }; } if (validated.qualityOptions.targetIndustry) { qualityOpts.targetIndustry = validated.qualityOptions.targetIndustry; } if (validated.qualityOptions.targetAudience) { qualityOpts.targetAudience = validated.qualityOptions.targetAudience; } jobOptions.qualityOptions = qualityOpts; } // narrativeOptions（デフォルト有効） if (validated.narrativeOptions) { jobOptions.narrativeOptions = { enabled: validated.narrativeOptions.enabled ?? true, saveToDb: validated.narrativeOptions.saveToDb ?? true, includeVision: validated.narrativeOptions.includeVision ?? true, visionTimeoutMs: validated.narrativeOptions.visionTimeoutMs ?? 300000, generateEmbedding: validated.narrativeOptions.generateEmbedding ?? true, }; } // responsiveOptions（デフォルト有効） if (validated.responsiveOptions) { const rOpts = validated.responsiveOptions; jobOptions.responsiveOptions = { enabled: rOpts.enabled ?? true, ...(rOpts.viewports !== undefined ? { viewports: rOpts.viewports } : {}), ...(rOpts.include_screenshots !== undefined ? { include_screenshots: rOpts.include_screenshots } : {}), ...(rOpts.include_diff_images !== undefined ? { include_diff_images: rOpts.include_diff_images } : {}), ...(rOpts.diff_threshold !== undefined ? { diff_threshold: rOpts.diff_threshold } : {}), ...(rOpts.save_to_db !== undefined ? { save_to_db: rOpts.save_to_db } : {}), ...(rOpts.detect_navigation !== undefined ? { detect_navigation: rOpts.detect_navigation } : {}), ...(rOpts.detect_visibility !== undefined ? { detect_visibility: rOpts.detect_visibility } : {}), ...(rOpts.detect_layout !== undefined ? { detect_layout: rOpts.detect_layout } : {}), }; } // respectRobotsTxt（Workerパスでもrobots.txtチェックに渡す） if (validated.respect_robots_txt !== undefined) { jobOptions.respectRobotsTxt = validated.respect_robots_txt; } const job = await addPageAnalyzeJob(queue, { webPageId, url: validated.url, options: jobOptions, }); if (isDevelopment()) { logger.info("[page.analyze] Job queued successfully", { jobId: job.id, webPageId, url: validated.url, }); } // 非同期レスポンスを返す const autoAsyncNote = autoAsyncEnabled ? " (Auto-enabled: Vision analysis requires async mode to avoid MCP timeout)" : ""; const asyncResponse: PageAnalyzeAsyncOutput = { async: true, jobId: webPageId, status: "queued", message: `Job queued successfully.${autoAsyncNote} Use page.getJobStatus(job_id="${webPageId}") to check progress.`, polling: { intervalSeconds: 10, // Vision処理は長時間かかるため10秒間隔を推奨 retentionHours: 24, howToCheck: `Call page.getJobStatus with job_id="${webPageId}" to check job status and retrieve results.`, }, }; return asyncResponse as unknown as PageAnalyzeOutput; } finally { await closeQueue(queue); } } // ===================================================== // MCP 570秒ハードタイムアウトガード（v0.1.0） // ===================================================== // MCP プロトコルの600秒タイムアウトを超えないよう、570秒（30秒安全マージン）で // sync mode全体をハードタイムアウトで保護する。 // CPU Vision延長やフェーズ個別タイムアウトが膨らんでも、このガードで確実に打ち切る。 // fetchExternalCss: true で多数の外部リソースを取得する際のハング防止が主目的。 const OVERALL_HARD_TIMEOUT_MS = 570000; // 570秒 = MCP 600秒 - 30秒安全マージン // タイマーIDを保持してクリーンアップ可能にする let hardTimeoutId: ReturnType<typeof setTimeout> | undefined; const syncProcessingResult = await Promise.race([ executeSyncProcessing( validated, normalizedUrl, overallStartTime, { getService: () => serviceFactory?.() ?? {}, getPrismaClient, }, progressContext ), new Promise<PageAnalyzeOutput>((_, reject) => { hardTimeoutId = setTimeout(() => { reject(new PhaseTimeoutError("page.analyze-overall", OVERALL_HARD_TIMEOUT_MS)); }, OVERALL_HARD_TIMEOUT_MS); }), ]) .catch((error): PageAnalyzeOutput => { const isTimeout = error instanceof PhaseTimeoutError; const elapsedMs = Date.now() - overallStartTime; if (isDevelopment()) { logger.error("[page.analyze] Overall hard timeout triggered", { timeoutMs: OVERALL_HARD_TIMEOUT_MS, elapsedMs, isTimeout, error: error instanceof Error ? error.message : String(error), }); } return { success: false, error: { code: PAGE_ANALYZE_ERROR_CODES.TIMEOUT_ERROR, message: `page.analyze exceeded MCP hard timeout limit (${Math.round(elapsedMs / 1000)}s / ${OVERALL_HARD_TIMEOUT_MS / 1000}s). Consider using fetchExternalCss: false or reducing analysis scope.`, }, }; }) .finally(() => { // Promise.race で executeSyncProcessing が先に完了した場合、タイマーをクリーンアップ if (hardTimeoutId) { clearTimeout(hardTimeoutId); } }); return syncProcessingResult; } - Tool definition including name, description, and input schema for 'page.analyze'.
export const pageAnalyzeToolDefinition = { name: "page.analyze", description: "Analyze a web page URL with layout detection, motion pattern extraction, and quality evaluation. Executes layout.ingest, motion.detect, and quality.evaluate in parallel and returns unified results.", annotations: { title: "Page Analyze", readOnlyHint: true, idempotentHint: true, openWorldHint: true, }, inputSchema: { type: "object" as const, required: ["url"], properties: { url: { type: "string", format: "uri", description: "Target URL to analyze (required)", }, sourceType: { type: "string", enum: ["award_gallery", "user_provided"], default: "user_provided", description: "Source type: award_gallery or user_provided (default)", }, usageScope: { type: "string", enum: ["inspiration_only", "owned_asset"], default: "inspiration_only", description: "Usage scope: inspiration_only (default) or owned_asset", }, features: { type: "object", description: "Feature flags for analysis (default: all true)", properties: { layout: { type: "boolean", default: true, description: "Enable layout analysis (default: true)", }, motion: { type: "boolean", default: true, description: "Enable motion detection (default: true)", }, quality: { type: "boolean", default: true, description: "Enable quality evaluation (default: true)", }, }, }, layoutOptions: { type: "object", description: "Layout analysis options", properties: { fullPage: { type: "boolean", default: true, description: "Full page screenshot (default: true)", }, viewport: { type: "object", properties: { width: { type: "number", minimum: 320, maximum: 4096, default: 1440 }, height: { type: "number", minimum: 240, maximum: 16384, default: 900 }, }, }, // MCP-RESP-03: snake_case正式形式（新規オプション推奨形式） include_html: { type: "boolean", default: false, description: "Include HTML in response (default: false) - snake_case正式形式", }, include_screenshot: { type: "boolean", default: false, description: "Include screenshot in response (default: false) - snake_case正式形式", }, // レガシー互換: camelCaseは後方互換として維持 includeHtml: { type: "boolean", default: false, description: "Include HTML in response (default: false) - レガシー互換、include_html推奨", }, includeScreenshot: { type: "boolean", default: false, description: "Include screenshot in response (default: false) - レガシー互換、include_screenshot推奨", }, saveToDb: { type: "boolean", default: true, description: "Save to database (default: true)", }, autoAnalyze: { type: "boolean", default: true, description: "Auto analyze sections and generate embeddings (default: true)", }, fetchExternalCss: { type: "boolean", default: true, description: "Fetch external CSS files for layout analysis (default: true)", }, useVision: { type: "boolean", default: true, description: "Use Vision API (Ollama + llama3.2-vision) to analyze screenshot for section detection. Delegates to layout.inspect screenshot mode. (default: true)", }, }, }, motionOptions: { type: "object", description: "Motion detection options", properties: { fetchExternalCss: { type: "boolean", default: false, description: "Fetch external CSS files (default: false)", }, minDuration: { type: "number", minimum: 0, default: 0, description: "Minimum animation duration in ms (default: 0)", }, maxPatterns: { type: "number", minimum: 1, maximum: 4000, default: 100, description: "Maximum patterns to detect (default: 100)", }, includeWarnings: { type: "boolean", default: true, description: "Include warnings in response (default: true)", }, saveToDb: { type: "boolean", default: true, description: "Save motion patterns to database (default: true)", }, // Video Mode Options (Phase 5) enable_frame_capture: { type: "boolean", default: true, description: "Enable frame capture for scroll animation analysis (default: true)", }, frame_capture_options: { type: "object", description: "Frame capture configuration", properties: { frame_rate: { type: "number", minimum: 1, maximum: 120, default: 30, description: "Frame rate (default: 30fps)", }, frame_interval_ms: { type: "number", minimum: 1, maximum: 1000, default: 33, description: "Frame interval in milliseconds (default: 33ms = 30fps)", }, scroll_speed_px_per_sec: { type: "number", minimum: 1, description: "Scroll speed in pixels per second (optional)", }, scroll_px_per_frame: { type: "number", minimum: 0.01, default: 15, description: "Scroll pixels per frame (default: 15px)", }, output_format: { type: "string", enum: ["png", "jpeg"], default: "png", description: "Output image format (default: png)", }, output_dir: { type: "string", default: "/tmp/reftrix-frames/", description: "Output directory for frames (default: /tmp/reftrix-frames/)", }, filename_pattern: { type: "string", default: "frame-{0000}.png", description: "Filename pattern with frame number placeholder (default: frame-{0000}.png)", }, page_height_px: { type: "number", minimum: 100, maximum: 100000, description: "Manual page height in pixels (optional, auto-detected if omitted)", }, scroll_duration_sec: { type: "number", minimum: 0.1, maximum: 300, description: "Scroll duration in seconds (optional)", }, }, }, analyze_frames: { type: "boolean", default: true, description: "Enable frame image analysis with pixelmatch (default: true)", }, frame_analysis_options: { type: "object", description: "Frame analysis configuration", properties: { frame_dir: { type: "string", description: "Frame image directory (optional, uses frame_capture_options.output_dir if omitted)", }, sample_interval: { type: "number", minimum: 1, maximum: 100, default: 1, description: "Analyze every Nth frame (default: 1 = all frames)", }, diff_threshold: { type: "number", minimum: 0, maximum: 1, default: 0.01, description: "Minimum diff percentage to consider as change (default: 0.01 = 1%)", }, cls_threshold: { type: "number", minimum: 0, maximum: 1, default: 0.1, description: "CLS (Cumulative Layout Shift) warning threshold (default: 0.1)", }, motion_threshold: { type: "number", minimum: 1, maximum: 500, default: 5, description: "Minimum pixels to detect motion vector (default: 5)", }, output_diff_images: { type: "boolean", default: false, description: "Save diff images to output_dir (default: false)", }, parallel: { type: "boolean", default: true, description: "Process frames in parallel (default: true)", }, }, }, // JS Animation Options (v0.1.0) detect_js_animations: { type: "boolean", default: false, description: "Enable JavaScript animation detection via CDP + Web Animations API + library detection (default: false, requires Playwright)", }, js_animation_options: { type: "object", description: "JS animation detection configuration", properties: { enableCDP: { type: "boolean", default: true, description: "Enable Chrome DevTools Protocol animation detection (default: true)", }, enableWebAnimations: { type: "boolean", default: true, description: "Enable Web Animations API detection (default: true)", }, enableLibraryDetection: { type: "boolean", default: true, description: "Enable library detection (GSAP, Framer Motion, anime.js, Three.js, Lottie) (default: true)", }, waitTime: { type: "number", minimum: 0, maximum: 10000, default: 1000, description: "Wait time in ms after page load before detecting animations (default: 1000)", }, }, }, // v0.1.0: Motion検出タイムアウト（asyncモードでは長時間検出可能） timeout: { type: "number", minimum: 30000, maximum: 600000, default: 180000, description: "Motion detection timeout in milliseconds. MCP Protocol has a 60-second tool call limit. In async mode (page.analyze with async=true), this limit does not apply, allowing longer detection times for heavy WebGL/Three.js sites. (default: 180000 = 3 minutes, max: 600000 = 10 minutes)", }, }, }, qualityOptions: { type: "object", description: "Quality evaluation options", properties: { weights: { type: "object", properties: { originality: { type: "number", minimum: 0, maximum: 1, default: 0.35 }, craftsmanship: { type: "number", minimum: 0, maximum: 1, default: 0.4 }, contextuality: { type: "number", minimum: 0, maximum: 1, default: 0.25 }, }, }, targetIndustry: { type: "string", maxLength: 100, description: "Target industry for contextual evaluation", }, targetAudience: { type: "string", maxLength: 100, description: "Target audience for contextual evaluation", }, strict: { type: "boolean", default: false, description: "Strict mode for AI cliche detection (default: false)", }, includeRecommendations: { type: "boolean", default: true, description: "Include recommendations in response (default: true)", }, }, }, summary: { type: "boolean", default: true, description: "Return summary response (default: true). Set to false for full details.", }, timeout: { type: "number", minimum: 5000, maximum: 300000, default: 60000, description: "Overall timeout in ms (default: 60000)", }, waitUntil: { type: "string", enum: ["load", "domcontentloaded", "networkidle"], default: "load", description: "Page load completion criteria (default: load)", }, auto_timeout: { type: "boolean", default: false, description: "Enable Pre-flight Probe for dynamic timeout calculation (v0.1.0). Analyzes page complexity (WebGL, SPA, heavy frameworks) before analysis and calculates optimal timeout. Results are included in preflightProbe response field.", }, responsiveOptions: { type: "object", description: "Responsive layout analysis options. Captures layouts at multiple viewport sizes (desktop/tablet/mobile) and detects differences in typography, spacing, navigation, and layout structure.", properties: { enabled: { type: "boolean", default: true, description: "Enable responsive analysis (default: true)", }, viewports: { type: "array", description: "Custom viewport configurations. Default: desktop (1920x1080), tablet (768x1024), mobile (375x667)", items: { type: "object", required: ["name", "width", "height"], properties: { name: { type: "string", description: "Viewport name (e.g., desktop, tablet, mobile)", }, width: { type: "number", minimum: 320, maximum: 4096, description: "Width in pixels", }, height: { type: "number", minimum: 240, maximum: 16384, description: "Height in pixels", }, }, }, }, include_screenshots: { type: "boolean", default: false, description: "Include screenshots for each viewport in response (default: false, DB-first workflow)", }, include_diff_images: { type: "boolean", default: false, description: "Include diff images in viewport comparison results (default: false)", }, diff_threshold: { type: "number", minimum: 0, maximum: 1, default: 0.1, description: "Pixel diff threshold for viewport comparison (0-1, default: 0.1)", }, save_to_db: { type: "boolean", default: true, description: "Save responsive analysis results to DB (default: true)", }, detect_navigation: { type: "boolean", default: true, description: "Detect navigation pattern changes (horizontal-menu to hamburger-menu, etc.) (default: true)", }, detect_visibility: { type: "boolean", default: true, description: "Detect element visibility changes between viewports (default: true)", }, detect_layout: { type: "boolean", default: true, description: "Detect layout structure changes (grid columns, flex direction, etc.) (default: true)", }, }, }, }, }, };