web perception
webear
AIに本物の感覚を — あらゆるWebアプリを聞き、見て、感じ取らせましょう。
AIコーディングアシスタントに、実行中のWebアプリケーションへの直接的な感覚アクセスを提供するMCPサーバー + ブラウザSDKです。オーディオ、ビジュアル、パフォーマンス、ネットワーク、セキュリティ、コンソール — ブラウザからキャプチャし、リアルタイムで分析し、MCP経由で配信します。
「ビートが濁って聞こえる」 → あなたのAIが3秒をキャプチャし、スペクトル重心が580Hz、250Hz以下のエネルギーが45%であることを測定し、その正確な理由を教えてくれます。

機能
ツール | 説明 |
| 現在Webアプリが出力している内容の短いクリップ(500ms〜30秒)を録音します |
| 信号分析:RMS、ピークdB、クリッピング、スペクトル重心、周波数帯域、BPM、タイミングジッター |
| 平易な英語でのAI説明 — 「キックがブーミーで、80Hz付近に重いサブベースの蓄積がある」 |
| 2つのキャプチャを比較し、変化した点(ラウドネス、トーン、タイミング、クリッピング)を指摘します |
Related MCP server: broca-machina
仕組み
Browser (Web Audio API)
↓ MediaRecorder taps the AudioContext output node
↓ Uploads WebM blob via HTTP POST
Express Middleware (your dev server)
↓ Stores captures in memory, dispatches commands via SSE
MCP Server (stdio — runs inside your IDE)
↓ Retrieves captures, sends to CodedSwitch analysis API
AI Coding Assistant
→ "Your bass band is 42% of the mix (high), spectral centroid
is 580 Hz (muddy), and timing jitter is 23ms — the scheduler
is drifting under load."他のオーディオMCPとの決定的な違い:Web Audioグラフに直接アクセスし、部屋の音響、マイクハードウェア、ファイルのエクスポートの必要性をすべて回避します。
クイックスタート
1. インストール
npm install webear2. Expressミドルウェアを開発サーバーに追加
import express from 'express'
import { webearMiddleware } from 'webear/middleware'
const app = express()
app.use(express.json())
// Mount the audio debug bridge (automatically disabled in production)
app.use('/api/webear', webearMiddleware())
app.listen(5000)3. クライアントスニペットをWebアプリに追加
オプションA — すべてを自動検出(Tone.jsまたは生のWeb Audio)
import WebEar from 'webear/client'
WebEar.init()オプションB — 明示的なAudioContext
const ctx = new AudioContext()
const masterGain = ctx.createGain()
masterGain.connect(ctx.destination)
WebEar.init({ audioContext: ctx, outputNode: masterGain })オプションC — Tone.jsプロジェクト
import * as Tone from 'tone'
WebEar.init({ toneJs: true })オプションD — Three.js WebGLゲーム
import * as THREE from 'three'
const listener = new THREE.AudioListener()
camera.add(listener)
WebEar.init({ tapNode: listener.getInput() })オプションE — プレーンなscriptタグ
<script src="node_modules/webear/client-snippet.js"></script>
<script>WebEar.init()</script>4. IDEを設定
Claude Code(プロジェクトルートの.mcp.json):
{
"mcpServers": {
"webear": {
"command": "npx",
"args": ["webear"],
"env": {
"WEBEAR_BASE_URL": "http://localhost:5000",
"CODEDSWITCH_API_KEY": "your-key-here"
}
}
}
}Cursor(.cursor/mcp.json):
{
"mcpServers": {
"webear": {
"command": "npx",
"args": ["webear"],
"env": {
"WEBEAR_BASE_URL": "http://localhost:5000",
"CODEDSWITCH_API_KEY": "your-key-here"
}
}
}
}Windsurf(mcp_config.json):
{
"webear": {
"command": "npx",
"args": ["webear"],
"disabled": false,
"env": {
"WEBEAR_BASE_URL": "http://localhost:5000",
"CODEDSWITCH_API_KEY": "your-key-here"
}
}
}5. APIキーを取得 — 任意であり、開始には不要
analyze_audioはキーもアカウントも不要で動作します。 ffmpegがPATHに存在する場合、キャプチャをお使いのマシン上でデコードして分析し、基本的なレポート(再生時間、ラウドネス、ピークレベル、オーディオがクリッピングしているかどうか)を返します。何もアップロードされません。登録する前に、このツールを試してみてください。
キーを取得すると、算術演算以上の処理が必要な機能が解放されます:
キーなし | キーあり | |
| ✓ | ✓ |
| 基本 — 再生時間、ラウドネス、ピーク、クリッピング(ローカル) | フル — スペクトル重心、帯域エネルギー、クレストファクター、BPM、タイミングジッター |
| — | ✓ |
| — | ✓ |
| — | ✓ |
取得方法:
codedswitch.com で無料アカウントを作成します。
codedswitch.com/developer に移動します(アカウントメニューの Developer API からもアクセスできます)。
Generate API Key をクリック — その値が
CODEDSWITCH_API_KEYです。キーはwbr_で始まります。
無料枠:1日50回の分析。クレジットカードは不要です。
6. 開発サーバーを起動し、アプリを開き、オーディオを再生してから、AIに質問します:
「3秒キャプチャして、ベースが濁って聞こえる理由を教えて」
「最後のコミット前後のオーディオを比較して」
「高周波数帯域にクリッピングはありますか?」
出力例
analyze_audio
── Audio Analysis Report ──────────────────────────────
Duration: 3.02s
── Loudness ─────────────────────────────────────────
RMS: -12.4 dBFS
Peak: -1.2 dBFS
Dynamic range: 11.2 dB
Crest factor: 3.63
Clipping: none
── Tone ──────────────────────────────────────────────
Spectral centroid: 2847 Hz
DC offset: 0.00012 (ok)
── Frequency Bands ───────────────────────────────────
Sub (20-80 Hz): 8.2%
Bass (80-250 Hz): 22.1%
Mid (250-2k Hz): 38.4%
Hi-mid (2-6k Hz): 21.8%
High (6k+ Hz): 9.5%
── Rhythm ────────────────────────────────────────────
Estimated BPM: 92
Onset count: 12
Timing jitter: 4.2 ms std dev
── Summary ───────────────────────────────────────────
Loudness: -12.4 dBFS RMS, peak -1.2 dBFS. Tone: balanced (centroid 2847 Hz).
Band mix — sub: 8% | bass: 22% | mid: 38% | hi-mid: 22% | high: 10%.
Rhythm: estimated 92 BPM, 12 onsets detected. Timing: very tight (< 5 ms jitter).diff_audio
── Audio Diff: a1b2c3d4… → e5f6g7h8… ──
── Loudness ──────────────────────────────────────────
RMS: -14.2 dBFS → -12.4 dBFS (+1.8 dBFS)
⚠ Peak: -3.1 dBFS → -0.2 dBFS (+2.9 dBFS)
⚠ CLIPPING INTRODUCED — gain staging regression
── Tone ──────────────────────────────────────────────
⚠ Spectral centroid: 2847.0 Hz → 1920.0 Hz (-927.0 Hz)
── Interpretation ────────────────────────────────────
A gain bug was introduced that causes clipping.
Tonal character changed noticeably — EQ or filter behaviour may have shifted.設定
環境変数
変数 | デフォルト | 説明 |
|
| 開発サーバーのURL(ミドルウェアがマウントされている場所) |
| — | codedswitch.com のAPIキー — |
|
| 分析APIのベースを上書き(上級者向け/セルフホスト) |
ミドルウェアオプション
webearMiddleware({
maxCaptures: 50, // Max captures in memory (default: 50)
maxAgeMins: 10, // Auto-evict after N minutes (default: 10)
maxUploadBytes: 50e6, // Max upload size (default: 50MB)
devOnly: true, // Disable in production (default: true)
})クライアントオプション
WebEar.init({
audioContext: myCtx, // Your AudioContext instance
outputNode: myGainNode, // The node to tap (defaults to destination)
toneJs: true, // Auto-detect Tone.js context
bridgeBase: '/api/webear', // Override API path
devOnly: true, // Only init outside of production (default: true)
})要件
Node.js >= 18
MediaRecorderをサポートするブラウザ(Chrome、Firefox、Edge、Safari 14以降)分析用の
CODEDSWITCH_API_KEY(codedswitch.com で無料)
対象ユーザー
Web Audio / Tone.js 開発者 — IDEから離れずにビート、シンセ、エフェクト、ミキシングをデバッグ
ゲームオーディオ開発者 — サウンドエフェクト、空間オーディオ、ミキシングをリアルタイムで検証
音楽アプリ開発者 —
diff_audioでコード変更間のリグレッションを検出ポッドキャスト / ストリーミングアプリ — オーディオ品質、レベル、エンコーディングを検証
音を出すアプリを持つすべての人 — Web Audioグラフがあれば、あなたのAIは今それを聞くことができます
マイクを使わない理由
マイクMCPは部屋の音をキャプチャします — ファンのノイズ、椅子のきしみ、部屋の残響がすべて録音に含まれます。webear はWeb Audio APIをDACに到達する前にタップするため、部屋のアーティファクトのないクリーンなデジタル信号が得られます。
Web Perception — フルセンサースイート
WebEarは当初オーディオのみでした。Web Perception はそれを6つの感覚に拡張します:
センサー | 知覚するもの |
WebEar | オーディオ — ミックス品質、リズム、楽器、クリッピング |
WebEye | ビジュアル — キャンバス、UIレイアウト、アニメーション、スクリーンショット |
WebSense | パフォーマンス — フレームレート、メモリ、オーディオレイテンシー |
WebNerve | ネットワーク — APIレイテンシー、接続品質、ストレージ |
WebShield | セキュリティ — クッキー、ストレージ露出、CSP、フレーミング |
WebLog | コンソール — ログ、警告、エラー、未捕捉の例外 |
フルブラウザSDKをインストール
import { WebPerception } from 'webear/perception'
WebPerception.init({
apiKey: 'wbr_YOUR_API_KEY',
relayUrl: 'https://www.codedswitch.com',
sensors: ['ear', 'eye', 'sense', 'nerve', 'shield', 'log'],
})または単一のセンサーを使用:
import { WebEar } from 'webear/perception'
WebEar.init({
apiKey: 'wbr_YOUR_API_KEY',
ear: { audioContext: myCtx, audioNode: masterGain },
})MCP経由で接続(ホステッドリレー — ローカルサーバー不要)
{
"mcpServers": {
"webear": {
"url": "https://www.codedswitch.com/api/webear/mcp/sse",
"headers": {
"Authorization": "Bearer wbr_YOUR_API_KEY"
}
}
}
}利用可能なMCPツール
センサー | ツール | クレジット | 説明 |
Ear |
| 無料 | ライブタブオーディオを録音 |
Ear |
| 1 | BPM、ラウドネス、周波数帯域、クリッピング、ダイナミックレンジ |
Ear |
| 2 | AIによる平易な英語での説明 — 楽器、ジャンル、雰囲気、ミックスノート |
Ear |
| 1 | 2つのキャプチャを比較 — ラウドネス、トーン、タイミングの差分 |
Ear |
| 2 | グリッドアライメント、スイングファクター、一貫性(0〜100%) |
Ear |
| 1 | キャプチャ+分析を1回の呼び出しで |
Ear |
| 3 | 構造化されたミキシングフィードバック |
Eye |
| 無料 | タブからキャンバス/ビデオを録画 |
Eye |
| 2 | AIによるビジュアル説明 — レイアウト、色、バグ |
Eye |
| 2 | 2つのビジュアルキャプチャを比較 |
Sense |
| 無料 | FPS、メモリ、レイアウトシフト、オーディオレイテンシー |
Sense |
| 1 | フレームドロップ、メモリプレッシャー、オーディオアンダーラン |
Nerve |
| 無料 | APIタイミング、接続品質、ストレージサイズ |
Nerve |
| 1 | 遅いAPI、接続品質、ストレージ肥大 |
Shield |
| 無料 | クッキー、CSP、ストレージ露出、フレーミング |
Shield |
| 1 | CORS問題、HttpOnlyでないクッキー、CSP欠落 |
Log |
| 無料 | コンソール出力+未捕捉の例外 |
Log |
| 1 | エラーパターン、スタックトレース、繰り返される警告 |
APIキーを取得
codedswitch.com で無料アカウントを作成します。
codedswitch.com/developer を開きます — アカウントメニューの Developer API からもリンクされています。
Generate API Key をクリックしてコピーします。キーは
wbr_で始まります。
無料枠:1日50回の分析、クレジットカード不要。
変更履歴
2.0.1
APIキーの取得手順を修正しました。 以前の説明(「設定 → WebEar」)は誤りでした — 設定の下にWebEarセクションはありません。キーは codedswitch.com/developer にあります(アカウントメニューの Developer API からリンクされています)。クイックスタートとWeb Perceptionの両セクションが正しい場所を指すようになりました。
SDKの「APIキーがありません」というコンソールエラーが、キーページに直接リンクするようになりました。
コントリビューション
CONTRIBUTING.md を参照してください。
ライセンス
MIT — LICENSE を参照
作者
@asume21 によって構築 — CodedSwitch
Available Tools
4 toolsanalyze_audioA
Run signal analysis on a captured audio clip. Returns RMS, peak dB, clipping, spectral centroid, frequency band energy, estimated BPM, and timing jitter.
| Name | Required | Description | Default |
|---|---|---|---|
| capture_id | Yes | The capture ID returned by capture_audio |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully carries the burden of behavioral disclosure. It lists the analysis outputs and implies a non-destructive, read-only operation, which is appropriate for a signal analysis tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise at one sentence, front-loaded with the action ('Run signal analysis'), and efficiently enumerates outputs without extraneous content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given only one parameter and no output schema, the description adequately lists the return values, providing enough context for an agent to understand what the tool produces.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter (capture_id), which already has a description. The tool description adds that the parameter should come from capture_audio, but this is marginal value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies the verb ('Run signal analysis'), resource ('captured audio clip'), and lists specific metrics returned (RMS, peak dB, clipping, etc.), distinguishing it from sibling tools like capture_audio or describe_audio.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool requires a prior captured audio clip (via capture_id), but does not explicitly state when to prefer this tool over alternatives like describe_audio or diff_audio, leaving room for ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
capture_audioA
Record a short clip of what the running web app is currently outputting. Returns a capture ID you can pass to analyze_audio or describe_audio.
| Name | Required | Description | Default |
|---|---|---|---|
| duration_ms | No | How many milliseconds to record (default 3000, max 30000) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It states 'short clip' and mentions the duration parameter, but does not disclose what audio source is captured (e.g., system output, microphone), potential failure modes, or any destructive effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the verb and resource, no redundant information. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one optional parameter and no output schema, the description fully covers the purpose, return value, and relationship to sibling tools. Nothing critical is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (duration_ms fully documented). The main description provides no additional parameter semantics beyond what the schema already states (default 3000, max 30000). Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool records a short clip of the running web app's audio output and returns a capture ID. Distinguishes itself from sibling tools (analyze_audio, describe_audio, diff_audio) by specifying that the ID can be passed to them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says to use the capture ID with analyze_audio or describe_audio, implying a workflow. However, it does not provide explicit when-not-to-use scenarios or alternatives beyond the mentioned siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
describe_audioA
Send a captured audio clip to Gemini or GPT-4o to get a plain-English description of what it sounds like — useful when something sounds wrong but you cannot describe it.
| Name | Required | Description | Default |
|---|---|---|---|
| capture_id | Yes | The capture ID returned by capture_audio to describe |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It mentions the AI models but omits side effects like cost, latency, or external dependencies. This is insufficient for a tool that sends audio to external APIs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that conveys purpose, usage context, and outcome without any wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, no output schema), the description covers the essential information. Minor gaps exist regarding output format or latency, but overall it is complete enough for the complexity level.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already describes the single parameter with 100% coverage. The description adds minimal value beyond restating the purpose, but no further semantic information is needed given the schema's completeness.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('send'), resource ('captured audio clip'), and outcome ('plain-English description'). It differentiates from sibling tools like analyze_audio, capture_audio, and diff_audio by focusing on description generation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a usage context ('useful when something sounds wrong but you cannot describe it') but does not explicitly mention when not to use it or compare to alternatives like analyze_audio.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
diff_audioA
Compare two audio captures and flag what changed — loudness, tone, timing, clipping. Use this before and after a code change to verify the audio impact.
| Name | Required | Description | Default |
|---|---|---|---|
| capture_id_a | Yes | First capture ID (the "before") | |
| capture_id_b | Yes | Second capture ID (the "after") |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states the tool 'flags what changed,' implying a read-only operation, but does not explicitly disclose whether it is safe, destructive, or requires permissions. Given the simple nature of a diff tool, the lack of explicit behavioral disclosure is a minor gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: the first clearly states the purpose and what is flagged, the second gives a concise use case. No unnecessary words. Well-structured and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the purpose and usage but lacks information about return values or potential errors. Given the tool's simplicity and the absence of an output schema, a hint at the output format would improve completeness. Sibling tools are explained in their own descriptions, so differentiation is adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with each parameter described as the 'before' and 'after' capture ID. The tool description adds context about comparing captures and the aspects examined, but does not significantly enhance the semantic meaning beyond the schema. With high schema coverage, baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool compares two audio captures and lists specific aspects (loudness, tone, timing, clipping). It distinguishes itself from siblings (capture_audio, analyze_audio, describe_audio) by focusing on comparative analysis and providing a concrete use case ('before and after a code change').
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use this tool: 'before and after a code change to verify the audio impact.' It provides clear context but does not explicitly state when not to use it or mention alternative tools, though the use case implies a comparison scenario.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
4 tool updates
v2.0.0- First observed
analyze_audio - First observed
capture_audio - First observed
describe_audio - First observed
diff_audio
TDQS
Scored across 4 tools
Each tool has a clearly distinct purpose: capture records audio, analyze does signal analysis, describe provides a language description, and diff compares two captures. No overlap.
All tool names follow the consistent verb_noun pattern (capture_audio, analyze_audio, describe_audio, diff_audio), making them predictable and easy to understand.
Four tools cover the essential operations for audio perception without being too few or too many, perfectly scoped for the server's purpose.
The tool set covers capture, analysis, description, and comparison, providing a complete workflow for assessing audio output. No obvious missing operations.
Maintenance
Related MCP Connectors
- PrimetaOAuthai.primeta
Give your AI a face, a voice, and a personality. 3D avatars with custom personas.
Images, video, music, speech, and public social data for AI agents.
Audio for your agent: transcribe, speak, translate, summarise, plus sound effects and music.
Lets AI agents use a real human as a tool: visual checks, taste, phone calls, unblocking, approvals
Related MCP Servers
- AlicenseAqualityCmaintenanceGive your AI assistant eyes and ears — analyze any video, audio, or image, entirely on your machine.253 npm2Apache 2.0
- AlicenseNot gradedqualityAmaintenanceGives any text-based AI a voice and ears inside a Discord voice channel by transcribing speech, relaying to an LLM/agent, and speaking replies back.MIT
- AlicenseNot gradedqualityDmaintenanceEnables AI systems to control the Reachy Mini robot—speak, listen, see, and express emotions through physical movement. Compatible with Claude, GPT, Grok, and other MCP-compatible AIs.MIT
- AlicenseAqualityDmaintenanceProvides AI agents with real-time physical perception data including object detections, positions, sounds, and environmental readings via MCP tools.531 npmMIT