Eyewitness MCP Server
by birserg
README.md
# Eyewitness — paid photo inspections for AI agents, settled on NEAR
[](https://github.com/birserg/eyewitness/actions/workflows/ci.yml)
[](LICENSE)
[](https://testnet.nearblocks.io/address/eyewitness-live.testnet)
[](.mcp.json)
> **Scope, stated plainly:** Eyewitness delivers **human-submitted, hub-accepted observations** with money rules enforced on NEAR. The on-chain challenge code is a **replay deterrent**: an old photo is unlikely to contain a fresh 4-digit code (codes do repeat, 1 in 10,000), and a worker could still edit the code into an existing or generated image, so it is not proof that a scene is real or that the photo shows what was asked. What *is* checkable: which exact photo was paid for (sha256 on-chain), that the hub's AI read the exact code before payment, and that no one was paid without a delivery or kept a reward after expiry. Testnet only.
**AI agents can read the whole internet, but they can't look at a shelf, a street or a storefront right now.**
Eyewitness is a **paid human-in-the-loop primitive**: when an agent can't continue because it needs to *see* something in the physical world, it pauses, pays a person to look, checks the evidence, and resumes its task. Physical photo inspections are the first capability; the same escrow-and-receipt flow extends to human judgments and corrections (roadmap).
Eyewitness lets an AI agent commission a **paid inspection**: a real person with a phone photographs what the agent asks for, the agent looks at the result, and if something is still uncertain (an unreadable label, the wrong angle) it **commissions a precise follow-up from the same person**. Every step is escrowed and receipted on NEAR.
1. The agent posts a photo task (*"Photo of the shelf left of the door, include the code on paper"*) with a reward. The reward **plus the task's storage cost** is paid into the NEAR contract; the reward stays locked.
2. The contract generates a random **4-digit challenge code** from the block's random seed.
3. A worker opens the Eyewitness phone page (camera capture, no gallery picker), writes the code on paper, and takes the photo.
4. **AI check before payment:** a vision model running locally on the hub must read the **exact** code in the photo. It is *not* told the expected code. No code, no payment: the task stays open and the worker can retry.
5. The hub records the photo's **sha256** on-chain with `fulfill`, and the contract sends the worker's payment as a follow-up transfer a few seconds later (90% of the reward, 10% network fee). Only then does the hub release the photo.
6. The agent verifies the photo against the on-chain hash of the task it asked for, judges whether it shows what was asked, and can commission a follow-up with `worker = <same photographer>`.
7. Nobody delivered before the deadline? **Anyone** can trigger a refund and the agent gets the full reward back.
Built for **NEAR HACKATHON** (Nov 2026) — track **07 Agent Economy** ("agents that pay for APIs, compute, or data").
```
AI agent (Claude via MCP) NEAR contract Worker's phone (web page)
─────────────────────────── ───────────────────── ───────────────────────────
post_task(reward + storage) ─────► locks reward, makes code 4821
list_open_tasks ◄────── hub polls ─────► pushes task to phone
capture (+ code on paper)
hub AI must read the exact code (no code → retry, no pay)
fulfill(id, sha256) ◄─ hub ◄────────── photo bytes (kept private until paid)
photo + hash check ◄── hub pays worker 0.009, hub 0.001
"label unreadable" → post_task(worker = same person) → close-up → second receipt
(no delivery by deadline → refund() → agent gets the reward back)
```
## Status
| Part | Status |
|---|---|
| Contract: escrow, challenge codes, fulfill/refund, requester-paid storage, events | Done: live on testnet (`eyewitness-live.testnet`), 16 unit tests |
| Hub + AI code check (exact match) | Done: verified with a real Android phone: 2 photos without the code rejected, the photo with the code paid ([log](docs/verification.md)) |
| Worker phone page | Done: camera capture, native-camera fallback on Android, per-worker link credential, rejection and payment states |
| Agent side: MCP server + CLI | Done: results bound to the requested task; any task/hash mismatch is an error and no image is returned |
| Website with live on-chain stats | Done: `docs/` (GitHub Pages) |
| Trustless hub (TEE), camera-signed photos, payout recovery ledger | Not yet: roadmap |
| Mainnet | Not yet: not deployed |
### Fixed after an external strict review (Nov 14)
Exact code match (OCR `948210` no longer passes `4821`) · agent results bound to the requested task, hash/task mismatches are errors · per-worker phone credentials (HMAC of the hub secret) · photos private until the on-chain payment succeeds · requester pays task storage (minimum-reward tasks no longer drain the contract) · always-on "window mode" removed and force-disabled on returning phones · refunds reported only after on-chain confirmation · worker registration pays its own storage and posting pre-pays the bytes a delivery adds · paid photos stay servable if metadata persistence fails (rebuilt from the chain on restart) · claims corrected throughout. Checked by `scripts/regression.ts` and 16 contract tests.
## Why NEAR is necessary here
- **Strangers pay strangers, per photo, in cents.** A 0.01 NEAR reward to a person the agent has never met is not practical on card rails. The contract holds the money, so neither side has to trust the other.
- **The money rules are code** (`contract/src/lib.rs`): no payment without a hub-confirmed delivery, no delivery after the deadline, no double payment, refunds only for undelivered tasks, reserved tasks only for their worker, storage paid by the requester. (This testnet account still holds a full-access key, so its owner can upgrade the code; the rules hold as deployed.)
- **A fresh code from the chain:** the challenge comes from `env::random_seed()` at the block where the task is posted, so an old photo is unlikely to contain it (a replay deterrent, not authenticity proof; 4-digit codes repeat).
- **A public receipt for every accepted photo:** each delivered task stores the photo's sha256, the worker and the time on-chain (`EVENT_JSON` events), so anyone can audit which exact image was paid for and when.
- **Agents only need the contract id:** the hub publishes its URL on-chain (`set_hub_url`), the job board is the chain (`list_open_tasks`).
## What is proven and what is not (honest trust model)
| Claim | Status in this version |
|---|---|
| Agent can't get a photo without paying | Yes: reward is locked when the task is posted; the hub serves a photo only after its on-chain payment succeeded |
| Worker can't be paid without delivering | Yes: only `fulfill` (hub-signed, with the photo's sha256) pays; otherwise refund |
| Agent gets refunded if nobody delivers | Yes: `refund` after `expires_ms`, callable by anyone |
| The photo the agent got is the one paid for | Yes: sha256 recorded on-chain at delivery; the agent re-checks it against the task it requested and rejects mismatches |
| An old photo can't be replayed | Partly: replay deterrent only: the hub's AI must read the exact on-chain code, but a code can be edited into an old or generated image, and 4-digit codes repeat (1 in 10,000) |
| The photo shows what was asked | No: a photo with the correct code but the wrong subject can still be paid; the requesting agent judges the content after payment. Payment means "hub-accepted", not "task verified complete" |
| The hub reports deliveries honestly | Partly: trusted in v1 (the hub account signs `fulfill`). Roadmap: run the hub in a TEE (NEAR AI Cloud / Shade Agent) and have phones sign photos with a device key |
| The scene is real, not a photo of a screen | No: not provable by software alone. Roadmap: C2PA content credentials signed by the phone's camera hardware |
## Current limitations
- Payouts and refunds are fire-and-forget transfers; if a transfer fails (e.g. the account was deleted), there is no recovery ledger yet.
- `list_open_tasks` scans the newest 200 tasks, so a very old open task can drop out of the hub's feed (it can still be refunded).
- One hub, one demo worker; worker discovery and location matching are roadmap.
## Related work
"AI agents hiring human eyes" is an established category: **RentAHuman** already lets agents hire people through MCP/API, including photo tasks with escrow. Eyewitness's angle is an **open-source, self-hostable** implementation where the escrow and a per-photo receipt live on NEAR, the code check runs locally, and the agent drives **follow-up inspections** from what it sees.
## Deployed (testnet)
- **Demo contract:** [`eyewitness-live.testnet`](https://testnet.nearblocks.io/address/eyewitness-live.testnet) — hub `hub.eyewitness-live.testnet`, agent `agent.eyewitness-live.testnet`, worker `worker.eyewitness-live.testnet`
- Website with live on-chain stats (read straight from NEAR RPC in the browser): https://birserg.github.io/eyewitness/
- Demo video: https://x.com/neareyewitness/status/2121753087555867077 (updates: https://x.com/neareyewitness)
- First test deployment: [`eyewitness-ac7e.testnet`](https://testnet.nearblocks.io/address/eyewitness-ac7e.testnet). Example transactions from its smoke test:
[post_task](https://testnet.nearblocks.io/txns/AgPKUDwsH5xtsT6MHj4tvGZs6aEY6dhL4bGUoFoLFLkA) →
[fulfill (worker paid 0.009 NEAR)](https://testnet.nearblocks.io/txns/AY3NFRRnRzvw8WLvx6be9CH3cgHiHgz52M3CPREMrFoj) ·
[refund after expiry](https://testnet.nearblocks.io/txns/ATCaNwKdgP4BaRdkmQC2j2A6ZpSfcDomaa78YssX8gTU)
## Verify it yourself (do X → see Y)
| Do | See |
|---|---|
| `cd contract && cargo test` | `16 passed` — pay/refund/expiry/double-pay/reserved-worker/hash/storage rules |
| `node scripts/test-ai-check.ts 4821 scripts/fixtures/with-code-4821.jpg scripts/fixtures/no-code.jpg` (needs `ollama serve`) | `PASS … read="4821"` for the photo with the code, `FAIL` for the one without |
| `VERIFY=off npm run hub`, then `node scripts/sim-phone.ts`, then `node agent/src/demo.ts post "Photo of the sky" --timeout 60` | full loop without a phone: `[sim-phone] PAID for task #…`, agent prints `Hash verified` |
| `node scripts/regression.ts` (hub + Ollama running, a delivered task #1) | 12 × `PASS`: wrong-task and wrong-bytes results rejected, hub-claimed refunds checked on-chain, HTTP 429 RPC failover, per-worker credentials enforced, unpaid photos not served, exact code match |
| `node scripts/smoke.ts` (after setup + deploy) | `worker earned after: … +0.009`, `on-chain hash matches: yes`, `early refund rejected (expected)`, `refunded #… status=Refunded` |
| `npm run hub`, open `http://localhost:8787` | live dashboard: workers, task feed, verified photo gallery |
| open `<https-url>/phone?worker=<workerAccount>&token=<workerToken>` on a phone | live camera, incoming tasks with big challenge code |
| `node agent/src/demo.ts post "Photo of the sky right now" --reward 0.01` | task id + code; the phone shows it; after capture: photo saved, `hash`, explorer links |
| in Claude Code in this folder: *"Use eyewitness to read the model number on the device on my desk"* | Claude calls `request_photo`, pays, verifies the photo against the chain, and orders a close-up from the same worker if the label is unreadable |
## Use the MCP server in your agent
The MCP server signs as the `agentAccount` in `config.json` (run the setup steps in "Run it" first). It exposes `request_photo`, `list_eyewitnesses` and `refund_task`.
- **Claude Code, inside this repo:** nothing to install; `.mcp.json` registers it. Run `claude` here and approve the `eyewitness` server.
- **Claude Code, anywhere else:**
```bash
claude mcp add eyewitness -- node /absolute/path/to/eyewitness/agent/src/mcp.ts
```
- **Claude Desktop / Cursor / other MCP clients** (add to the client's MCP config):
```json
{ "mcpServers": { "eyewitness": { "command": "node", "args": ["/absolute/path/to/eyewitness/agent/src/mcp.ts"] } } }
```
Then ask your agent something it can't look up online, e.g. *"Use eyewitness to read the expiry date on the milk carton on my desk."* Roadmap: an `npx eyewitness-mcp` package configured by environment variables, and listings in the MCP registry and NEAR Agent Market.
## Build with AI
Paste this into Claude Code or Codex in a fresh folder:
> Clone https://github.com/birserg/eyewitness, read AGENTS.md, then set it up on NEAR testnet: install dependencies, run `bash scripts/setup-testnet.sh` and `bash scripts/deploy.sh`, run the contract tests and `node scripts/smoke.ts`, start the hub with `VERIFY=off`, and prove the full loop with `node scripts/sim-phone.ts` plus `node agent/src/demo.ts post "Photo of the sky" --timeout 60`. Report every transaction link.
## Run it
Prerequisites: Node ≥ 22.18 (runs the TypeScript directly), Rust via rustup (toolchain pinned in `contract/rust-toolchain.toml`), [`cargo-near`](https://github.com/near/cargo-near), [`near` CLI](https://github.com/near/near-cli-rs), [`cloudflared`](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/downloads/) (phones only allow the camera on HTTPS pages), [Ollama](https://ollama.com) for the AI check (`VERIFY=off` disables it).
```bash
npm install # or pnpm install
ollama serve & ollama pull qwen2.5vl:3b # local vision model for the AI check (~3 GB)
bash scripts/setup-testnet.sh # creates contract/hub/agent/worker testnet accounts via the faucet, writes config.json
bash scripts/deploy.sh # builds + deploys the contract, registers the demo worker
node scripts/smoke.ts # optional: proves pay + refund on-chain
cloudflared tunnel --url http://localhost:8787 # prints https://<random>.trycloudflare.com
PUBLIC_URL=https://<random>.trycloudflare.com npm run hub
# dashboard: http://localhost:8787
# phone: https://<random>.trycloudflare.com/phone?worker=<workerAccount>&token=<workerToken> (printed at startup)
node agent/src/demo.ts post "Show me the view from your window right now" --reward 0.01
# or let Claude do it: this repo ships .mcp.json, so `claude` in this folder gets the eyewitness tools
```
Keys live in `~/.near-credentials/testnet/` (NEAR legacy keychain). `config.json` is generated and gitignored; see `config.example.json`.
## Repo map
| Path | What |
|---|---|
| `contract/src/lib.rs` | NEAR contract (near-sdk 5): tasks, challenge codes, escrow, fulfill/refund, workers, NEP-297 events, 12 unit tests |
| `hub/src/server.ts` | hub: polls the chain job board, WebSocket to phones, confirms delivery on-chain, stores photos + metadata |
| `hub/src/verify.ts` | AI check: local vision model (Ollama) must read the challenge code in the photo before the hub pays |
| `hub/public/phone.html` | worker phone page: camera capture, tasks with challenge code, AI-check and rejection states, instant "paid" feedback, Blink helper |
| `hub/public/dashboard.html` | live dashboard + accepted-photos gallery |
| `agent/src/mcp.ts` | MCP server: `list_eyewitnesses`, `request_photo`, `refund_task` for Claude Code / Claude Desktop |
| `agent/src/demo.ts` | CLI for posting tasks, watching, refunds |
| `shared/near.ts` | small near-api-js wrapper shared by hub and agent; read calls fail over between public RPCs (they rate-limit) |
| `scripts/` | testnet setup, deploy, on-chain smoke test, AI-check test with fixture photos |
| `docs/` | project website (GitHub Pages) with live stats read from the contract |
| `SPEC.md` | interface spec all parts were built against |
## Where this goes
People are already paid per photo for real-world data: YC-backed apps pay per accepted photo task for AI training data, data vendors recruit freelance "photo data collectors", and retail-audit gig apps pay for shelf and store photos. Their shared problem is **trust**: recycled, off-site or manipulated photos, policed today with GPS, timestamps and duplicate checks inside a closed database.
Eyewitness makes part of that trust publicly checkable (which exact image was paid for, when, to whom), while the hub that accepts deliveries is still trusted in this version:
| Buyer | Need | What Eyewitness adds |
|---|---|---|
| AI agents (now) | "What does it look like there *right now*?" | MCP tool, pay-per-photo escrow, answer in minutes |
| Field audits / retail checks | Shelf, price, storefront, delivery proof | Freshness code from the chain, AI pre-check, public receipt per photo, instant payouts anywhere |
| AI labs (next) | Real, human-taken, consented images, not synthetic data | On-chain provenance per photo (who, when, hash), a reviewer layer that is paid per review |
Quality grows with a **reviewer network**, similar to the expert-reviewer marketplaces used for AI training data: reviewers stake reputation, approve or reject deliveries, and are paid per review. Worker and reviewer track records are public on NEAR, so good work becomes portable reputation.
## Stronger verification: options we evaluated
Today a photo is paid when the hub's AI reads the exact on-chain code; the requesting agent then judges the content. These are the ways to make each step harder to fake, in the order we plan to build them:
| Method | What it would prove | How hard to fake | Effort |
|---|---|---|---|
| EXIF metadata (time, GPS, device) | when and where the file *claims* it was taken | trivial (`exiftool` rewrites it); the phone page strips EXIF on purpose for privacy | small |
| Challenge code on paper (**today**) | an old photo is unlikely to contain the fresh code | medium: a code can be edited into an image | done |
| **Agent approval escrow** (reward stays locked until the agent accepts the content, auto-paid after a review window) | the photo shows what was asked; closes the "right code, wrong subject" gap | n/a | contract + agent tool |
| Short video with a random action ("tilt left") | a live scene rather than a still image | hard | medium |
| Two independent workers, pay on agreement | the scene matches across people | hard (needs collusion) | medium |
| C2PA content credentials from phone camera hardware | a real camera produced this exact image | very hard | native app |
| Device attestation (Play Integrity / App Attest) | the photo came from the genuine app on a real device | very hard | native app |
## Roadmap
- **Remove the trusted hub:** run delivery confirmation and the AI check inside a TEE (NEAR AI Cloud / Shade Agents) and have each phone sign its photos with a device key registered on-chain.
- **Camera-signed photos:** accept C2PA content credentials from phones whose camera hardware signs images, as a stronger "this is a real photo" signal.
- **Location-targeted tasks:** "a photo within 300 m of this point", matched to nearby workers.
- **Reputation and disputes:** agents rate deliveries; repeat bad actors lose access.
- **Payouts anywhere:** settle rewards to USDC on any chain through NEAR Intents.
- **More kinds of human help:** typed answers next to the photo ("read the model number"), judgments between options, corrections of ambiguous data; same escrow, deadline and receipt.
- **Dataset access:** license the growing set of on-chain-receipted, human-submitted photos to model builders, with revenue shared to the workers who took them.
## Built with / disclosure
Written during the NEAR HACKATHON window (Nov 2026) with the help of AI coding agents (Claude Code). Libraries: `near-sdk` (Rust), `near-api-js`, `ws`, `@modelcontextprotocol/sdk`, `zod`; tooling: `cargo-near`, `near-cli-rs`, `cloudflared`, Ollama with Qwen2.5-VL 3B. Test fixture photos in `scripts/fixtures/` are synthetic (generated by `make-test-images.py`).
## License
Apache-2.0, see [LICENSE](LICENSE). Contributions are accepted under the same license (see [CONTRIBUTING.md](CONTRIBUTING.md)); please follow the [Code of Conduct](CODE_OF_CONDUCT.md).
This server cannot be deployed
Maintenance
ActivityMaintained
ResponsivenessOutdated data