# Hearsay

[![Live](https://img.shields.io/badge/live-gethearsay.vercel.app-0b7a3e)](https://gethearsay.vercel.app) [![CI](https://github.com/A1VARA5/hearsay/actions/workflows/ci.yml/badge.svg)](https://github.com/A1VARA5/hearsay/actions/workflows/ci.yml) [![License: MIT](https://img.shields.io/badge/license-MIT-blue)](LICENSE)

Hearsay checks an AI's media plan against Qloo's audience data before anyone spends money on it. Built for the [Qloo Agentic Hackathon](https://qloo.devpost.com/), October 2026.

Ask an assistant which TV shows Taco Bell should advertise on and it says Rick and Morty, South Park, Stranger Things. Shows that fit the brand's image. Qloo says Taco Bell's customers score Peppa Pig higher than Rick and Morty, and what they actually love is Diners, Drive-ins and Dives, Man v. Food and Chopped. Hearsay is the planning agent that knows the difference: it drafts the plan, scores every pick against real customers, and rewrites the guesses.

**Live:** https://gethearsay.vercel.app (no login, nothing to install)

## For the judges

| Criterion | Where to see it |
|---|---|
| Use of Qloo | Every verdict is a Qloo insights score next to five controls and the brand's real top 50. Without Qloo there is no product. See [How it uses Qloo](#how-it-uses-qloo) and the [field notes](https://gethearsay.vercel.app/field-notes) |
| Agentic | An agent drafts the plan, gets its guesses back with Qloo's evidence, and rewrites them. It also runs as an MCP server other agents can call |
| Does it work | 100 question eval: picks no better than random went from 22% and 21% to 0% and 2% after one fix. See [Does it work](#does-it-work) |
| Impact | Media planning for brands and agencies, where AI shortlists already decide real ad money. See [Who it's for](#who-its-for) |

## Try it in 60 seconds

1. Open the site. The Taco Bell plan loads as Claude wrote it: 9 of 10 picks are no better than a random famous show. Three seconds later it flips to the checked plan: 10 of 10 backed.
2. Click a pick to see its evidence: its score, a famous show nobody would pick for these customers, and where real customers' top 50 starts.
3. Type your own brand, pick TV shows, podcasts, music artists, video games or films, and press "Make a checked plan". You watch the draft arrive, then the verdicts, then the rewrite. Usually under a minute. Five a day per visitor, the saved plans are free.
4. Copy the plan as text, or copy its link and send it on.

To use it from your own agent, add the MCP server:

```json
{ "mcpServers": { "hearsay": { "type": "http", "url": "https://gethearsay.vercel.app/api/mcp" } } }
```

`check_taste_claims` checks any "fans of X like Y" answer and returns a verdict per pick with what the audience prefers. `audience_picks` returns what fans of one thing love in another category.

## How it uses Qloo

Qloo is the part that decides. Without it Hearsay is just another assistant guessing by theme.

For one plan:

1. The planning agent (Claude Sonnet 5.5 or GPT-5.4 mini) drafts the plan.
2. Claude Haiku 5.5 pulls out every pick with the exact words that name it. A pick whose quote isn't in the answer is dropped, so the extractor can't invent one. If it returns fewer than half the names the answer puts in bold, it gets one more try, and if it's still short the plan says so.
3. Every name is resolved with `GET /search?query=...&types=urn:entity:<type>`. Exact names win, ambiguous ones ("Columbia" has three brands) are reported, never guessed.
4. One `GET /v2/insights` call scores all the picks plus five famous controls for the brand's audience at once (`signal.interests.entities=<brand>`, `filter.results.entities=<picks and controls>`, `take=<count>`).
5. A second insights call gets the brand's real top 50 in that category (`take=50`).
6. Verdicts: a pick inside the real top 50, or clearly above the middle control, is backed. A pick no higher than the middle control is no better than a random famous pick. Raw affinity alone isn't enough: Michael Bublé scores 0.76 with Grand Theft Auto fans.
7. The agent gets its guesses back with their scores, the control score and the real top picks, and rewrites. Hearsay checks the rewrite the same way. A rewrite that passes by deleting picks is flagged.

Every Qloo call goes through one client (`src/qloo/client.ts`): a Postgres cache first, a 5 per second limiter, and a monthly budget guard that reads `x-month-ratelimit-remaining` on every response and stops live checks before the month runs out, plus a daily share of live calls, so the saved plans keep working and one busy day can't spend the month. The things I learned about the API along the way are on the [field notes page](https://gethearsay.vercel.app/field-notes), each with a request to reproduce it.

## Does it work

I put 100 questions to both agents ("What TV shows do Taco Bell fans watch?", "What music do Formula 1 fans listen to?"), checked every pick, ran the fix loop, and checked again. 192 runs, about $3 in model calls. Scripts: `eval/questions.json`, `eval/hunt.mts`, `eval/report.mts`.

| Agent | Answer | Checked picks | Backed | No better than random |
|---|---|---|---|---|
| Claude Sonnet 5.5 | first | 1008 | 62% | 22% |
| Claude Sonnet 5.5 | after the fix | 1042 | 85% | 0% (5 picks) |
| GPT-5.4 mini | first | 848 | 66% | 21% |
| GPT-5.4 mini | after the fix | 895 | 91% | 2% |

Where the guessing happens: picks that cross categories (a brand to TV shows, a team to music) were no better than random 23% of the time, against 10% inside one category. A media plan always crosses categories. Sport was the worst world (41%), then games (32%).

Honest limits of this eval. The verdicts say what Qloo's data supports, not that an AI was wrong. The sample review (60 "no better" picks and 30 "backed" picks) was done by a second model, not a person: 51 of the 60 looked like picks chosen by theme, 29 of the 30 backed picks held up. The podcast controls changed on 16 October (three of them were shows planners actually pick), after the eval ran.

## Who it's for

Brand and agency planners who already ask an assistant for sponsorship shortlists, and teams building planning agents. 54% of 211 US ad buyers surveyed by IAB in September 2026 expect to put more time into agentic ad buying and campaign execution this year ([IAB](https://www.iab.com/wp-content/uploads/2026/09/IAB_2H_2026_Ad_Spend_Outlook_Study.pdf)). US podcast advertising alone was $2.9 billion in 2025 ([IAB and PwC](https://www.iab.com/wp-content/uploads/2026/04/IAB_PwC_Internet_Ad_Revenue_Report_Full_Year_2025_April_2026.pdf)). A placement picked for the brand's image instead of its customers is money spent on someone else's audience.

## Run it locally

Node 22+. Local runs use an embedded Postgres (PGlite) in `.data/`, so there's nothing to install.

```bash
npm install
cp .env.example .env.local      # QLOO_API_URL, QLOO_API_KEY, ANTHROPIC_API_KEY, OPENAI_API_KEY
npm test                        # 179 tests, no keys needed
npm run dev                     # http://localhost:3000
```

The saved plans come from real runs that hold Qloo data, so they aren't in the repo. Make your own with `npx tsx scripts/make-plan.mts tacobell-tv "Taco Bell" tv_show claude` and load them with `npx tsx eval/seed.mts`. Production uses Neon: set `DATABASE_URL`, `HEARSAY_IP_SALT`, and optionally `CUSTOM_CHECKS_PER_IP`, `CUSTOM_CHECKS_PER_DAY`, `MCP_CHECKS_PER_IP`, `MCP_CHECKS_PER_DAY`, `QLOO_CALLS_PER_DAY` (a daily share of live Qloo calls, so nobody can spend the month in a day), `QLOO_BUDGET_FLOOR`, or `HEARSAY_LIVE=off` to stop live runs.

## Where to look

| What | Where |
|---|---|
| The whole check: extract, resolve, score, verdict | `src/check/index.ts`, `extract.ts`, `score.ts`, `verdict.ts` |
| Famous controls per category | `src/check/controls.ts` |
| Qloo client (cache, limiter, budget guard) and name resolution | `src/qloo/client.ts`, `resolve.ts` |
| Planning agent, fix loop, deletion guard | `src/agent/answer.ts`, `fix.ts`, `run.ts` |
| Public API with caps and the progress stream | `src/app/api/check/route.ts`, `src/lib/check-api.ts`, `caps.ts`, `stream.ts` |
| MCP server | `src/app/api/mcp/route.ts`, `src/mcp/tools.ts` |
| The plan sheet | `src/components/Planner.tsx`, `src/ui/sections.ts` |
| Eval | `eval/` |

## Limits

- Qloo knows a lot of podcasts but not every one; a pick it can't find is marked "can't check", never counted against the agent.
- Age group and location claims aren't checked: scoring one entity inside a city came back empty, and some age slices looked skewed (both in the field notes).
- Book audiences look young adult heavy, so book picks are marked low trust.
- Affinity measures how closely something is connected to a brand's audience, and Qloo normalises it per query, which is exactly why Hearsay compares every pick with controls from the same query instead of trusting a raw score. It isn't reach or price, so this is a first filter for a plan, not the whole plan.

## Credits

Qloo Insights API for every score. Claude Sonnet 5.5 and Claude Haiku 5.5 (Anthropic) and GPT-5.4 mini (OpenAI) as the agents and the extractor. Next.js, Neon, Vercel, PGlite, Zod.

## License

MIT, see [LICENSE](LICENSE).