crowdcheck
by avnath13
README.md
# crowdcheck
[](https://github.com/mvanhorn/last30days-skill)
### The contestation layer for last30days. Community Notes for AI agents.
**[last30days](https://github.com/mvanhorn/last30days-skill) tells you what people are saying about a topic right now, scored by real engagement. `crowdcheck` turns that into something an agent can act on: it maps who agrees, who disputes, and how sure the signal is.** Not a true/false verdict. A **contestation map**.
`crowdcheck` is a companion to [last30days](https://github.com/mvanhorn/last30days-skill) by [@mvanhorn](https://github.com/mvanhorn). last30days does the genuinely hard part: reaching Reddit, X, YouTube, TikTok, Hacker News, and real-money prediction markets, and ranking everything by what people actually engaged with. `crowdcheck` adds a thin, deterministic layer on top: stance classification, a two-axis calibration (**consensus** and **confidence**), and a typed contract your agent calls. It can run without last30days too (a keyless Hacker News plus prediction-market fallback), but it is built to sit on top of the full engine.
> **Status: experimental (v0.1).** The quality of the map depends on the quality of the evidence, so `crowdcheck` shines with last30days behind it (or with an agent doing the reasoning). The standalone keyless fallback is intentionally light. See [Status and honest limits](#status-and-honest-limits).
*A contestation map (shown in `--demo`). This is what a last30days-backed run looks like.*
## The one idea
Most tools collapse "do people agree?" and "do we have enough signal?" into a single fuzzy score. `crowdcheck` keeps them separate, because *"confidently contested"* is a completely different, and far more useful, answer than *"we're not sure."*
## How the two fit together
```
last30days crowdcheck your agent
---------- ---------- ----------
reach Reddit, X, YouTube, -> classify stance, -> branch on a typed
TikTok, HN, prediction calibrate consensus + contract instead of
markets; rank by engagement confidence; emit a map guessing from prose
```
last30days is the retrieval and engagement layer. `crowdcheck` is the judgment layer: it reads that ranked evidence and produces `consensus`, `confidence`, weighted `positions`, and a market signal, as a machine-readable contract.
## Quickstart
```bash
pip install "crowdcheck[mcp]" # pre-release: see the install note below
```
**Recommended - with last30days (the full experience):**
```bash
git clone https://github.com/mvanhorn/last30days-skill ~/.claude/skills/last30days
export CROWDCHECK_LAST30DAYS_DIR=~/.claude/skills/last30days
crowdcheck "AI agents will replace most junior developer jobs by 2027"
```
**No setup at all - offline demo:**
```bash
crowdcheck "AI agents will replace most junior developer jobs by 2027" --demo
```
Without last30days, `crowdcheck` falls back to a keyless Hacker News plus prediction-market search. That runs with zero setup but gives lighter coverage (Reddit blocks keyless access; see [Troubleshooting](#troubleshooting)).
> **Pre-release:** not on PyPI yet. Until the first release, install from source:
> `pip install "git+https://github.com/avnath13/crowdcheck"`. On macOS, add the `[native]` extra (`pip install "crowdcheck[native]"`) for a working CA bundle.
You get back a contestation map, not a thumbs-up or thumbs-down:
- **consensus `0.06`** - the crowd is deeply split.
- **confidence `0.85`** - but there is plenty of signal, so we are *sure* it is split.
- **positions** - 40% dispute / 34% support / 25% "it depends", each with the top quote and where it came from.
- **market** - a real-money prediction market puts the related bet at 34%.
## What `crowdcheck` adds
| | last30days | `crowdcheck` on top |
|---|---|---|
| **Answers** | what are people saying, and what did they engage with | do they *agree*, and can we *trust* the signal |
| **Output** | a brief for a human to read | a typed contract an agent branches on |
| **New fields** | engagement-ranked evidence | `consensus`, `confidence`, `positions[]`, `leaning` |
| **For** | research, catching up | agents deciding whether to assert a claim |
If you just want the human-readable brief, use last30days directly. `crowdcheck` is for when an agent (or a pipeline) needs a structured, uncertainty-calibrated read *in the loop*.
## Works anywhere you do (MCP)
`crowdcheck` ships a standard **stdio MCP server**, so it drops into any MCP-compatible tool. Install once, then point your client at the `crowdcheck-mcp` command.
**Claude Code**
```bash
claude mcp add crowdcheck -- crowdcheck-mcp
```
Or add to `.mcp.json` (project) or `~/.claude.json` (global), then verify with `/mcp`:
```jsonc
{ "mcpServers": { "crowdcheck": { "command": "crowdcheck-mcp" } } }
```
**Cursor, Claude Desktop, Windsurf, Cline** (the common `mcpServers` shape)
Add to `~/.cursor/mcp.json` (Cursor global) or `.cursor/mcp.json` (project). Claude Desktop uses `claude_desktop_config.json`; Windsurf uses `~/.codeium/windsurf/mcp_config.json`:
```jsonc
{ "mcpServers": { "crowdcheck": { "command": "crowdcheck-mcp", "args": [] } } }
```
**VS Code** (Copilot agent mode) - note the key is `servers`, not `mcpServers`. Add to `.vscode/mcp.json`:
```jsonc
{ "servers": { "crowdcheck": { "type": "stdio", "command": "crowdcheck-mcp" } } }
```
**Codex CLI** - `codex mcp add crowdcheck -- crowdcheck-mcp`, or add to `~/.codex/config.toml`:
```toml
[mcp_servers.crowdcheck]
command = "crowdcheck-mcp"
```
**Not on your PATH?** Use the module form anywhere `crowdcheck-mcp` appears (use an absolute path to `python` if the client does not inherit your shell PATH):
```jsonc
{ "mcpServers": { "crowdcheck": { "command": "python", "args": ["-m", "crowdcheck.mcp_server"] } } }
```
Once connected, your agent has one tool, `crowd_check`, and the built-in reflex:
```
high confidence + high consensus -> state it plainly
high confidence + low consensus -> say it is contested; show both sides
low confidence (thin) -> do not overclaim; flag the uncertainty
```
The agent calls it on its own, gets the JSON contract back, and acts deterministically instead of guessing from prose. This is where `crowdcheck` is strongest: the agent supplies the reasoning over the returned quotes on its own plan, so you do not need a separate model or key.
## The output contract
```jsonc
{
"claim": "AI agents will replace most junior developer jobs by 2027",
"mode": "live", // "live" | "demo"
"consensus": 0.06, // 0 = evenly split, 1 = one-sided agreement
"confidence": 0.85, // 0 = insufficient signal, 1 = strong signal
"leaning": "split", // "supported" | "disputed" | "split"
"read": "Highly contested, no clear side; market puts it at 34%. Signal: strong.",
"positions": [
{ "stance": "dispute", "weight": 0.40, "count": 3, "top_quote": "...agents can't own context...", "who": "reddit (2,600), hackernews (1,240)", "sources": ["https://..."] },
{ "stance": "support", "weight": 0.34, "count": 3, "top_quote": "...we stopped backfilling junior roles...", "who": "reddit (1,980), youtube (51,000)", "sources": ["https://..."] },
{ "stance": "qualify", "weight": 0.25, "count": 2, "top_quote": "...reshaping junior work, not deleting it...", "who": "x (910)", "sources": ["https://..."] }
],
"market_signal": { "question": "Will entry-level dev postings fall >30% by end of 2027?", "probability": 0.34, "source": "prediction market" },
"coverage": { "sources_hit": ["reddit","x","hackernews","youtube"], "item_count": 8, "distinct_authors": 8, "thin": false }
}
```
From Python:
```python
from crowdcheck import crowd_check
cmap = crowd_check("React Server Components are production ready", demo=True)
if cmap.confidence < 0.35:
print("thin signal, do not rely on this")
elif cmap.consensus < 0.5:
print("contested:", [p.stance for p in cmap.positions])
else:
print("settled:", cmap.leaning)
```
## Evidence sources
`crowdcheck` owns the judgment layer and treats retrieval as a pluggable provider. Pick the source with a one-flag toggle - same run, no code changes:
| Where | With last30days (recommended) | Keyless fallback |
|---|---|---|
| **CLI** | `crowdcheck "your claim" --source last30days` | `crowdcheck "your claim" --source native` |
| **Python** | `crowd_check("your claim", source="last30days")` | `crowd_check("your claim", source="native")` |
| **Agent (MCP)** | call `crowd_check` with `source: "last30days"` | tool default |
`auto` (the default) uses last30days when `CROWDCHECK_LAST30DAYS_DIR` (or `--last30days-dir PATH`) is set, and otherwise falls back to the native keyless provider. Valid sources: `auto`, `last30days`, `native`, `demo`.
- **`LastThirtyDaysProvider`** (recommended) - the full engine: Reddit, X, YouTube, TikTok, Hacker News, prediction markets, and more, ranked by engagement. Keyless for Reddit, HN, markets, and GitHub; bring your own keys or browser sessions for X, YouTube, and TikTok.
- **`NativeProvider`** (zero-setup fallback) - keyless Hacker News (stories and comments) plus a prediction market (Manifold), and Reddit on a best-effort basis (Reddit blocks keyless access on many networks). Claims are reduced to keywords before searching for better recall; sources fail independently and print a one-line notice rather than crashing.
- **`DemoProvider`** - bundled offline sample evidence for `--demo` and the tests.
Bring your own provider by implementing `fetch(claim) -> (list[Evidence], MarketSignal | None)`.
For LLM-grade stance labels (the built-in heuristic is intentionally simple), pass a classifier - no `crowdcheck` key required, you supply the model:
```python
from crowdcheck import crowd_check, PromptedLLMClassifier, openai_compatible_complete
complete = openai_compatible_complete(model="gpt-4o-mini", api_key="sk-...") # or any local/other endpoint
cmap = crowd_check("AI agents will replace junior devs by 2027",
classifier=PromptedLLMClassifier(complete))
```
## Viral is not true, and here is how `crowdcheck` handles it
People upvote things that are wrong. A confident falsehood can go viral while the correct, boring correction gets ignored. The design accounts for that in four concrete ways, and stays honest about the one thing it cannot fix:
- **It never says "true" or "false".** `consensus` measures whether people *agree*, not whether they are *right*. There is no verdict field, on purpose. Read high consensus as "widely believed," never as "correct."
- **One loud voice cannot dominate.** `confidence` includes an independence term, so a single viral post reposted a hundred times does not manufacture strong signal.
- **The dissent is always shown.** `positions` surfaces the dispute and qualify camps with their strongest quotes, right beside the popular view.
- **Money as a check on vibes.** For predictive claims, `market_signal` pulls in real-money prediction-market odds.
What it cannot do: catch a genuine, widely-held misconception where almost nobody pushes back. That reads as high consensus, and `crowdcheck` reports it as such. For settled-fact questions, pair it with an authoritative source. `crowdcheck` maps the conversation; it does not verify facts.
## Status and honest limits
- **Experimental (v0.1).** APIs and output shape may change.
- **The map is only as good as the evidence.** With last30days behind it you get Reddit, X, YouTube, TikTok, and more. The standalone keyless fallback is intentionally light: reliable Hacker News and prediction markets, best-effort Reddit.
- **Stance labeling is heuristic by default.** Titles rarely trip a keyword lexicon, so a keyless run often shows solid `confidence` but low `consensus`. Wire an LLM classifier, or run `crowdcheck` inside an agent, where the model reasons over the returned quotes itself.
## Troubleshooting
- **Live search returns 0 items or "thin" on macOS.** Almost always an SSL certificate problem (`CERTIFICATE_VERIFY_FAILED`) with a python.org interpreter. Fix: `pip install "crowdcheck[native]"` (bundles a CA store via certifi), or run your Python's `Install Certificates.command`.
- **`'reddit' source unavailable (HTTPError: 403: Blocked)`.** Reddit blocks keyless access on many networks. This is expected, not a bug; use last30days (`--source last30days`) for reliable Reddit.
- **Confidence is solid but consensus is ~0 with no positions.** The built-in heuristic classifier only keyword-matches. Wire an [LLM classifier](#evidence-sources), or run `crowdcheck` inside an agent.
## Roadmap
- [ ] Smarter query expansion and entity resolution for the native fallback
- [ ] LLM stance classifier presets (OpenAI, Anthropic, local)
- [ ] Compound-claim decomposition (split "A and B" into separate checks)
- [ ] Batch mode: contestation-check every claim in a document
- [ ] `crowd_check_batch` MCP tool
Contributions welcome, especially stance classifiers and evidence providers. See [CONTRIBUTING.md](CONTRIBUTING.md).
## Credits and acknowledgments
`crowdcheck` is built on and powered by **[last30days](https://github.com/mvanhorn/last30days-skill)** by [@mvanhorn](https://github.com/mvanhorn), which does the hard work of reaching a dozen walled-garden platforms and ranking by real engagement. If you find `crowdcheck` useful, star last30days first. last30days is MIT licensed; `crowdcheck` uses it as an optional external provider and is not a fork of it.
`crowdcheck` is MIT licensed (see [LICENSE](LICENSE)). It is an independent project and is not affiliated with last30days, its author, or any of the platforms it reads.
This server cannot be deployed
Maintenance
ActivityInactive
ResponsivenessNo issues