discourse-mcp

A Model Context Protocol (MCP) server that lets any MCP-compatible AI app search, fetch, and synthesize what tech communities are discussing — across Hacker News, Lobsters, and Reddit (optional).

Ask your assistant "what's Hacker News saying about MCP this week?" and it answers from the actual threads — no browser, no copy-paste.

11 tools + 1 resource, a LangGraph orchestrator, and an A/B model eval harness.

Quick start

Requires Python ≥ 3.11. Works on Windows, macOS, and Linux.

git clone https://github.com/anushapundir/discourse-mcp.git
cd discourse-mcp

python -m venv .venv
source .venv/bin/activate           # Windows (PowerShell): .venv\Scripts\Activate.ps1

pip install -e .

No API keys needed. Hacker News and Lobsters are free, no-auth APIs, so search, threads, cross-source synthesis, and the watchlist all work immediately. Keys are only for the optional Reddit source and the optional orchestrator.

pip install -e . matters: installing as a package is what makes python -m server.main resolve no matter which directory your MCP host launches it from.

Check that it runs:

fastmcp dev inspector server/main.py

The Inspector should discover 11 tools and the watchlist://topics resource.

Related MCP server: hn-mcp

Connect it to your AI app

Your MCP host launches the server as a subprocess, so point it at the virtual-env Python and use the module form -m server.main.

Claude Desktop

Edit claude_desktop_config.json — on macOS at ~/Library/Application Support/Claude/, on Windows at %APPDATA%\Claude\:

{
  "mcpServers": {
    "discourse-mcp": {
      "command": "/absolute/path/to/discourse-mcp/.venv/bin/python",
      "args": ["-m", "server.main"]
    }
  }
}

On Windows, "command" is C:\\absolute\\path\\to\\discourse-mcp\\.venv\\Scripts\\python.exe.

Cursor

Same block, in Settings → MCP → Add new server or .cursor/mcp.json.

Restart the app, and you're done.

Then ask it things

"What's Hacker News saying about MCP this week?"

"Synthesize what tech communities are discussing about AI agents."

"Pull the top comments from that HN thread and tell me the main objection."

"Add 'rust async' to my watchlist."

What it gives the model

Tools (11)

Resources (1)

Reddit is optional and key-gated. It is read through ScrapeBadger as a temporary stopgap while official Reddit API access is pending. That is a paid service on trial credits, so the Reddit tools stop working when the credits expire — the official Reddit API is the intended long-term backend, and swapping to it is confined to server/clients/reddit.py. Without a key, the Reddit tools return a clean "not configured" message and everything else is unaffected.

How it works

discourse-mcp is a single local Python process speaking MCP over stdio. It does not listen on a port. The design is strictly layered — tools format, clients fetch, and the layers never reach sideways:

• server/tools/ — MCP-aware. Validates inputs (Pydantic), calls a client, formats markdown. Knows nothing about HTTP.

• server/clients/ — pure httpx. Talks to external APIs, returns typed models. Knows nothing about MCP.

• server/main.py — the FastMCP app. Registers tools and resources; implements none of them.

• server/db.py + server/resources/ — SQLite-backed watchlist state.

• orchestrator/ — a LangGraph agent that consumes this server as a real MCP client over stdio, with a provider-swappable model layer.

Two rules make it robust: stdout is reserved for the protocol (all logs go to stderr), and errors are data — a failed upstream call returns an "Error: …" string the model can reason about, never an exception that kills the server.

flowchart TB
    subgraph consumers["Consumers (MCP hosts)"]
        direction LR
        desktop["Claude Desktop / Cursor<br/>(interactive)"]
        orch["LangGraph orchestrator<br/>(programmatic)"]
    end

    consumers -- "stdio + JSON-RPC (MCP)" --> main

    subgraph server["discourse-mcp server (single Python process)"]
        direction TB
        main["server/main.py<br/>FastMCP app · registration"]

        subgraph tools["server/tools/ — MCP-aware"]
            direction LR
            hn_t["hackernews.py"]
            lob_t["lobsters.py"]
            reddit_t["reddit.py (optional)"]
            syn_t["synthesis.py"]
            wl_t["watchlist.py"]
        end

        subgraph clients["server/clients/ — pure HTTP (httpx)"]
            direction LR
            base["base.py<br/>retry + backoff"]
            hn_c["hackernews.py"]
            lob_c["lobsters.py"]
            reddit_c["reddit.py (optional)"]
        end

        subgraph state["State + resources"]
            direction LR
            db["db.py<br/>SQLite"]
            res["resources/watchlist.py<br/>watchlist://topics"]
        end

        fmt["formatting/markdown.py<br/>shared renderer"]

        main --> hn_t & lob_t & reddit_t & syn_t & wl_t
        main --> res
        hn_t --> hn_c
        lob_t --> lob_c
        reddit_t --> reddit_c
        syn_t --> hn_c & lob_c
        hn_c & lob_c & reddit_c --> base
        wl_t --> db
        res --> db
        hn_t & lob_t & reddit_t & syn_t -.-> fmt
    end

    subgraph external["External APIs"]
        direction LR
        algolia["Algolia HN Search<br/>(free)"]
        firebase["HN Firebase<br/>(free)"]
        lobsters_api["Lobsters JSON API<br/>(free)"]
        scrapebadger["ScrapeBadger<br/>(Reddit · paid, API key)"]
    end

    base -- HTTPS --> algolia & firebase & lobsters_api & scrapebadger

For the full design rationale — every decision, and what broke along the way — read docs/PROJECT_DEEP_DIVE.md. For the engineering standards, see CLAUDE.md.

The orchestrator (optional)

A LangGraph ReAct agent that consumes this server the same way Claude Desktop does — launching python -m server.main over stdio via langchain-mcp-adapters. The model provider is swappable by environment variable alone, with zero code changes.

pip install -e ".[orchestrator]"
cp .env.example .env                # Windows: copy .env.example .env

# Ask a one-off question (the agent picks the tools)
python -m orchestrator.main --ask "what is HN saying about rust this month"

# Run the autonomous daily-digest workflow over your watchlist
python -m orchestrator.main --workflow daily-digest

Set MODEL_PROVIDER in .env to anthropic (needs ANTHROPIC_API_KEY) or ollama (needs Ollama running locally — no key, no cost).

Configuration

Everything is environment variables. The orchestrator reads .env; the server reads the process environment (and falls back to .env for local runs, without overriding anything a host injects).

Evaluation: hosted vs. local model

The same agent and the same MCP server run under both providers — only MODEL_PROVIDER changes. A fixed judge (Claude Haiku, temperature=0) scores groundedness and relevance 1–5. Full methodology in evals/model_comparison.md.

Both providers genuinely drive the MCP tools, so the provider swap works end to end. But the local 7B model is ~50–200× slower on CPU, and on the synthesis prompt the judge caught it fabricating — invented URLs and future dates — where the hosted model stayed grounded.

python -m evals.run_eval            # free structural gate (14 cases, no API cost)
python -m evals.run_eval --compare  # paid A/B comparison → regenerates the table

Project layout

server/
├── main.py              # FastMCP app — tool + resource registration only
├── db.py                # SQLite watchlist storage
├── clients/             # pure HTTP (httpx) — base.py adds retry/backoff
├── tools/               # MCP tools — validate, call a client, format markdown
├── resources/           # read-only MCP resources
└── formatting/          # shared source-neutral StoryCard renderer

orchestrator/            # LangGraph agent (consumes the server over stdio)
evals/                   # hybrid eval harness (structural gate + judge)
docs/                    # architecture diagram + design deep dive

Engineering notes: async throughout (asyncio.gather for cross-source fan-out) · Pydantic at every boundary · a shared HTTP layer that retries only transient failures (connection errors, timeouts, 5xx, 429) and never 4xx, so a bad story id fails fast in ~0.7s instead of after three slow retries · errors returned as data so a tool call can never kill the server.

Roadmap

• Reddit via the official Reddit API, replacing the ScrapeBadger stopgap

• Reddit folded into get_trending_synthesis as a third source

• Streamable HTTP transport with OAuth 2.1

• Published to the public MCP registry

• Prompt-injection guardrails on tool outputs

Contributing

Contributions welcome — new sources especially. See CONTRIBUTING.md for setup and the layering rules, and CLAUDE.md for the full engineering standards. Pull requests need one approving review before merging.

License

MIT — see LICENSE.