Tender RAG — an MCP server that answers questions about tenders

This is a small AI app, exposed as an MCP server so any AI assistant (Claude Desktop, Claude Code, …) can use it as a tool. You ask a question about a government tender (a public contract offer), and it gives you an answer based on that tender's real documents — with references to the exact document and page it came from.

It is built on top of the tender scrapers in c:\anshul\MVP (separate programs that download tenders and their files from 9 government websites).

Example:

You: What is the closing date for tender RW10408567/26? App: The closing date of this tender is 28 August 2026 at 12:00 pm. (from the tender's cover page, page 1)

1. What problem does it solve?

A tender comes with long PDF/Word documents (sometimes 100+ pages). Reading them to find one fact — the closing date, who to contact, what documents you must submit — is slow. This app reads the documents for you and answers questions in a few seconds, and it shows its sources so you can trust the answer.

Very important: it only answers from the actual documents. If the answer is not in the documents, it says "not available" instead of making something up.

It also understands you even when you use different words — "ending date", "deadline" and "closing date" all mean the same thing — and it writes the answer as a clean sentence with nicely formatted dates and amounts, not raw data.

Related MCP server: Taiwan Tender MCP

2. How it works (in simple words)

Think of it like a very fast librarian:

1. You give it tenders. It reads each tender's documents and remembers them.

2. You ask a question. It finds the few paragraphs most related to your question, reads them, and writes a short answer with references.

To do this it uses three ideas. Here they are in plain English:

• Embedding = turning a piece of text into a list of numbers that captures its meaning. Two texts about the same thing get similar numbers. (Think of it as giving every paragraph a "GPS coordinate" based on meaning.)

• Vector database (pgvector) = a database that can store those number-lists and quickly find the ones closest in meaning to your question. (Like finding the nearest coffee shops to your location — but for meaning.)

• LLM (Large Language Model) = the "brain" that reads the found paragraphs and writes a human answer. Here we use Llama 4 (Meta's latest open model), reached over the cloud through LangChain.

Putting it together — this pattern is called RAG (Retrieval-Augmented Generation): Retrieve the relevant text, then Generate an answer from it.

Your question
   │
   ▼
turn question into numbers (embedding)        ← in-process, free (fastembed)
   │
   ▼
find the closest paragraphs (pgvector search) ← PostgreSQL
   │
   ▼
give those paragraphs + your question to the AI (Llama 4, via LangChain)
   │
   ▼
Answer + references (which document, which page)

3. The parts of the system (and why)

Why MCP instead of a website? Previously this ran as a web app you opened in a browser. Now it is an MCP server: instead of you typing into a chat page, your AI assistant calls the tender tools itself. That means you can ask "is the Rand Water cathodic-protection tender worth bidding for?" in a normal conversation, and the assistant will look it up in this system, read the real documents, and answer — mixing it freely with everything else it can do. There is no web page and no HTTP API any more; the assistant is the interface.

Why is the "meaning numbers" part built into the app? Making embeddings in-process (with a small model called bge-small, 384 numbers each) means there is no extra service to run, no API key, and no daily limit — it just works, on your PC and in the cloud. The documents never leave the app.

Why Llama 4, and why via LangChain? Llama 4 is a strong, current open model, and LangChain lets the app talk to it — or any OpenAI-compatible model — through one tidy interface. The app tries models in order: Llama 4 Maverick → Llama 4 Scout (both on OpenRouter), then, if those are rate-limited, a backup model on Groq (OpenAI gpt-oss), and only as a last resort a local model (Ollama, if you configure one) — so you always get an answer. Switching provider is just a new URL + key + model name in .env; no code changes.

Why not just ask ChatGPT/an LLM directly? Because a plain LLM doesn't know your specific tender's documents, and it can make things up. RAG forces the answer to come from the real documents and cite them.

4. Key words cheat-sheet

• Tender — a public contract a government advertises. Also called a "bid".

• Document — a file attached to a tender (PDF, Word, Excel).

• Chunk — a small piece of a document (about a paragraph). We split documents into chunks so we can find the exact relevant part, not the whole 100-page file.

• Embedding / vector — the list of 384 numbers representing a chunk's meaning.

• Ingest — the process of reading a tender + its documents into the database (and making the embeddings). "Ingested" = already loaded and ready to answer.

• RAG — Retrieval-Augmented Generation (the retrieve-then-answer method above).

5. What you need installed (already done on this PC)

• PostgreSQL 18 running, with the pgvector add-on installed.

• Python 3.12, with this project's packages installed in tender_rag/.venv (the embedding model, fastembed, is one of those packages — nothing extra to install or run).

• A free OpenRouter API key (in the .env file) so the app can reach Llama 4. An optional Groq key adds a backup model.

• Ollama is optional — only if you want an offline chat fallback when there's no internet. Embeddings do not need it.

If you ever set this up from zero, see Section 10 (Full setup) below.

6. How the data flows

6a. Ingesting a tender (loading it in)

1. A scraper has already produced a file like randwater/output/tender_RW10408567_26.json (tender info + the documents' text, page by page).

2. A mapper converts that file into one standard format (each of the 9 websites uses a different layout, so there's one mapper per website).

3. The tender info goes into the tenders table; each document's text goes into documents.

4. Each document is split into chunks; each chunk is turned into an embedding (by the built-in fastembed model) and saved in chunks (pgvector).

6b. Answering a question

1. Your question is turned into an embedding (in-process).

2. pgvector finds the top 8 most-similar chunks.

3. Those chunks + the tender's basic info + your question are sent to Llama 4 (through LangChain).

4. Llama 4 writes a short, well-formatted answer and cites the document + page.

7. How to use it (the fun part)

You don't start a website. You connect the server to an AI assistant once, and from then on you just talk to the assistant normally.

Connect it (one time)

Claude Code — from this folder:

claude mcp add tender-rag -- "c:\anshul\MVP - Copy\tender_rag\.venv\Scripts\python.exe" -m app.mcp_server

Claude Desktop — edit %APPDATA%\Claude\claude_desktop_config.json and add:

{
  "mcpServers": {
    "tender-rag": {
      "command": "c:\\anshul\\MVP - Copy\\tender_rag\\.venv\\Scripts\\python.exe",
      "args": ["-m", "app.mcp_server"],
      "cwd": "c:\\anshul\\MVP - Copy\\tender_rag"
    }
  }
}

Then restart Claude Desktop. A tools icon appears — the tender tools are in it.

The cwd matters: the server reads .env from the project folder.

Then just ask

• "What is this tender for and when does it close?" → the assistant calls ask_tender (give it the source + tender ID, e.g. ppadb / PR/PPADB/055).

• "Which tenders involve construction work?" → ask_all_tenders, across everything loaded.

• "Give me a full brief on the Rand Water cathodic protection tender." → summarize_tender.

• "What tenders do we have?" → list_tenders.

⚠️ Copy a Tender ID exactly, including any / slashes and spaces.

Answers come back in a few seconds, and always cite the document + page.

Which tenders are loaded?

Ask the assistant "list the loaded tenders" (the list_tenders tool). It takes an optional search filter, e.g. "list tenders from Rand Water". The scrapers cover 9 portals: etenders, transnet, sadc, zppa, ppadb, randwater, capetown, cpbn, nra.

Asking about a tender that isn't loaded yet (local only)

On your PC, you don't need to load it first — just ask with its source + ID and the app will run the scraper, load it, then answer (the first time is slower because it has to download it). Works for 8 of the 9 sites; nra needs a human check, so load that one by hand. (Set ENABLE_SCRAPING=false to turn auto-scrape off — e.g. when the server runs somewhere without the scraper programs. It then answers only tenders already loaded. See DEPLOY.md.)

Good questions to try

• "What is the published date?" vs "What is the closing date?" (different dates)

• "What is the deadline?" (a synonym — it still finds the closing date)

• "Are alternative bids allowed?" (an opposite/yes-no question)

• "Who is the contact person?" · "What documents must I submit?"

• "List all the construction tenders." (ask across all loaded tenders)

8. The MCP tools (for developers)

The assistant picks these automatically from their descriptions — you rarely name them yourself. Every tool is a thin wrapper over app/services, so the RAG pipeline is identical to the old HTTP version.

It also exposes:

• Resources — tender://catalogue (everything loaded) and tender://{source}/{tender_id} (one tender's record), for clients that browse context rather than call tools.

• A prompt — bid_assessment, a ready-made "should we bid on this?" workflow.

Reading the answer. The question-answering tools return markdown: the answer, then a Sources section listing document, page and relevance score. The catalogue/ingest tools return structured JSON as well, so scripts can consume them.

Inspect it by hand with the MCP Inspector (a browser UI for calling tools directly — needs Node.js installed, since it runs via npx):

.venv\Scripts\mcp dev app\mcp_server.py

9. Folder structure (what's where)

tender_rag/
  app/
    mcp_server.py      the MCP server — every tool, resource and prompt
    config.py          settings (read from the .env file)
    db.py              connects to PostgreSQL
    models.py          the 3 tables: Tender, Document, Chunk
    schemas.py         shapes of the tool inputs/outputs
    services/
      normalize.py     the 9 mappers (one per website)
      ingest_service.py loads tenders; also auto-fetches missing ones (local)
      scrape.py        runs a website's scraper on demand (local)
      chunking.py      splits documents into chunks
      embeddings.py    makes embeddings in-process (fastembed); Ollama/Gemini optional
      llm.py           answers via LangChain over a chat chain
                       (Llama 4 Maverick -> Scout on OpenRouter -> Groq gpt-oss backup -> optional local)
      retriever.py     the pgvector "find similar chunks" search
      prompts.py       the instructions given to the AI (synonyms, formatting, citing)
      rag.py           ties retrieve + answer together
      health.py        checks database / pgvector / providers; warms the models
  db/init.sql          creates the database tables (embedding VECTOR(384))
  scripts/             setup + loading helper scripts (load_data.py for a cloud DB)
  docs/architecture.md pictures/diagrams of the system
  Dockerfile           builds a container that serves the MCP server over stdio
  DEPLOY.md            connecting clients + using a hosted Postgres
  .env                 your settings + secret keys (keep private!)
  requirements.txt     the Python packages needed

(The old main.py, routers/ and static/index.html are gone — they were the FastAPI HTTP layer that mcp_server.py replaces. services/ is untouched.)

10. Full setup (from a blank machine)

Only needed if setting up somewhere new — this PC is already done.

1. Install PostgreSQL 18.

2. Add pgvector (the vector add-on): powershell -ExecutionPolicy Bypass -File scripts\install_pgvector.ps1

3. Create the Python environment (this also installs the embedding model):

   py -3.12 -m venv .venv
   .venv\Scripts\python -m pip install -r requirements.txt

4. Settings: copy .env.example to .env. Fill in:

   • POSTGRES_PASSWORD — your database password.

   • LLAMA_API_KEY — a free key from https://openrouter.ai (chat = Llama 4).

   • (optional) GROQ_API_KEY — a free key from https://console.groq.com (backup model).

   • EMBED_PROVIDER=fastembed (the default in-process embeddings).

5. Create the database + tables: powershell -ExecutionPolicy Bypass -File scripts\setup_db.ps1

6. Load some tenders: .venv\Scripts\python scripts\ingest_all.py

(Optional: install Ollama only if you want an offline chat fallback — winget install Ollama.Ollama, pull a small chat model, and set CHAT_MODEL in .env. Not used for embeddings.)

7. Connect it to your assistant: see Section 7.

Putting it online

By default the server runs on your machine over stdio, launched by your assistant. To reach it from anywhere, serve the HTTP transport instead:

.venv\Scripts\python -m app.mcp_server --http

That exposes /mcp and requires MCP_AUTH_TOKEN — it refuses to start without one, since the endpoint is public. render.yaml deploys it free on Render; see DEPLOY.md for the walkthrough. Hosted means query-only (no scraper binaries) and needs a hosted Postgres.

11. Start / stop the server

You don't start it yourself. Once connected (Section 7), your AI assistant launches the server when it needs it and shuts it down when it closes — there is no window to leave open and no port to remember.

To restart it after changing code or .env, restart the assistant (in Claude Desktop: quit and reopen).

Run it by hand — only useful for debugging; it will just sit waiting for JSON-RPC on stdin:

cd "c:\anshul\MVP - Copy\tender_rag"
.venv\Scripts\python -m app.mcp_server

Press Ctrl + C to stop. To poke at the tools interactively, use the Inspector (Section 8) instead.

The first question after a start takes a few seconds (the embedding model loads once — the server pre-warms it in the background); after that, most of each answer's time is the Llama 4 cloud call (~1–3s).

12. Common questions

• Do I need internet? Only for the Llama 4 call (the answer-writer). The embeddings and the database run on your machine. Set a local CHAT_MODEL (via Ollama) and drop the cloud keys to run fully offline with a local model (slower).

• Is it free? Yes. PostgreSQL and the embeddings are free; Llama 4 runs on an OpenRouter free key; and the whole thing can be hosted free (see DEPLOY.md).

• Does it use my ChatGPT/Claude account? The tender answer is written by Llama 4 (via OpenRouter, through LangChain), grounded in the documents, with embeddings from a small built-in model. Your assistant (e.g. Claude) only decides which tool to call and relays the result — it does not read the documents itself or invent tender facts.

• The first question after starting is slow (~a few seconds). That's the embedding model loading once. After that, each answer is mostly the Llama 4 cloud call (~1–3s).

• Asking about a brand-new tender is slow (local). The first time, it downloads it (scrape). After that it's instant. Large sites can take a few minutes.

• It said "not available for this tender". That's correct behaviour — the fact isn't in that tender's documents, so it won't guess.

• A Tender ID didn't work. Copy it exactly — many IDs contain / slashes and spaces (e.g. RW10414743/25 R). An underscore instead of a slash won't match.

• Where are my secrets? In tender_rag/.env (database password + the Llama 4 OpenRouter key, plus the optional Groq backup key). Keep this file private — don't put it in a shared repository.

13. Limits (honest)

• It can only answer about a tender it has loaded. It won't invent facts about tenders it has never seen.

• Answer quality depends on the scraped text. If a tender's scraper only captured a cover page, deep questions will correctly say "not available" — that's missing data, not the AI failing.

• A tender must still exist on the website to be scraped fresh; a closed/removed one can't be fetched — you'll get a clear message, not a guess.

• nra can't be auto-downloaded (human-verification check); load it manually.

• With ENABLE_SCRAPING=false the server is query-only — it answers loaded tenders but doesn't scrape new ones.

• One client at a time. A stdio MCP server is launched by (and belongs to) the assistant that started it. Two assistants each get their own copy of the process; they share the database, not the server.

For the technical diagrams (system, database, and step-by-step flows), see docs/architecture.md.