Skip to main content
Glama

HKEx Filing Scraper

An open-source scraper for 25+ years of HKEx regulatory filings — into any of nine databases, with full-text extraction, graph linking, and a read-only MCP server for AI agents.

HKEx Filing Scraper — one scraper, many databases

CI MCP Registry Glama MCP GitHub Release PyPI Python 3.10+ Docs MCP Ruff License: MIT

Overview

The US has EDGAR full-text search. Japan has EDINET. Hong Kong has a search form that returns one page at a time. There is no bulk, machine-readable, full-text corpus of HKEx filings. This builds one.

An open-source Python tool that scrapes 25+ years of Hong Kong Stock Exchange (HKEx) regulatory filings and ingests them into any combination of nine databases — with full-text and table extraction, chunk-level coverage, optional graph linking, and a read-only MCP server so AI agents can query the corpus or the live site.

It speaks the undocumented HKEx JSON API directly, which is faster and more resilient than driving a browser.

Related MCP server: internet-context-mcp

Vendors & Integrations

Databases — nine first-class destinations, in documented popularity order (see the support matrix):

  • PostgreSQL — production-grade open-source relational

  • MySQL / MariaDB — GPL relational servers, one driver

  • SQLite — zero-server file database, no install needed

  • MongoDB — document database

  • Neo4j — property-graph database

  • ClickHouse — columnar analytics engine

  • DuckDB — in-process analytical engine

  • SurrealDB — multi-model graph + document database

AI clients — any MCP-capable agent; ready-made configuration for Claude, ChatGPT, Cursor, VS Code/Copilot, Gemini CLI, opencode, Manus, and Perplexity.

Available on — PyPI · Glama · MCP Registry · hosted gateway.

Two Ways to Use It

Hosted MCP gateway

Local pipeline

What

A public endpoint you point an AI agent at

The hkex-scraper CLI

Setup

None — paste a URL

pip install + one environment variable

Data

Live from HKEx, nothing stored

Stored in your database(s)

Docs

Live MCP gateway · AI agent support

Getting started

Example: install, scrape filings into SQLite, then query the hosted MCP gateway from an AI agent

Use the Hosted MCP Gateway

POST, Streamable HTTP, no API key:

https://hkex-listco-updates.ascent-partners.com/api/mcp

Four read-only tools: get_server_info, search_filings (a window of at most 31 days, with optional stock-code, title, document-type, category, and stock-name filters), list_filing_facets (browse what a window contains), and get_filing (downloads one document and extracts its text and tables).

Two ways to reach HKEx filings from an AI agent: the hosted MCP gateway or the local stdio server

Point a client at it — for example opencode:

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "hkex-live": {
      "type": "remote",
      "url": "https://hkex-listco-updates.ascent-partners.com/api/mcp"
    }
  }
}

Then ask:

Use hkex-live to list the filings published between 2026-09-01 and 2026-09-18,
then summarise the interim report.

Ready-made configuration for Claude, ChatGPT, Cursor, VS Code/Copilot, Gemini CLI, opencode, Manus, and Perplexity is in AI agent support — and for a stored corpus, the stdio MCP server exposes a wider tool catalog and is published on Glama. The gateway is listed in the official MCP Registry as com.ascent-partners/hkex-filings.

Featured on Glama — the read-only stdio MCP server is also published on Glama, where Glama scans the built server and scores tool-definition quality (currently 4.7/5).

Quick Start (≤ 5 minutes)

Fastest path: no install — use the hosted gateway at https://hkex-listco-updates.ascent-partners.com/api/mcp (Streamable HTTP), or install and run locally:

pip install hkex-filing-scraper        # core; SQLite needs no server
pip install "hkex-filing-scraper[all]" # Excel + dotenv + every driver + the MCP server
cp .env.example .env                   # then set DATABASE_TARGET (below)
hkex-scraper --metadata-only --limit 100

Optional extras: excel, postgres, mysql, duckdb, mongodb, clickhouse, neo4j, mcp, pdf, all, dev.

DATABASE_TARGET is an ordered, comma-separated list of sink ids; the order decides which sink serves reads. To start with no server:

DATABASE_TARGET=sqlite
SQLITE_PATH=hkex.db

hkex-scraper runs the full pipeline (metadata + documents + graph); hkex-scraper --full-history covers everything since April 1999. The schema is created automatically. Full install options and per-sink settings are in Getting started.

Database Support

Every sink is a first-class destination; rows are in documented popularity order. The full matrix — licenses, capability differences, per-engine notes — is in Database sinks.

Sink

Model

License

Extra

Idempotent upsert

postgres

relational

PostgreSQL License

postgres

ON CONFLICT DO UPDATE

mysql / mariadb

relational

GPLv2

mysql

ON DUPLICATE KEY UPDATE

sqlite

relational

Public domain

—

ON CONFLICT DO UPDATE

mongodb

document

SSPL¹

mongodb

update_one(upsert=True)

neo4j

graph

GPLv3 (Community)

neo4j

MERGE

clickhouse

columnar

Apache-2.0

clickhouse

ReplacingMergeTree + read-merge

duckdb

relational

MIT

duckdb

ON CONFLICT DO UPDATE

surrealdb

graph + document

BSL 1.1¹

—

UPSERT / RELATE

¹ Source-available, not OSI-approved — labeled exceptions per ADR 0003.

Valid sink ids, in documented order: postgres, mysql, sqlite, mongodb, mariadb, neo4j, clickhouse, duckdb, surrealdb. Set one variable and the same run feeds every sink:

# Order sets read precedence.
DATABASE_TARGET=postgres,sqlite
POSTGRES_DSN=postgresql://user:password@localhost:5432/hkex
SQLITE_PATH=hkex.db

How This Compares

Four ways to get HKEx filings, and what each one costs you.

This project

HKEXnews web search

Browser automation you write

Licensed HKEx feed

Bulk export

Yes

No — page-at-a-time

Yes

Yes

History to April 1999

Yes

Yes, manually

Depends on your code

Yes

Full text of documents

Extracted from PDF/HTML/Excel

No — you open each file

You build the extractor

Varies by contract

Structured tables

Extracted to Markdown

No

You build it

Varies

Coverage verification

Per-chunk, auditable

Not applicable

You build it

Vendor SLA

Lands in your engine

9 engines, any combination

No

Whatever you wire up

Usually one format

Speed

JSON API, no browser

Manual

Slower — renders pages

Fast

Cost

Free, MIT

Free

Your time

Subscription

Commercial redistribution

See docs/legal.md

Restricted

Restricted

Licensed

If you need licensed, redistributable, SLA-backed data, buy the feed. If you need a complete local corpus for research, compliance, or RAG, this replaces the pipeline you would otherwise write yourself.

How It Works

flowchart LR
    A[HKEx JSON API] --> B[Phase 1: metadata]
    B --> C[Canonical record]
    C --> D{DATABASE_TARGET}
    D --> E[(PostgreSQL)]
    D --> F[(MySQL / MariaDB)]
    D --> G[(SQLite)]
    D --> H[(MongoDB)]
    D --> I[(Neo4j)]
    D --> J[(ClickHouse)]
    D --> K[(DuckDB)]
    D --> L[(SurrealDB)]
    B --> M[Graph linking]
    M --> D
    B --> N[Phase 2: download and extract]
    N --> C
  • Phase 1 scrapes filing metadata through a JSF session, splitting the range into monthly chunks and deduplicating on a 16-character MD5 filingId.

  • Phase 2 downloads each filing's PDF/HTML/Excel document, extracts text and tables to Markdown, and writes the payload.

  • Graph linking (optional) writes has_filing and references_filing edges when COMPANY_TABLE is set.

  • Failure isolation — a failure on one sink is logged and counted but never blocks another; the run exits non-zero if any configured sink failed.

Deeper detail: Architecture · ADR 0002.

HKEx API session

sequenceDiagram
    autonumber
    participant C as hkex-scraper
    participant J as HKEx site (JSF)
    participant A as HKEx JSON servlet
    C->>J: GET /search/titlesearch.xhtml
    J-->>C: HTML + javax.faces.ViewState
    C->>J: POST form (from/to dates + ViewState)
    C->>A: GET /search/titleSearchServlet.do (rowRange paging)
    A-->>C: JSON page of filings
    Note over C: generate_monthly_chunks() splits ranges > 1 month
    C->>C: dedupe on 16-char MD5 filingId

Data model

erDiagram
    COMPANY ||--o{ EXCHANGE_FILING : has_filing
    EXCHANGE_FILING ||--o{ DOCUMENT : has_document
    EXCHANGE_FILING ||--o{ EXCHANGE_FILING : references_filing
    EXCHANGE_FILING ||--o{ SCRAPE_COVERAGE : chunk_of
    EXCHANGE_FILING {
        string filingId "16-char MD5"
        string stockCode
        string title
        datetime dateTime
    }
    DOCUMENT {
        text documentText
        array documentTables
    }
    SCRAPE_COVERAGE {
        int apiCount
        int ingestedCount
        int uniqueCount
        string runId
    }

exchange_filing and scrape_coverage are SCHEMAFULL; has_filing / references_filing are the graph edges written by graph.py.

Sink contract and read routing

flowchart LR
    CANON["canonical record<br/>per filing / document / coverage / edge"] --> DISPATCH["dispatch to every configured sink"]
    DISPATCH --> S1["sink A"]
    DISPATCH --> S2["sink B"]
    DISPATCH --> SN["sink N"]
    S1 -.->|error| LOG["logged + counted<br/>never blocks the others"]
    READ["reads: pending filings · tickers · titles · coverage"] --> FIRST["first configured sink<br/>whose capabilities include reads"]
    DISPATCH --> EXIT["sink_exit_code()<br/>non-zero if any sink failed"]

Features

  • Fast API scraping — direct HKEx JSON API; no browser or Selenium.

  • Full history — every filing from April 1999 to today, with chunk-level coverage checks.

  • Document processing — PDF/HTML/Excel text and structured tables, extracted to Markdown.

  • Multi-sink — any ordered combination of nine databases, each with native idempotent upserts.

  • AI-ready — a hosted live MCP gateway plus a local stdio MCP server.

  • Resumable and observable — batching, parallel downloads, stalled-job detection, per-sink counters, and --coverage-report / --parity-report / --verify.

  • Optional dependencies — the core is requests + beautifulsoup4; drivers and document extraction are extras with graceful fallbacks.

Documentation

Development

pip install -e ".[dev,all]"
ruff check           # lint (py310, line-length 100)
ruff format --check  # formatting
pytest               # unit tests (no DB or network required)

Tests are pure unit tests; SQLite and DuckDB contract tests run in-process, and integration tests that need a server are skipped unless that sink is configured. See Testing.

CI and deployment run through the APG workbench pipeline.

The deploy smoke check probes the public hostname through the balancer, not a local port.

Contributing

See CONTRIBUTING.md; report security issues per SECURITY.md. Ideas and questions are welcome in Discussions.

Built by Ascent Partners.

If this saves you time, a ⭐ on GitHub helps others find it.

Use with Context7

Up-to-date HKEx Filing Scraper documentation is indexed on Context7, so coding agents can pull it into context on demand. With the Context7 MCP server or ctx7 CLI installed, name the library in your prompt:

use library /simonmak-ascent/hkex-filing-scraper for API and docs

License

MIT — see LICENSE. That covers this project's code only; optional dependencies carry their own licenses, notably the pdf extra (PyMuPDF / pymupdf4llm), which is AGPL-3.0 and deliberately excluded from .[all]. See docs/legal.md.

Data & Terms of Use: this is a research tool for the undocumented HKEx JSON API, and it is not affiliated with or endorsed by HKEx. Commercial redistribution of HKEx data may require a licensed HKEx feed; see docs/legal.md.

Available Tools

12 tools
describe_schemaA
Read-onlyIdempotent
Inspect

Use this before filtering or interpreting results to learn the canonical fields.

Use search_filings or search_documents to query filings once you know the field names. Pass section to return just one section (saves tokens); default returns everything: filing and document fields, the known filing types/categories/statuses/types, graph edge kinds, and the supported query filters. Reads no filings.

Inputs: section. Returns: result

get_coverageA
Read-onlyIdempotent
Inspect

Use this to report scrape coverage per monthly chunk, with totals.

Use get_statistics for filing counts grouped by a dimension, or verify_sinks for cross-sink comparison. date_from/date_to (YYYY-MM-DD) filter by chunk month. limit caps how many chunks are returned, but totals always aggregate the full filtered range. Rows are newest-first. Read-only.

Inputs: limit, date_to, date_from. Returns: result

get_filingA
Read-onlyIdempotent
Inspect

Use this to read one filing's metadata and extracted document content.

Returns the canonical filing and document fields. document_text is a window of max_text_chars from text_offset; when text_truncated is true, call again with next_text_offset for more. Tables (document_tables) are included only when include_tables is true. Obtain ids from search_filings. This tool is read-only.

Inputs: filing_id (required), text_offset, include_text, include_tables, +1 more. Returns: result

get_filingsA
Read-onlyIdempotent
Inspect

Use this to read several filings in one call (up to 50 ids).

Use get_filing instead to read a single filing. Returns each filing's metadata plus, when include_text is true, a bounded text window. Ids not found are listed in not_found. Text is off by default. Parameter relationships: filing_ids must come from search_filings (1..50 ids); max_text_chars applies per filing and only takes effect when include_text is true; include_tables is independent of the text window. Read-only.

Inputs: filing_ids (required), include_text, include_tables, max_text_chars. Returns: result

get_server_infoA
Read-onlyIdempotent
Inspect

Use this first to learn the server version, configured sinks, and read sink.

Use include="config" for the raw configuration values or list_sinks for per-sink detail. include is one of: summary (server metadata only, default), sinks (also returns the list_sinks payload), or config (also returns the configuration payload) — so you can pull the summary and the detail in a single call. Returns server metadata only; it reads no filings. include="sinks" is an alias for the list_sinks payload — call list_sinks directly when sinks are the only thing you need. This tool is read-only.

Inputs: include. Returns: result

get_statisticsA
Read-onlyIdempotent
Inspect

Use this to count filings grouped by one dimension, or per sink.

Choose between: this tool gives a grouped breakdown of one population (or per-sink totals when group_by="sink"); get_coverage reports scrape coverage by month; verify_sinks compares across sinks. group_by is one of: company_ticker (default), filing_type, filing_category, document_status, exchange, or sink. Optional filters narrow the population and combine with AND semantics (a filing must match every filter you set); group_by="sink" honours only ticker, filing_type, filing_category, document_status, date_from, and date_to, and returns per-sink totals (relational sinks only). top_n caps the returned buckets and min_count drops small buckets, but total still counts every matching filing. Buckets are sorted by count descending. This tool is read-only.

Inputs: top_n, source, ticker, date_to, +9 more. Returns: result

list_companiesA
Read-onlyIdempotent
Inspect

Use this to list companies, or just their ticker codes, that have filings.

view="companies" (default) returns each company's ticker, name, and filing count, ordered by filing count descending (ties broken by ticker ascending). view="tickers" returns only the distinct ticker codes, alphabetically sorted and paged. ticker is a case-insensitive substring match (e.g. '0700' matches '0700.HK'). Not for a company's filings (use search_filings) or for aggregate counts (use get_statistics with group_by="company_ticker"). This tool is read-only.

Inputs: view, limit, offset, ticker. Returns: result

list_referencesA
Read-onlyIdempotent
Inspect

Use this to explore the graph edges between companies and filings.

Use search_filings(ticker=...) for a company's own filings or search_filings( referenced_ticker=...) to find mentions via the filing column. This reads the canonical edge tables keyed on ticker (format like 0700.HK; discover valid values via list_companies(view="tickers")): kind="referenced_by" returns filings whose title mentions the company (cross-references), kind="owned" returns the company's own filings. Results are paged via limit/offset and stay empty until graph linking has been run. This tool is read-only.

Inputs: ticker (required), kind, limit, offset. Returns: result

list_sinksA
Read-onlyIdempotent
Inspect

Use this when the user asks which databases are configured or their capabilities.

Use get_server_info (with include="config") for the raw DATABASE_TARGET string or a one-line summary instead. Pass sink_id to inspect one sink without the full list. Returns every known sink id with its license, optional extra, configured/available status, per-sink capabilities, and which sink serves reads. Reads no filings. get_server_info with include="sinks" returns this same payload; call this tool directly when per-sink detail is all you need.

Inputs: sink_id. Returns: result

search_documentsA
Read-onlyIdempotent
Inspect

Use this for full-text search over extracted document text.

Use search_filings instead to filter by metadata without a text query. Matches text_query case-insensitively inside document_text and returns filing rows with a snippet when the sink supports it (see snippets_supported). document_type accepts pdf, html, xlsx, docx, or unknown; source is a free-text origin like HKEx. All filters combine with AND semantics (a filing must match every filter you set). Returns nothing until documents are processed. This tool is read-only.

Inputs: text_query (required), offset, source, ticker, +7 more. Returns: result

search_filingsA
Read-onlyIdempotent
Inspect

Use this to find filings by ticker, type, status, date range, or title text.

Prefer search_documents for full-text search over extracted text, and get_filing or get_filings to read filings whose ids you already have. Filters are optional and combinable; comma-separate a value to match several (e.g. filing_type="Annual Report,Dividend"). document_status accepts the real statuses plus unprocessed (no document yet). document_type accepts pdf, html, xlsx, docx, or unknown, and source is a free-text origin like HKEx. date_from/date_to are YYYY-MM-DD inclusive. order_by is one of filing_date_desc (default), filing_date_asc, title_asc, filing_id_asc. Returns paged filing rows (no document text); call get_filing for the document. This tool is read-only.

Inputs: offset, source, ticker, date_to, +11 more. Returns: result

verify_sinksA
Read-onlyIdempotent
Inspect

Use this to check that configured sinks agree on their filings.

mode="hashes" (default) compares (filing_id, document_sha256) sets across comparable sinks and returns a bounded sample of any missing/extra/mismatched ids. mode="counts" runs a faster count-only parity check, returning per-sink counts and the spread (parity is OK when the spread is zero). For a per-sink total or a grouped breakdown use get_statistics, and for monthly scrape coverage use get_coverage. Parameter relationships: sinks must list two or more comparable sink ids from list_sinks (empty compares every comparable pair); sample_size bounds the examples returned per problem bucket, not the comparison itself, and is ignored by mode="counts". This tool is read-only.

Inputs: mode, sinks, sample_size. Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A4.7/5.0

    Scored across 12 tools

    Disambiguation4/5

    Most tools are clearly separated by resource and action, and descriptions explicitly cross-reference alternatives. The main overlap is list_sinks versus get_server_info(include="sinks"), which return the same payload; get_server_info also overlaps with describe_schema and the sink-listing behavior.

    Naming Consistency5/5

    All tool names use consistent snake_case verb_noun patterns (list_sinks, get_statistics, search_filings, get_filing, verify_sinks, etc.). Singular/plural distinctions like get_filing vs get_filings are handled cleanly and predictably.

    Tool Count5/5

    Twelve tools is well-scoped for a filing-data server covering discovery, search, retrieval, statistics, coverage, sink verification, and references. Each tool has an apparent role, and the set avoids both thinness and bloat.

    Completeness5/5

    For a read-only HKEX filing server, the surface covers the full exploration lifecycle: schema discovery, sink inspection, company lookup, metadata search, full-text search, single/bulk filing retrieval, statistics, coverage checks, sink parity, and graph references. No obvious core read operation is missing.

    Maintenance

    ActivityActive
    ResponsivenessLimited data

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      A
      quality
      C
      maintenance
      SEC EDGAR filing MCP for equity research agents: search 10-K/10-Q/8-K with CompanyFacts metrics, preview a free sample, and purchase full structured JSON via x402 USDC on Polygon. Public endpoint on xpay.tools.
      3
      2
      MIT
    • F
      license
      A
      quality
      D
      maintenance
      A read-only MCP server that enables AI assistants to search files, list directories, retrieve system info, and get file metadata on the local file system.
      4
      -