qlib-sync
# qlib-sync
[](https://github.com/randyy179/qlib-sync/actions/workflows/ci.yml)
[](https://pypi.org/project/qlib-sync/)
[](LICENSE)
[中文说明](README.zh-CN.md)
**Website:** [randyy179.github.io/qlib-sync](https://randyy179.github.io/qlib-sync/)
<!-- mcp-name: io.github.randyy179/qlib-sync -->
**One command to install and keep current a local [Qlib](https://github.com/microsoft/qlib) China A-share dataset, and an MCP server so AI assistants can read it.**
```bash
pip install qlib-sync
qlib-sync cn update
```
Or without installing anything first: `uvx --python 3.12 qlib-sync cn update`.
(`--python 3.12` because pyqlib publishes wheels up to Python 3.12; uv fetches that Python
for you if needed. With pip, use Python 3.10 to 3.12.)

That installs the daily community release from
[chenditc/investment_data](https://github.com/chenditc/investment_data) into
`~/.qlib/qlib_data/cn_data`, the location Qlib reads by default. Run the same command again
any time; when the data is already current it finishes in about a second.
## Why not just `wget` and `tar`?
<img src="docs/assets/comic-en.jpg" width="560" alt="Four-panel comic: a download dies at 99% at night; the next day 550 MB arrive and nothing changed; the qlib-sync robot checks the data and stamps it verified, keeping the old copy safe; the researcher sips tea and asks an AI for Moutai's recent closes">
Qlib's README points A-share users to the community release with a `wget` and `tar` one-liner.
That works until it doesn't:
- **An interrupted or truncated download** extracts into a half-written dataset, or fails
after the old one has already been overwritten.
- **Upstream republishes releases.** In November 2026 a data error was found and corrected
([investment_data#163](https://github.com/chenditc/investment_data/issues/163)) and releases
were republished under the same date. A date check alone never sees the fix.
- **Every check downloads 550 MB**, even when nothing changed.
The investment_data README itself describes a careful manual process: download the release
manifest, check the archive with `qlib/validate_archive.py --require-publishable`, then
extract. qlib-sync runs that validator itself, on every update: an unmodified copy (from
[commit 68fdc66](https://github.com/chenditc/investment_data/blob/68fdc66b109459e522050c94723684525aea23ab/qlib/validate_archive.py),
Apache-2.0) checks the manifest before the download and the archive after it. A test suite
feeds the same malformed releases to the upstream script and to qlib-sync and requires the same
verdict, and a weekly job flags any change upstream makes to the validator.
On top of that, qlib-sync skips the download when nothing changed, pins the manifest and the
archive to one release tag, extracts beside the live data and swaps it in with rollback
(rather than extracting on top of the existing folder), and picks up republished releases.
qlib-sync treats the dataset like a production system:
<picture>
<source media="(prefers-color-scheme: dark)" srcset="docs/assets/update-flow-dark.svg">
<img alt="Update flow: check the manifest, download, verify, stage, swap; any failure puts the previous dataset back" src="docs/assets/update-flow-light.svg">
</picture>
| Step | What happens |
|---|---|
| Check | Pins the latest release by tag and reads its 500-byte manifest. Already current: done, nothing downloaded. |
| Download | Resumes interrupted downloads; shows progress; optional mirror for slow links to GitHub. |
| Verify | investment_data's own `validate_archive.py --require-publishable`, unmodified: manifest fields and formats, size and sha256, every required calendar and index pool, date formats, ordering, and the future calendar. Anything it rejects is refused. |
| Stage | Extracts next to the active dataset; refuses downgrades. |
| Swap | Renames the new dataset into place, then runs a real Qlib read. |
| Roll back | Any failure after the swap moves the previous dataset back. |
| Report | New trading days, newly listed and delisted symbols. |
**Verification is required by default.** If the manifest is missing, unreadable, or rejected by
the validator, the update stops and your data stays as it was. `--allow-unverified` installs
anyway when the manifest cannot be verified, and the result and install record are then marked
unverified; a package whose checksum disagrees with a valid manifest is always refused.
The checks catch broken, truncated, tampered, or malformed packages. They cannot catch wrong
values inside a correctly published release (like the one in #163); for that, keep a rollback
copy with `--keep-backup`.
## Commands
```bash
qlib-sync status # every local Qlib dataset at a glance
qlib-sync cn status # local date, pools, and whether a newer release exists
qlib-sync cn update # install or update (add --json for machine-readable output)
qlib-sync cn health-check # structure checks plus a real Qlib read
qlib-sync cn restore --list # rollback backups; restore --yes switches back
```
Useful options for `cn update`:
| Option | Effect |
|---|---|
| `--data-dir PATH` | Install somewhere other than `~/.qlib/qlib_data/cn_data` |
| `--mirror URL` | Download through a prefix-style GitHub proxy such as `https://ghfast.top` (or set `QLIB_SYNC_MIRROR`) |
| `--keep-backup` | Keep the previous dataset for one rollback |
| `--protect FILE` | Carry a custom stock pool in `instruments/` across updates (repeatable) |
| `--force` | Reinstall even when the dates match |
| `--allow-unverified` | Install when the manifest cannot be verified (marked unverified; checksum mismatches are still refused) |
**The update replaces the whole `cn_data` folder.** Anything you added there is lost unless you
pass it with `--protect`.
### Mirrors
From mainland China, GitHub downloads can be slow or fail. Pass a mirror and qlib-sync downloads
the package through it while still reading the manifest from GitHub directly when it can, so the
checksum does not depend on the mirror. Mirrors are third-party services; qlib-sync does not
endorse any of them.
## Python
```python
from qlib_sync import update_cn, status_cn, health_check_cn
result = update_cn() # raises qlib_sync.UpdateError on failure; data unchanged
print("\n".join(result.summary_lines()))
print(result.new_trading_days, result.added_symbols)
```
## Use it from your AI assistant
qlib-sync includes an [MCP](https://modelcontextprotocol.io) server, the open standard most AI
tools use to call local tools. Any MCP client that can start a local command works, whichever
model it runs: Claude, GPT, Gemini, DeepSeek, Qwen, Kimi, GLM, or a local model. The server
runs on your machine and reads the dataset there; there is no service to host and nothing
leaves your computer. It is listed in the [MCP Registry](https://registry.modelcontextprotocol.io)
as `io.github.randyy179/qlib-sync`.
Every client needs the same two things: the command `uvx` and the arguments `qlib-sync mcp`.
| Client | How to add qlib-sync |
|---|---|
| Claude Desktop | Download `qlib-sync.mcpb` from the [latest release](https://github.com/randyy179/qlib-sync/releases/latest) and open it |
| Claude Code | `claude mcp add qlib-sync -- uvx --python 3.12 qlib-sync mcp` |
| OpenAI Codex CLI | `codex mcp add qlib-sync -- uvx --python 3.12 qlib-sync mcp` |
| Cursor | The JSON below, in `~/.cursor/mcp.json` |
| Gemini CLI | The JSON below, in `~/.gemini/settings.json` |
| Qwen Code | The JSON below, in `~/.qwen/settings.json` |
| Tencent CodeBuddy, ByteDance Trae, Alibaba Tongyi Lingma | MCP settings, add a server manually, paste the JSON below |
| Cherry Studio | Settings, MCP servers, add: type STDIO, command `uvx`, arguments `--python`, `3.12`, `qlib-sync`, `mcp` on separate lines |
| VS Code (Copilot agent mode) | `.vscode/mcp.json`: `{ "servers": { "qlib-sync": { "type": "stdio", "command": "uvx", "args": ["--python", "3.12", "qlib-sync", "mcp"] } } }` |
```json
{ "mcpServers": { "qlib-sync": { "command": "uvx", "args": ["--python", "3.12", "qlib-sync", "mcp"] } } }
```
Menu names and file locations change between versions; check your client's MCP documentation
if a path above has moved. Tested with Claude Code and OpenAI Codex CLI.
Web chat apps such as ChatGPT connect only to remote MCP servers over the internet, so they
cannot start this local server.
Set `QLIB_SYNC_DATA_DIR` to use a dataset outside `~/.qlib/qlib_data/cn_data`, and
`QLIB_SYNC_MIRROR` to download through a mirror.
| Tool | Does |
|---|---|
| `cn_status` | Local latest date, pools, and whether an update is available |
| `cn_update` | Install or update (safe to call: failures restore the previous data) |
| `cn_health_check` | Structure checks plus a real Qlib read |
| `cn_get_prices` | Daily bars for up to 20 symbols; real traded prices by default, adjusted on request |
| `cn_list_pool` | Members of csi300, csi500, and other pools on a date |
Then ask things like *"Is my A-share data current? What did Moutai close at over the last
week?"* Symbols accept `600519`, `SH600519`, or `600519.SH`.

[`skills/qlib-sync/SKILL.md`](skills/qlib-sync/SKILL.md) teaches an assistant when to update,
how to read each failure, and the price conventions. It uses the open `SKILL.md` format that
Claude, Codex, and other agents load from their skills folder; any assistant can also simply
be pointed at the file.
## Daily updates
A run that finds nothing new costs one small request, so schedule it daily. See
[docs/scheduling.md](docs/scheduling.md) for cron and launchd examples.
## Data and credits
All data comes from [chenditc/investment_data](https://github.com/chenditc/investment_data)
(Apache-2.0). qlib-sync only downloads the public release; it does not host or redistribute
data. Prices follow Qlib's convention: `$close` is adjusted and the real price is
`$close / $factor`.
qlib-sync is an independent project, not affiliated with Microsoft or the Qlib team.
Supported: macOS and Linux, Python 3.10 to 3.12 (pyqlib does not publish wheels for newer
versions yet; `uvx` picks a compatible Python automatically).
## Development
```bash
uv venv && uv pip install -e ".[dev]"
.venv/bin/python -m pytest
```
## License
MIT
TDQS
Scored across 5 tools
Each tool has a clearly distinct purpose: data update, price query, health validation, pool listing, and status reporting. There is no overlap or ambiguity between them.
All tools use a consistent 'cn_' prefix, but the second part mixes verb_noun (get_prices, list_pool) with noun phrases (health_check, status) and a bare verb (update). This is mostly consistent and still readable.
Five tools are well-scoped for an A-share data sync and access server. Each tool earns its place by covering a unique part of the data lifecycle.
The surface covers update, validation, status, price retrieval, and pool membership, which are the core operations. Minor gaps exist, such as querying a trading calendar or non-price data fields, but these are not critical for the stated purpose.