> **Absorbed by [Coase's Hoard](https://github.com/Luissalet/CoasesHoard).**
> This project's capabilities are maintained in the merged Hoard, with independent modules initially disabled.
> The code and instructions below remain as historical reference; use the linked repository for the current installation.

# Midas's Hoard

A local market-research lab for one person. It freezes market data into snapshots with provenance, keeps falsifiable investment theses with rules that invalidate them, runs an honest strategy backtester with a sealed holdout and overfitting checks, and audits every model-written number against the evidence. Assistants reach it over MCP; you reach it in the browser.

**Research, not trading.** There is no broker, no order, no buy or sell advice. Results are historical analyses, not advice; past performance does not guarantee future results.

## What it does

- **Market** - search symbols, fetch a series and freeze it as a snapshot (canonical CSV, sha256, provider, fetch time, currency, adjustment flag). Compare series with correlation and relative performance; mixed currency, adjustment or frequency is refused unless you allow it, and an exchange-rate conversion is recorded step by step. Charts are hand-drawn SVG.
- **Theses** - a claim, a forced rival hypothesis, a knowledge cutoff (`as_of`), a horizon, evidence for and against, and invalidation rules written in a small language with no `eval` (`close(AAPL) < 150`, `yoy(CPIAUCSL) > 4`, `drawdown(^GSPC) < -20`, `sma(x,50) < sma(x,200)`). A true rule means the thesis is invalidated. Evidence metrics never use data after the cutoff. Checks are idempotent and an invalidation emits one event.
- **Lab** - declarative strategy specs (JSON, no code). A signal on bar t only affects the position earning bar t+1 (a test proves it). Costs, sizing, rebalancing, long/short, benchmark, sealed holdout, anchored walk-forward with an optional parameter grid, block-permutation test, block-bootstrap Sharpe interval, Bonferroni haircut and deflated Sharpe. Every variant is logged, failures included, with a multiple-testing warning.
- **Committee** - two advocates, an independent risk reviewer and an arbiter over a fixed evidence pack. Citations are validated and every number goes through the number ledger (observed, derived, cited, proposed or unmatched). Without a model it returns the evidence pack and deterministic metrics.
- **Portfolio** - holdings from a table or CSV, valuation at a date, allocation, currency exposure, concentration, volatility, drawdown, correlation, with every FX conversion recorded.
- **Reports** - Markdown or JSON for a thesis, a run or a portfolio, with provenance for every number. A run's report follows the lab report contract (below) and also exports as HTML.
- **Data health** - nulls, zeros, coverage by date, 3-sigma outliers and long gaps per series, computed before any metric is read.
- **Duplicates** - the same idea under another name is found by mechanism, not only by spec hash, and counts as a trial.

## Lab report contract

The report of a backtest (`report_export` with `kind: "run"`, `GET /api/lab/backtests/{run_id}/report`, the Lab screen) always has the same eight sections, in this order:

1. **Identity and scope** - strategy, universe, window, which part is in-sample and which out-of-sample (sealed or opened holdout; walk-forward windows), and a rule-based evidence-quality label.
2. **Summary**.
3. **Health gates** - data health, sample size, permutation, walk-forward, bootstrap, multiple testing, holdout and duplicate risk, each `pass`, `warn`, `fail` or `not_run`.
4. **Metrics** - one table per window (development, walk-forward out-of-sample, opened holdout); windows are never mixed in a table.
5. **Mechanism with alternatives**.
6. **Risks and evidence gaps**.
7. **Next experiments**, prioritised.
8. **Decision**: `reject`, `investigate`, `review` or `adopt`, with the reasons.

Every statement carries a tag: **Observed** (read from, or computed on, the stored artifacts), **Inference**, **Alternative** or **Experiment**. In JSON each claim has `kind`; in Markdown the tag is bold (`**[Observed]**`); in HTML and in the UI it is a coloured label. The committee's arguments, risks and reasons carry `kind: "inference"`.

The decision is computed by fixed rules from the validations that exist (walk-forward windows that were positive, permutation p-value, bootstrap interval, multiple-testing correction, holdout), the data health and the variants tried, including near-duplicates under other names. A model never writes or changes it. Missing evidence only lowers it: without validations, with a failing data-health gate or coverage under 90 %, with too few bars or trades, or with a window under a year, the most it can be is `investigate`; `adopt` also needs an opened holdout that held up and no high duplicate risk. A negative net Sharpe over at least 120 bars, a walk-forward with no positive window, or a failed holdout gives `reject`. The thresholds are returned with the decision.

Rules against invented metrics, enforced in code: a null stays null and reads `no data` (never 0); each table states its window; there is no composite score (the evidence-quality grade labels how much was tested and is not a verdict); the report states its conventions: drawdown is a negative fraction of the previous peak, and turnover is the sum of |change in weight| per year, not halved.

### Data health

`data_health` (tool, `GET /api/lab/data-health`) measures, per series and window: share of null price cells, share of exact zeros, coverage (valid dates over expected dates), returns beyond 3 standard deviations and long gaps. It runs on the raw frozen series before any metric, is stored with each run and shows as a gate in the report. The thresholds used travel with the result. A missing close is never filled; the backtest warns where it had to treat one as no move.

### Duplicates

At registration each experiment is compared with those of other families: Jaccard overlap of the indicator kinds, of the rule leaves (indicators compared by kind, so renaming `fast` to `ma_short` changes nothing) and of the universe symbols, plus the similarity of name and note (embeddings through Hoard Link when available, otherwise TF-IDF). `duplicate_risk` is `low`, `medium` or `high` with the similar experiments and why. It never blocks. Variants of another family at high mechanism overlap count in `effective_variants`, which drives the warning level and the multiple-testing correction (`experiments_duplicates`, `GET /api/lab/duplicates`).

## Data providers

| Provider | Data | Notes |
|---|---|---|
| Yahoo Finance (default) | daily prices for equities, ETFs, indices, FX and crypto: `AAPL`, `^GSPC`, `^IBEX`, `SAN.MC`, `EURUSD=X`, `BTC-EUR` | no key; unofficial public chart endpoint, personal and research use only; closes adjusted for splits and dividends, events recorded; may change or throttle without notice |
| FRED | macro series | Source: Federal Reserve Bank of St. Louis; values are as last revised, not vintages |
| European Central Bank | euro reference rates and statistics (SDMX) | Source: ECB Data Portal |
| CoinGecko | crypto prices, up to 365 days on the free tier | data provided by CoinGecko; throttled |
| Alpha Vantage | daily prices | optional, off until you add a free key (`MIDAS_ALPHAVANTAGE_KEY` or Settings); raw bars, small daily allowance |
| Tiingo | daily adjusted prices, mostly US | optional, off until you add a free key (`MIDAS_TIINGO_KEY` or Settings) |
| Stooq | end-of-day prices | currently blocked: it answers scripts with a JavaScript browser check, which Midas does not try to get around; fetches fail with `provider_unavailable` and the source is marked blocked |
| CSV | your own files | you declare currency or unit; nothing is guessed |
| fake | deterministic synthetic series | for tests, demos and offline use |

API keys entered in Settings are write-only: they are stored locally and the API only reports `configured` and the last four characters. Providers show as `ok`, `needs_key` or `blocked` in the Sources page and in `market_providers`.

Responses are cached on disk by the shared Hoard Link JSON API client (identifying User-Agent, one retry on network errors and server errors, a stale copy when the provider is down, `Retry-After` on rate limits, public addresses only); `MIDAS_OFFLINE=1` (or the Settings switch) answers only from the cache. Read each provider's terms, shown with every snapshot.

## Run it

```
pip install -r requirements.txt
python -m midas_hoard            # http://127.0.0.1:5192
```

The client is built and committed under `midas_hoard/static`; rebuild it with `npm install && npm run build`. `python scripts/launch.py` picks a free port and opens the browser; `python scripts/dev.py` runs the API with reload plus the Vite dev server.

Settings come from the environment: `MIDAS_DATA_DIR` (default `./data`), `MIDAS_PORT` (5192), `PORT_STRICT`, `MIDAS_ALLOWED_HOSTS`, `MIDAS_HTTP_TIMEOUT_S`, `MIDAS_CACHE_TTL_S`, `MIDAS_OFFLINE`. Data lives in `data/midas.db` (SQLite, WAL) plus `data/snapshots`, `data/runs` and `data/reports`; files are written atomically. The launcher, request guard, error envelope, stable `mcp-token`, SQLite layer, agent catalogue (20 KB result cap) and MCP bridge are the shared commons of Hoard Link (`midas_hoard/hoard_link/`); a second `python -m midas_hoard` on the same port exits instead of starting another copy.

## Faustus assistant

Once the shared Hoard Link launcher is installed, the app shows the family's Faustus panel (a floating button). The conversation, the model and the tools live in Faustus; this app only tells the panel what is open.

- **Context per view.** The panel receives the objects open in the view, with real ids read through the app's own API: `thesis` (title, status, as-of date, reference), `portfolio` (name, currency, number of holdings), `experiment` (run id, strategy or family), `strategy` and `snapshot` (symbol, provider; the snapshots selected in Market). The view also gives the page and tab. Creation views (a new thesis, portfolio or strategy) report `new` and give no object. No holdings, evidence text or keys are sent.
- **Unsaved fields.** The panel asks for them with the `midas:context-request` window event. The forms `thesis-form`, `evidence-form`, `portfolio-form`, `strategy-form` (spec text cut to 20,000 characters when it does not parse) and `snapshot-fetch-form` answer with their typed fields and `dirty`.
- **Refresh.** When the assistant finishes a turn the open view reloads in place (`hoard:copilot-turn-end` / `window.hoardCopilot.onTurnEnd`) without remounting or resetting forms.
- **Saving.** Forms send only what changed. New: `PATCH /api/portfolios/{name}` and `portfolio_set` with `action: "patch"` (`upsert` holdings matched by symbol, `remove` symbols, `currency`; one transaction); `PATCH /api/lab/strategies/{id}` and `strategy_save` with `id` + `spec_patch` (objects merge, arrays replace, `null` removes a key; it cannot rename). Portfolios with repeated symbols are still replaced whole. See `docs/API.md`.
- **Tests.** `node --test client/copilot-context.test.mjs` and `python -m pytest -q tests/test_partial_update.py tests/test_manifest.py`.

## Assistant tools (MCP)

`mcp_server.py` is the shared Hoard Link stdio bridge. It never opens the database: it proxies every call to `POST /api/agent/call` with the token in `data/mcp-token`, reads the tool list from `GET /api/agent/tools`, and starts the app if nothing answers (`MIDAS_BRIDGE_AUTOSTART=0` disables that). Variables: `MIDAS_URL`, `MIDAS_TOKEN_FILE`, `MIDAS_DATA_DIR`. Errors keep their `code` and `hint`. Deletes need `confirm: true` on the owning tool. The 29 tools:

### Market data

| Tool | What it does |
|---|---|
| `midas_status` | Health: providers, data counts, models, disk |
| `market_providers` | List data providers with terms, delay and licence |
| `market_search` | Search symbols (indices, FX, stocks, macro series, crypto) |
| `market_fetch` | Fetch a series and freeze it as a provenance-tagged snapshot |
| `market_series` | View a snapshot: points, returns, volatility, drawdown, resample, rebase |
| `market_compare` | Compare 2-12 snapshots: correlation, relative performance, aligned dates |
| `snapshots_list` | List stored snapshots with provenance |

### Theses

| Tool | What it does |
|---|---|
| `thesis_create` | Create an investment thesis with a rival hypothesis and rules |
| `thesis_get` | Get a thesis with evidence, rules and last check state |
| `thesis_list` | List theses by status or text |
| `thesis_update` | Edit a thesis (claim, rules, status...) or delete it (action=delete, confirm=true) |
| `thesis_evidence_add` | Add evidence for/against a thesis (metric, url, note, hoard:// ref) or delete one |
| `thesis_check` | Check a thesis's invalidation rules on fresh data; may set it invalidated |
| `committee_run` | Run the committee on a thesis: advocates, risk reviewer, arbiter, number ledger |

### Strategy lab

| Tool | What it does |
|---|---|
| `strategy_validate` | Validate a strategy spec (JSON, no code) with fixable errors |
| `strategy_save` | Save a validated strategy by name, or delete one (action=delete, confirm=true) |
| `strategies_list` | List saved strategies with the variants tried, or fetch one spec |
| `backtest_run` | Backtest a strategy: metrics, equity, trades, benchmark, variants tried |
| `backtest_validate` | Stress a backtest: walk-forward, permutation test, bootstrap Sharpe CI, multiple testing |
| `experiments_list` | List every backtest variant ever run, failures included, or one run in detail |
| `data_health` | Data health of series: % nulls, % zeros, coverage by date, 3-sigma outliers, long gaps |
| `experiments_duplicates` | Find the same idea under another name: shared indicators, rules, universe, description |

### Portfolio and reports

| Tool | What it does |
|---|---|
| `portfolio_set` | Set a portfolio's holdings (list or CSV) or delete it (action=delete, confirm=true) |
| `portfolio_analyze` | Value and analyse a portfolio: allocation, currency exposure, vol, drawdown, correlation |
| `report_export` | Export a thesis, run or portfolio as Markdown/JSON with provenance for every number |

Full reference: [docs/API.md](docs/API.md).

## Method and limits

- Snapshots are immutable and verified by hash on load; identical bytes reuse the snapshot.
- `as_of` makes analyses point-in-time for the data you hold. FRED and some other providers publish revised values, so revisions can still leak into past dates; this is stated in each snapshot.
- Variant counts are per strategy family (distinct spec hashes) plus the variants of other families that are the same idea (high mechanism overlap). The deflated Sharpe needs at least 3 variants.
- Backtests are daily-bar simulations with proportional costs: no intraday fills, no liquidity model, no taxes, no survivorship correction for delisted names.
- The committee can only be as good as the evidence you give it; unmatched numbers are flagged, not removed.
- CoinGecko history is limited to 365 days on the free tier.

## Tests

`python -m pytest -q` (offline: a fake provider, mocked HTTP transports and a scripted model). CI runs on Ubuntu and Windows.

## Licence

MIT (c) Luis María Salete Cuartero. Data remains subject to each provider's terms.

## Saved watchlists / Listas de seguimiento

Market now keeps named sets of pinned snapshots. Create a list from the selected series, rename it, add selections or remove individual entries without deleting snapshot files. Open each series in the existing chart. Filter by symbol/currency, percentage change and drawdown, with dated windows and source SHA. The date cutoff affects observations; it does not reconstruct provider vintages. Refresh fetches new snapshots per asset, keeps imported CSV data, reports partial failures and preserves concurrent edits.

Four native tools use the same API: `watchlists_list`, `watchlist_save`, `watchlist_view`, `watchlist_refresh`. The assistant receives the saved list identity and unsaved watchlist editor fields. No additional model or external application is required. [API and verified behavior](docs/WATCHLISTS.md).