# Sophotron

> *sopho-* (σοφία, _wisdom_) + *-tron* (_instrument_) — a wisdom-instrument for philosophy research.

**Sophotron** is a tool for searching and retrieving **philosophy papers**, built for AI research assistants that need to ground themselves in the actual philosophical literature rather than the shallow, mis-classified philosophy slice of general academic databases. It is exposed today as an [MCP](https://modelcontextprotocol.io) server; a CLI and other frontends can sit on the same interface-agnostic core.

It combines four open sources, each doing what it does best:

| Source | Role |
|---|---|
| **Semantic Scholar** | Search, metadata, citations, references, SPECTER-based related papers (the backbone) |
| **OpenAlex** | Open-access discovery — agent-fetchable PDFs, ~11.5M OA philosophy works |
| **Unpaywall** | DOI → best open-access PDF location |
| **CrossRef** | Authoritative DOI metadata fallback (`get_paper` for DOIs Semantic Scholar lacks) |
| **PhilArchive** | Philosophy-specific open-access cache (metadata + best-effort PDF links) |
| **PhilPapers taxonomy** | Browse 6,134 expert-curated philosophy categories (cached snapshot) |

**Runs fully keyless.** An optional Semantic Scholar API key raises rate limits; optional contact emails opt into the OpenAlex/Unpaywall "polite pool." No source requires registration.

> **Why four sources?** Philosophy is under-served by general infrastructure: low open-access (~15%) and DOI-assignment (~60%) rates, canonical works predating DOIs, and systematic mis-classification (Rawls's *A Theory of Justice* tagged "Sociology" in Semantic Scholar; Heidegger's *Being and Time* with a null field). And as of 2026, the PhilPapers/PhilArchive **web** APIs sit behind Cloudflare bot-protection. OpenAlex + Unpaywall therefore supply the open-access PDFs an automated client can actually fetch, while a cached PhilArchive harvest adds philosophy-specific coverage.

## Install

```bash
npm install
npm run build
npm run smoke   # optional: MCP stdio smoke test
```

Requires Node.js ≥ 22.

## Configure

All configuration is optional — copy [`.env.example`](.env.example) to `.env` and fill in what you want:

```bash
# Optional: raises Semantic Scholar rate limits (request at
# https://www.semanticscholar.org/product/api). Without it, S2 still works but
# rate-limits aggressively (HTTP 429) on its shared unauthenticated pool.
SEMANTIC_SCHOLAR_API_KEY=

# Optional: OpenAlex/Unpaywall "polite pool" contact (a courtesy id, not a key).
# OPENALEX_EMAIL=you@example.com
# UNPAYWALL_EMAIL=you@example.com
```

Advanced overrides (`SOPHOTRON_OAI_ENDPOINT`, `SOPHOTRON_DATA_DIR`, `SOPHOTRON_LIBRARY_PATH`) are documented in `.env.example`.

## Use with an MCP client

**Claude Code / any stdio MCP client** (`~/.claude/settings.json`):

```json
{
  "mcpServers": {
    "sophotron": {
      "command": "node",
      "args": ["/absolute/path/to/sophotron/dist/index.js"],
      "env": { "SEMANTIC_SCHOLAR_API_KEY": "optional" }
    }
  }
}
```

**Claude Desktop / Cowork (MCPB bundle):** a [`manifest.json`](manifest.json) is included. Pack it with the MCPB CLI and install the resulting `.mcpb`:

```bash
npx @anthropic-ai/mcpb pack
```

## Tools

| Tool | Description |
|------|-------------|
| `search_papers` | Search via Semantic Scholar (auto-falls back to OpenAlex when S2 is rate-limited); PhilArchive PDF enrichment. Optional `category_id` reranks results relative to an inferred taxonomy region |
| `search_openalex` | Search via OpenAlex with open-access PDF discovery (`philosophy_only`, `open_access_only` filters) |
| `get_paper` | Paper details by S2 ID, DOI, arXiv ID, or PhilArchive code |
| `get_citations` | Papers citing a paper |
| `get_references` | A paper's bibliography |
| `get_related` | SPECTER-based related papers. Optional `category_id` reranks results relative to an inferred taxonomy region |
| `get_pdf` | Resolve an open-access PDF via the fallback chain (below) |
| `search_philarchive` | Search the local PhilArchive open-access cache |
| `harvest_philarchive` | Incrementally harvest the PhilPapers/PhilArchive OAI-PMH feed into the cache |
| `get_cache_stats` | PhilArchive cache + cross-reference statistics |
| `browse_taxonomy` | Browse the expert-curated PhilPapers category taxonomy (6,134 categories) |
| `category_papers` | Papers near an inferred taxonomy-category region, ranked by influence — a semantic lens over an inferred mapping, not an authoritative index |
| `download_paper` | Resolve an open-access PDF, extract its text, and store it as markdown (arxiv-mcp-server-compatible path) |
| `read_paper` | Read back the stored text of a downloaded paper, paginated |
| `library_add` / `library_remove` / `library_list` | A personal paper library (JSON-on-disk) with tags and notes |
| `recommend` | "More like these," seeded by your library, via Semantic Scholar's hosted Recommendations API (SPECTER2) |

### Taxonomy navigation (the discourse-cartographer layer)

The PhilPapers taxonomy is more than a list to browse: each category is treated as an **inferred
SPECTER2 region** (a coordinate in embedding space, seeded from the category name), so you can
navigate papers *relative to a category*. Pass a `category_id` (the id `browse_taxonomy` returns)
to `search_papers` or `get_related` to rerank the results by proximity to that region, or call
`category_papers` for influence-ranked papers near it.

This is a **semantic lens over an inferred mapping, not a classification claim** — results are
tagged with relative proximity (nearer / farther / unplaceable), `placement: "inferred"`, and an
honest confidence tier (`distinct` for semantically-clear coarse categories, `low` for fine,
catch-all, or thinly-seeded ones). Transient Semantic Scholar outages are reported as
`unavailable`, never as a confident empty region.

### PDF resolution (`get_pdf`)

Tries sources in order of reliability and stops at the first hit, reporting where the PDF came from and whether an automated client can fetch it:

1. **Semantic Scholar** `openAccessPdf` → `agentFetchable: true`
2. **Unpaywall** by DOI → `agentFetchable: true`
3. **OpenAlex** by DOI (or title search) → `agentFetchable: true`
4. **PhilArchive** best-effort URL → `agentFetchable: false` (philarchive.org is Cloudflare-walled to automated clients; open these in a browser)

```jsonc
// get_pdf { "paper_id": "10.1007/s11019-014-9560-2" }
{ "pdfUrl": "https://…/paper.pdf", "source": "unpaywall", "agentFetchable": true, "oaStatus": "green", … }
```

### Cross-referencing (PhilArchive ↔ Semantic Scholar)

PhilArchive's OAI-PMH metadata omits DOIs, so papers are linked by **fuzzy title + author matching** (title 50% / author 35% / year 15%; accept > 0.85). Match results carry a confidence score so uncertain links can be reviewed.

## Rate limits & access reality

| Source | Limit | Handling |
|--------|-------|----------|
| Semantic Scholar | **429s even at slow pace** without a key — the unauthenticated pool is globally shared (empirically 7/8 requests 429'd at 5s spacing) | bounded retry/backoff (`src/utils/http.ts`) **+ automatic OpenAlex fallback** in `search_papers`; a key improves S2-native ranking/citations |
| OpenAlex / Unpaywall | generous, no key | polite-pool email recommended |
| PhilArchive OAI (`philpapers.org/oai.pl`) | 2 req/sec | built-in 500ms throttle |
| `philarchive.org` (PDFs, web) | **Cloudflare 403 to agents** | PDF URLs returned as best-effort, browser-only |

## Project structure

```
src/
├── index.ts                  # MCP server frontend (18 tools)
├── types.ts                  # shared types
├── adapters/                 # one module per external API; never throw to callers
│   ├── semantic-scholar.ts   # S2 Graph/Recommendations API (+ SPECTER2 embeddings)
│   ├── openalex.ts           # OpenAlex works + OA discovery
│   ├── unpaywall.ts          # DOI → OA PDF
│   ├── crossref-api.ts       # api.crossref.org DOI metadata fallback
│   ├── philarchive.ts        # PhilArchive/PhilPapers OAI-PMH + cache
│   └── taxonomy.ts           # PhilPapers category hierarchy (cached)
├── services/                 # cross-source logic
│   ├── crossref.ts           # S2 ↔ PhilArchive fuzzy cross-reference
│   ├── pdf-resolver.ts       # S2 → Unpaywall → OpenAlex → PhilArchive chain
│   ├── paper-download.ts     # download_paper / read_paper full-text store
│   ├── library.ts            # personal library (JSON-on-disk)
│   └── taxonomy-bridge.ts    # taxonomy→papers semantic bridge (inferred centroids)
└── utils/
    ├── fuzzy-match.ts         # title/author matching
    ├── vector.ts             # cosine / centroid / proximity ranking
    ├── rate-limit.ts         # serial rate limiter (S2 1 req/s)
    └── http.ts               # shared retry/backoff fetch

data/                         # philarchive-cache.json, crossref-cache.json, category-centroids.json (regenerable)
scripts/smoke-test.mjs        # MCP stdio smoke test (also runs in CI)
```

## Develop

```bash
npm run build       # tsc (tsconfig.build.json — excludes tests/experiments)
npm run typecheck   # tsc --noEmit (full, incl. tests)
npm test            # vitest (set SKIP_INTEGRATION=true to skip live tests)
npm run smoke       # MCP stdio smoke test
```

Start with [`CONTRIBUTING.md`](CONTRIBUTING.md); the operational guide (architecture, conventions, correctness contracts) is [`CLAUDE.md`](CLAUDE.md). Tests are the verification spine — every adapter has unit tests with mocked `fetch`, plus a network-gated integration suite.

## License

[MIT](LICENSE)