Scholar Engine MCP 🔬⚡

High-speed Scientific Literature Semantic Compiler & FastMCP Server
Powered by arXiv, OpenAlex, DuckDB HTTP range queries, and TypeSafe Jev System One.

💡 Why Scholar Engine?

Traditional Retrieval-Augmented Generation (RAG) relies on dense vector similarity (cos(query, chunk)). Vector similarity is great for answering "what text looks like my query?", but fundamentally fails on structural, empirical scientific questions, such as:

"Which published paper experimentally proved loaded latency reduction on real fixed wireless hardware without requiring Wi-Fi 6 PHY?"

Vector search will return dozens of papers full of simulation equations or theoretical surveys that merely mention "wireless" and "latency" thousands of times.

Scholar Engine rethinks scientific discovery for AI agents by combining:

1. Zero Storage Bloat: Queries Hugging Face's 3.15M arxiv-complete Parquet dataset directly using DuckDB HTTP range queries. No need to download a 16 TB PDF corpus.

2. Citation Graph Enrichment: Instantly pulls citation metrics and author graphs via OpenAlex.

3. Probabilistic Semantic Gates (Jev System One): Natural language questions compile into persistent, 154ms probabilistic predicates (e.g. P_real_hardware > 0.85, P_empirical > 0.75).

4. Materialized Predicate Cache: Evaluated predicates are cached in SQLite bitmaps so subsequent runs reuse past judgments with zero inference cost.

5. Double Jev Gate (Pre-RAG & Post-RAG):

   • Pre-RAG: Discards non-empirical or irrelevant papers before reading full text.

   • Post-RAG: Verifies every claim synthesized by the reasoning agent against extracted evidence passages (supported, partial, unsupported, contradicted).

Related MCP server: paper-mcp

📊 Comparison Matrix

🏛️ Architecture

flowchart TD
    subgraph INGESTION["1. Zero-Storage Discovery"]
        ARXIV["arXiv Atom API (Search & Metadata)"]
        OPENALEX["OpenAlex Graph API (Citations & Authors)"]
        HF["Hugging Face arxiv-complete (3.15M Papers)"]
        DUCKDB["DuckDB HTTP Range Scanner (Targeted LaTeX fetch)"]
        HF --> DUCKDB
    end

    subgraph ENGINE["2. Scholar Semantic Engine"]
        DISCOVER["Candidate Retrieval Engine"]
        ARXIV --> DISCOVER
        OPENALEX --> DISCOVER
        DUCKDB --> DISCOVER

        subgraph GATES["Jev System One Gates"]
            PRE_RAG["Pre-RAG Gate (P_empirical, P_applicable)"]
            POST_RAG["Post-RAG Verifier (Claim vs Evidence)"]
            PRED_CACHE[("SQLite Predicate Cache & Bitmaps")]
        end

        DISCOVER --> PRE_RAG
        PRE_RAG <--> PRED_CACHE
        PRE_RAG --> POST_RAG
        POST_RAG <--> PRED_CACHE
    end

    subgraph MCP_INTERFACE["3. FastMCP Server Interface"]
        TOOL_SEARCH["scholar_search (Fast paper discovery)"]
        TOOL_INSPECT["scholar_inspect (Deep paper metrics & OpenAlex)"]
        TOOL_RESEARCH["scholar_research (Full autonomous loop)"]
    end

    ENGINE --> MCP_INTERFACE
    MCP_INTERFACE --> CLIENTS["AI Agents: Claude Desktop, Cursor, Codex, Antigravity"]

🚀 Quick Start

1. Installation

# Clone the repository
git clone https://github.com/wolverin0/scholar-mcp.git
cd scholar-mcp

# Install dependencies (or install in a virtual environment)
pip install -e .

2. Configuration (.env)

Copy .env.example to .env:

cp .env.example .env

# Optional: TypeSafe Jev API Key for live 154ms System One probabilistic decisions
# If unset, Scholar Engine automatically runs in high-fidelity deterministic simulation mode!
TYPESAFE_API_KEY=your_typesafe_key_here

3. Run Tests

Verify everything is working locally:

pytest -v

🤖 MCP Server Setup

Add Scholar Engine to your favorite agentic tools:

Claude Desktop (claude_desktop_config.json)

{
  "mcpServers": {
    "scholar": {
      "command": "python",
      "args": ["-m", "scholar.mcp.server"],
      "env": {
        "TYPESAFE_API_KEY": "your_key_here"
      }
    }
  }
}

Cursor (.cursor/mcp.json)

{
  "mcpServers": {
    "scholar": {
      "command": "python",
      "args": ["-m", "scholar.mcp.server"]
    }
  }
}

Antigravity / Wezbridge (.mcp.json)

{
  "mcpServers": {
    "scholar": {
      "type": "stdio",
      "command": "python",
      "args": ["-m", "scholar.mcp.server"]
    }
  }
}

🛠️ FastMCP Tools Reference

1. scholar_search

Quickly search arXiv for scientific papers by topic, keyword, or title.

• Arguments:

  • query (str): Search topic or keywords (e.g. "loaded latency wifi").

  • limit (int, default=5): Number of candidates to return.

• Returns: Structured JSON list of IDs, titles, categories, abstracts, and PDF links.

2. scholar_inspect

Inspect a paper in detail by its arXiv ID. Merges OpenAlex citation metrics, authors, and stored semantic predicates.

• Arguments:

  • paper_id (str): arXiv paper ID (e.g. "2007.07174" or "2306.04338").

• Returns: Full abstract, categories, citation counts, DOI, OpenAlex ID, and cached semantic features.

3. scholar_research

Autonomous scientific discovery loop:

1. Discovers candidates on arXiv.

2. Runs Jev System One semantic gatekeeper to filter out non-empirical or incompatible papers.

3. Resolves citation graphs and verifies claims against evidence text.

• Arguments:

  • query (str): Research question or engineering topic.

  • domain (str, default="general"): Domain hint (e.g. "wireless", "databases", "ai").

  • threshold (float, default=0.50): Minimum probability score required to pass semantic gates.

  • limit (int, default=10): Max candidates to evaluate.

• Returns: Surviving evidence-backed papers with confidence metrics and screened-out audit breakdown.

💻 CLI Usage

You can also use Scholar Engine directly from your terminal:

# Search arXiv papers
scholar search "neural network verification" --limit 5

# Inspect a paper with OpenAlex citation metrics
scholar inspect "1711.00455"

# Run the autonomous semantic research loop
scholar research "wireless loaded latency scheduler" --threshold 0.50 --limit 10

🤝 Contributing & Community Roadmap

We built Scholar Engine to bring rigorous scientific grounding to AI coding and research agents. We welcome contributions to make it even more capable!

• Domain Predicate Packs: Community-curated semantic predicates for specific fields (Biomedical, Systems/Networking, Cryptography, Robotics).

• Local Small-Model Distillation: Distill common predicate evaluations into local open-weights SLMs (e.g., local 0.5B - 3B models) for 100% offline edge execution.

• Roaring Bitmaps on DuckDB: Materialize precomputed semantic indices across all 3.15M arXiv papers into compact Parquet bitmap columns.

• PubMed & bioRxiv Source Adapters: Expand beyond arXiv into life sciences and medical literature.

📄 License

MIT License. See LICENSE for details.