ama-mcp
Runs each pipeline role as a fresh agent session through the user's own OpenAI Codex (codex) CLI login, allowing the agentic multiverse analysis to be backed by OpenAI.
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
agentic_multiverse_analysis
Agentic multiverse analysis of an empirical paper, as an MCP server (ama-mcp) and a command line (ama).
ama-mcp runs an agentic multiverse analysis (AMA) of one published empirical paper and
reports how much of the paper's defensible specification space keeps its main conclusion. It
is the pipeline used for the two accompanying papers (see Citation), packaged as an
MCP server (ama-mcp) and a command line (ama). Every role in the pipeline runs as a fresh
agent session through your own Claude Code (claude) or OpenAI Codex (codex) CLI login.
Safety first. The agent sessions run code from replication packages, install packages, download data and browse the web, with permission prompts turned off. Run
ama-mcpinside a container or a virtual machine that holds only what the analysis needs (the paper, the data you are licensed to use, the agent CLI login). Do not run it on a machine whose files or credentials you would not hand to an unattended agent. See Safety.
What AMA does
You give it a paper. It then:
Finds the authors' replication package (journal archives, data repositories, author pages). If there is no complete package, an agent rebuilds the data from the original sources you can access, following the paper's own description, and an independent reviewer checks every sample rule and reported sample size against the paper.
Pins down the main estimate. One agent writes a configuration naming the paper's headline coefficient, its table location and the claimed sign; another agent checks it against the paper.
Reproduces the main estimate with its own code. A gate script compares the result with the published number (within 10% by default). On a miss, an independent auditor looks for implementation errors before the run may continue.
Lists the analysis choices. One agent lists every choice the analysis makes (sample, variable definitions, controls, fixed effects, estimator, inference, ...) and the defensible alternatives. A different agent judges each option against a fixed rubric and designs a grid of 100 to 250 specifications, and fixes the inference method, before any alternative estimate exists.
Runs the grid. An implementer writes and runs the grid; an independent auditor reads the code line by line and recomputes cells; fix rounds continue until the auditor finds no blocking problem.
Measures what the auditor's exclusions cost the paper (a "companion" grid with the paper's own choices restored).
Reports. The papers' measures are computed by scripts; an agent collects what the authors say about the decisive choices, and a reporter writes a report whose every number is regenerated by a script and checked byte for byte.
Three rules make the output worth reading: no agent checks its own work; the space of choices
is judged before any estimate exists; every step ends at a gate decided by a program or by an
independent agent, never by the agent whose work is gated. Gate decisions are appended to
gates.log, and artifacts are hashed when a gate passes so a later edit is detected.
The prompts, gate scripts and orchestration contract in src/ama_mcp/core/ are the ones the
study ran, changed only where a public user's machine needs it. Every change is listed, with
its reason, in docs/CHANGES_FROM_STUDY.md.
Related MCP server: ai-architect
Requirements
Linux or macOS, Python 3.10 or newer.
At least one agent CLI, installed and logged in:
Claude Code:
claudeon yourPATH; log in once interactively (claude, then/login) or set an API key as Claude Code documents.OpenAI Codex CLI:
codexon yourPATH;codex login.
A Python environment for the agents' analysis code. The agents write Python by default; a typical environment has
pandas,numpy,statsmodels,linearmodels,pyfixest,wildboottest,pyreadstat,pyarrowandmatplotlib. Pass it with--analysis-python. Stata or R are used only if a replication package needs them and they are installed.Access to the data the paper uses, unless the authors' package ships it (see Data access).
Install
python -m venv .venv
.venv/bin/pip install . # from a clone of this repository
.venv/bin/ama doctor # checks the core files, Python packages and agent CLIsama doctor reports whether each CLI is installed and logged in, whether the runs directory
is writable, and whether it is running inside a container.
Use it from Claude Code or Codex (MCP)
Register the server once. Use the absolute path of the ama-mcp executable in your
environment if it is not on your PATH (ama-mcp --help prints both registrations with that
path filled in).
Claude Code
claude mcp add --scope user ama -e AMA_RUNS_ROOT="$HOME/ama-runs" -- /path/to/.venv/bin/ama-mcpCodex (~/.codex/config.toml)
[mcp_servers.ama]
command = "/path/to/.venv/bin/ama-mcp"
env = { AMA_RUNS_ROOT = "/path/to/ama-runs" }Then ask your agent, for example: "Use the ama tools to run an agentic multiverse analysis of
~/papers/paper.pdf with the claude backend." The server also ships an MCP prompt,
run_analysis, that walks an agent through the steps.
The backend that runs the analysis is independent of the client you talk to: a Codex session
can start a run on the claude backend and the other way round.
Tool | What it does |
| Is |
| Creates a run directory and starts the analysis in the background; returns a |
| Status, current stage, gate outcomes so far, the headline once it exists, the last log lines |
|
|
| All runs, newest first |
| Stop a run; continue it from where it stopped |
| Read a text file of a run (the held-out |
Resources: the pipeline documents (ama://docs/...) and the 18 role prompt templates
(ama://prompts/...).
Use it from the command line
ama run paper.pdf --backend claude \
--citation "Author and Author (2015), Title, Journal" \
--data-sources my_data_sources.md --analysis-python /path/to/analysis-env/bin/python
ama status # all runs
ama status <run_id> # one run: stage, gates, headline, last log lines
ama results <run_id> # headline, strict survival, measures (--all, --definitions), report excerpt
ama stop <run_id>
ama resume <run_id> # continue after a stop, an error or a pause
ama doctorama run starts the analysis in the background and returns; add --foreground to keep it
attached to the terminal. Runs live in $AMA_RUNS_ROOT (default ~/ama-runs). The
orchestrator can also be driven directly: python -m ama_mcp.orchestrator run --run-dir RUN.
Inputs
Input | Required | What it is for |
Paper PDF | yes | Copied into the run's |
| no | Authors, year, title, journal: helps the agents identify the paper |
| no | The authors' replication package (directory or archive) if you already have it; otherwise the package finder searches for it |
| no, but important | A file describing the data you can access; start from the template ( |
| no | Directories with data the agents may read (passed to the agent CLI) |
| no | The paper's internet appendix or other supplementary files |
| no | JSON listing the outcomes eligible as the headline ( |
| no | Sources no agent may read, e.g. a published comment on or replication of this paper, so the assessment stays independent of it |
| no | A third-party source that only the data and reproduction roles may use for data details (e.g. law adoption dates); the other roles are told not to read it |
| no | Reproduction-gate tolerance; the study used 0.10 |
| no | Parallel worker processes and total memory for the specification grid |
| no | Override the model of one step ( |
| no, but important for | Reasoning effort of every agent session: |
| no | Pause once the audited choice space exists (or after any stage), e.g. to inspect it before the grid runs |
Data access
The analysis is only as good as the data the agents can reach. If the authors' package ships data and code, the data are used as shipped. If not, the builder rebuilds the data from the original sources you can access: describe them in the data sources file (databases and how to connect, local copies and their paths, what is not available). If the main outcome cannot be built from what is available, the run stops at the package gate as infeasible; that is a recorded result, not a crash.
Respect your data licenses. Agents copy extracts of the data into the run directory
(package/, repro/, grid outputs), so a run directory can contain licensed data: do not
publish or share it.
Time and cost
In the study, the median paper took about 22 hours from the first agent session to the last and used about 126 million model tokens, most of them cache reads.
Outputs
Everything about a run is in its directory:
File | What |
| The paper and the replication package (found or rebuilt) |
| The main estimate and the independent review |
| The reproduction and, after a miss, the reproduction audit |
| The analysis choices and their audit; the designated grid and inference |
| The grid and its audit |
| The companion grid |
| The headline measures (share supported first), written before the report |
| Strict survival and kill power ( |
| The report |
| Every gate decision; artifact hashes |
| The run record, the strict survival headlines and every per-paper measure |
| What ran, what each agent was told, and its transcript |
A run can end before the report: a gate may decide that the package cannot be built, that the
main estimate cannot be estimated, that a reproduction gap stays unexplained, or that the
implementation audit does not close. run_status and ama_result.json say where and why.
How the results map to the papers' measures
A run reports two sets of numbers.
The headline: the per-paper measures of An AI-Powered Multiverse Analysis of Empirical Finance Research (computed by
ama_mcp.metrics; each value comes with its definition). The orchestrator writes them topaper_measures.jsonafter the grid is scored and before the report, so the report leads with them;ama statusshows them on one line, andama resultsandrun_resultslist them first. They are the share supported (published sign and two-sided p < 0.10 under the designated inference; failed and distinct-estimand specifications left out), the median signed z, the share with the published sign, and whether the main estimate is reproduced (published sign, p ≤ 0.10 under the paper's own inference, and within 25% of the published estimate or, in linear models, less than 0.10 standard deviations of the outcome away). Some of these need facts the pipeline does not write (the claim type, the outcome's standard deviation, the reproduced p-value under the paper's inference); give them in the run'smeasure_inputs.json(format insrc/ama_mcp/metrics.py), otherwise the value is null or rests on a stated assumption.ama_result.jsoncarries every measure of that paper.The strict survival rate of When AI Analysts Disagree (
survival.py): over the audited grid, the share of specifications whose estimate has the published sign (sign_consistency_rate) and the share that also has p < 0.05 under the designated inference (survival_rate), plus each choice's kill power (how much survival moves across its options). The companion headline repeats this with the paper's own excluded choices restored. The report gives it after the headline.
Resuming and stopping
The orchestrator records every finished step and every gate decision in
orchestrator_state.json. ama resume (or resume_run) continues after a stop, an error, a
crash or a pause without repeating finished work; an agent session that was cut off is
launched again with a fresh session. A run that a gate stopped stays stopped; --from-stage
runs a stage and every later stage again with fresh agents (existing artifacts stay in place).
A paused run continues to the end unless ama resume --stop-after STAGE (or resume_run with
stop_after) sets a new pause point at a later stage.
Safety
Agent sessions run with permission prompts off (Claude Code
bypassPermissions; Codex without its sandbox) because they must run long jobs unattended. Use a container or VM.Agents can read and change anything the operating-system user running
ama-mcpcan. That includes the installed package, so install it where that user cannot write (for example a root-owned environment in the container, with the analysis running as an unprivileged user); otherwise an agent could edit the prompts or gate scripts of its own run.Claude Code sessions start without your MCP servers and auto-memory and, with
--safe-mode(used when the installed CLI has it), without your CLAUDE.md files, hooks, skills and plugins. Codex sessions turn off your configured MCP servers one by name and switch off Codex's memories, apps and plugins features; yourAGENTS.mdfiles still load. A Codex MCP server whose name has characters other than letters, digits,-and_cannot be turned off this way, nor can any server whencodex mcp listfails; the orchestrator then writes a warning toorchestrator.log. Register this server under a plain name such asama. Details:docs/CHANGES_FROM_STUDY.md.Keep only the data you are licensed to use inside the environment; run directories may contain licensed data.
benchmark/in a run directory is for held-out material (e.g. a published critique of the paper) that you want to compare against afterwards. Every prompt forbids agents to read it, but nothing enforces this technically: a session could open it. The MCP server'sread_run_filerefuses to read it. Keep such material out of the environment if independence matters.
Tests
pip install '.[test]'
pytestThe orchestrator tests play every role with a fake agent and run the real gate scripts; no
test starts a real agent session unless AMA_LIVE_TESTS is set. tests/test_release.py
checks that the repository holds no machine paths, e-mail addresses, tokens or build products.
Maintainers can also point AMA_RELEASE_DENYLIST at a private file, kept outside the
repository, that lists names which must not appear (one per line).
License
MIT License. See LICENSE.
Citation
If you use this tool, please cite the accompanying papers:
Gao, Ruijiang, and Steven Chong Xiao, 2026, An AI-Powered Multiverse Analysis of Empirical
Finance Research: Evidence from Reused Natural Experiments, Working paper.
Gao, Ruijiang, and Steven Chong Xiao, 2026, When AI Analysts Disagree: Agentic Multiverse
Analysis and the Robustness of Empirical Research, Working paper.@unpublished{gao2026multiverse,
author = {Gao, Ruijiang and Xiao, Steven Chong},
title = {An {AI}-Powered Multiverse Analysis of Empirical Finance Research: Evidence from Reused Natural Experiments},
note = {Working paper},
year = {2026}
}
@unpublished{gao2026disagree,
author = {Gao, Ruijiang and Xiao, Steven Chong},
title = {When {AI} Analysts Disagree: Agentic Multiverse Analysis and the Robustness of Empirical Research},
note = {Working paper},
year = {2026}
}Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
Maintenance
Related MCP Connectors
ArXiv to code, replication audits, paper contradictions, citation audit, prose forensics.
Deterministic research automation with live OpenAlex search and reusable workflow programs.
Research-backed lint + generation for agent context files (CLAUDE.md, AGENTS.md) + prompt enhancer.
Machine-native research commons for agent evidence, discovery, rooms, and bounded research quests.
Related MCP Servers
- AlicenseNot gradedqualityAmaintenanceRun dynamic, multi-agent workflow scripts — agent(), parallel(), pipeline() — over real coding agents (Claude Code and OpenAI Codex), with deterministic journaling, resume, token budgets, and git-worktree isolation.7Apache 2.0
- AlicenseNot gradedqualityFmaintenanceAn autonomous software-engineering pipeline for Claude Code that runs on real evidence: codebase intelligence tools, git analytics, deterministic verification rules, and zero LLM-judges-LLM. Every claim traces to a paper and every PR passes its own gates.1MIT
- AlicenseNot gradedqualityCmaintenanceEnables multi-agent code review and iterative improvement loop using different LLM backends, with deterministic scoring and audit trail.5MIT
- AlicenseAqualityDmaintenanceEnables structured extraction of methods and reproducibility heuristics from academic papers, allowing AI agents to obtain metadata, full text, structured methods, code repository discovery, and a no-clone reproducibility verdict from a paper URL.845 PyPIMIT