Skip to main content
Glama
jiawei686

jev-paper-review-mcp

by jiawei686
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Remote

jev-paper-review-mcp

πŸ‡¨πŸ‡³ δΈ­ζ–‡ζ–‡ζ‘£

Let your agent score paper excerpts with confidence-calibrated structured decisions instead of parsing free-form LLM text. Powered by TypeSafe Jev (System One decision model).

A single-purpose MCP tool that turns a paper's abstract / method / results excerpt into a calibrated review score (three ordinal dimensions β€” soundness / novelty / clarity β€” plus an overall accept / revise / reject verdict). Your agent gets a structured, verifiable decision, not a prose essay.

npm install && npm run build
node dist/index.js doctor      # print dependency/mode diagnostics

Connect node dist/index.js in your MCP client and use the review_paper tool.

Why not just wrap another LLM

Jev is a decision model, not a chat model. You give it { state, questions } and it returns a probability distribution + confidence β€” not a JSON blob you have to guess at. That makes it a natural fit for "peer review", a judgment task with a standard rubric:

Dimension

Scale (low β†’ high)

soundness

weak Β· adequate Β· strong Β· exceptional

novelty

incremental Β· moderate Β· original Β· breakthrough

clarity

confusing Β· acceptable Β· clear Β· excellent

verdict

accept Β· minor_revision Β· major_revision Β· reject

Each dimension returns a score (ordinal index), label, probabilities (distribution), and confidence (calibrated confidence).

Related MCP server: squad-mcp

Tool: review_paper

Input

Param

Type

Description

text

string

The paper excerpt to review (abstract / method / results β€” any of them)

venue

string?

Optional, e.g. ICLR / NeurIPS; affects expectations only, never the logic

Output fields

Field

Meaning

soundness / novelty / clarity

Three-dimension scores, each with score / label / probabilities / confidence

verdict

{ choice, probabilities, confidence } β€” overall conclusion

confidence

Minimum of each dimension's confidence (see gate logic below)

recommendation

accept / minor_revision / major_revision / reject / desk_reject

reasoning_flags

Human-readable hints (e.g. weak_or_flawed_methodology, verify_novelty_before_accept)

mocked

Whether running in mock mode

Decision gate logic (agents must follow)

  • desk_reject: verdict=reject and soundness ≀ weak β†’ high-confidence direct reject

  • reject: verdict=reject

  • major_revision: verdict=major_revision (append soundness_concerns if soundness is low)

  • minor_revision / accept: per verdict

⚠️ Safety note: a high probability on one dimension does not mean it can decide for a human. Before any action, check confidence first β€” this service only emits strong conclusions when confidence is sufficient. Output is advisory only.

Example

{
  "soundness": { "score": 3, "label": "exceptional", "confidence": 0.82, "probabilities": {"0":0.0,"1":0.0,"2":0.15,"3":0.85} },
  "novelty":   { "score": 2, "label": "original",     "confidence": 0.74, "probabilities": {"0":0.1,"1":0.15,"2":0.75,"3":0.0} },
  "clarity":   { "score": 2, "label": "clear",        "confidence": 0.79, "probabilities": {"0":0.05,"1":0.2,"2":0.75,"3":0.0} },
  "verdict":   { "choice": "accept", "confidence": 0.8, "probabilities": {"0":0.8,"1":0.05,"2":0.05,"3":0.1} },
  "confidence": 0.74,
  "recommendation": "accept",
  "reasoning_flags": []
}

Extremely low cost

A single decision is just one Jev call; state is truncated at 60k characters, so it's tens of thousands of tokens per call, with no LLM calls at all β€” far cheaper per unit than a chat model. There's no token-usage chart because it's already that cheap β€” think of it as "one cheap gate in your review pipeline".

Zero-key / local-first

  • No TYPESAFE_API_KEY set β†’ automatically enters mock mode (deterministic offline stub, zero-config for CI / demos)

  • JEV_MCP_MOCK=1 β†’ force mock

  • Set TYPESAFE_API_KEY β†’ call the real Jev API (default jev-latest @ https://api.typesafe.ai/v1/systemone)

Environment variables

Variable

Default

Description

TYPESAFE_API_KEY

β€”

Required for real calls; blank falls back to mock

JEV_MODEL

jev-latest

Model name

JEV_BASE_URL

Official endpoint

Override for self-hosted / proxy

JEV_MCP_MOCK

0

1 forces mock

JEV_MCP_TIMEOUT_MS

30000

Per-request timeout

Tests

npm test     # smoke + MCP protocol handshake (mock mode, no key needed)

License

MIT

Available Tools

1 tool
review_paperA

Academic paper-review scorer powered by Jev (System One decision model). Given a paper excerpt, returns calibrated ratings on soundness / novelty / clarity (ordinal scales) plus an overall verdict (accept / minor_revision / major_revision / reject) and a recommendation. Treat the verdict as advisory β€” always check confidence before acting on it.

ParametersJSON Schema
NameRequiredDescriptionDefault
textYesThe paper excerpt to review: abstract, method section, results, or any passage.
venueNoOptional venue hint, e.g. 'ICLR', 'NeurIPS', 'journal'. Shapes expectations only.

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses that the tool is advisory, that `confidence` should be checked, and that the venue hint 'shapes expectations only' (i.e., it does not guarantee venue-specific behavior). It does not disclose details like rate limits, failure modes, or how confidence is computed, but the core behavioral caveat is clearly stated.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no filler. The first sentence front-loads the tool's purpose and outputs; the second adds the critical advisory caveat. Every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 2-parameter tool with 100% schema coverage and no output schema, the description covers the input, the output dimensions, and the key behavioral caveat. It could mention what happens with malformed or empty input, but nothing essential for a correct call is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters. The description adds context for `venue` ('shapes expectations only') and clarifies that `text` can be any passage, but it does not add substantial meaning beyond the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('returns calibrated ratings'), a clear resource ('Academic paper-review scorer'), and the exact outputs (soundness/novelty/clarity ratings, verdict, recommendation). It distinguishes itself from generic text-processing tools by naming the decision model (Jev/System One) and the ordinal scales. No sibling tools exist, so no sibling differentiation is needed.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use it: given a paper excerpt, it returns ratings and a verdict. It also gives a clear usage caveat: treat the verdict as advisory and check `confidence` before acting. It does not explicitly state when not to use it or name alternatives, but with no siblings and a clear input type, the context is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 1 tool updatev0.1.0
    • First observedreview_paper
Deploy Server
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A4/5.0

Scored across 1 tool

Disambiguation5/5

Only one tool exists, so there is zero ambiguity between tools. No other tools to confuse it with.

Naming Consistency5/5

The single tool follows the verb_noun pattern with snake_case (review_paper), which is clear and consistent. No other tools to compare against, but the naming is predictable and conventional.

Tool Count2/5

With only one tool, the server feels extremely thin for a paper-review MCP. Even niche servers typically need at least a few operations (e.g., scoring, retrieval, history). This is borderline trivial.

Completeness2/5

The server covers only the scoring action. Obvious gaps exist such as listing papers, retrieving previous reviews, or handling multiple paper submissions. The workflow appears incomplete, forcing agents to rely on external state.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • openreview-mcp

    OpenCodice-Research
    A
    license
    Not graded
    quality
    C
    maintenance
    MCP server for searching and retrieving submissions, reviews, meta-reviews, rebuttals, and decisions from OpenReview venues like NeurIPS and ICLR, enabling peer review analysis.
    2
    MIT

  • squad-mcp

    ggemba
    A
    license
    A
    quality
    C
    maintenance
    Exposes the squad-dev workflow as deterministic MCP tools, enabling task classification, risk scoring, specialist reviewer selection, and advisory verdict consolidation for code changes.
    27
    43 npm
    4
    Apache 2.0

  • LongBook Verifier

    Mormolykos
    A
    license
    Not graded
    quality
    D
    maintenance
    An MCP server that evaluates whether retrieval methods and AI outputs are grounded in long narrative manuscripts by retrieving evidence and scoring coverage deterministically, without external model APIs. It provides tools for chunking, indexing, retrieval, and evaluation.
    1
    MIT

  • spec-gate

    PedramMadani
    A
    license
    A
    quality
    B
    maintenance
    Provides MCP tools to score request sufficiency against a profile, block underspecified work, and write committed decision records with evidence and overrides.
    2
    4 npm
    Apache 2.0