Skip to main content
Glama
masa-med-ai

typesafe-screening-mcp

by masa-med-ai
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Remote

typesafe-screening-mcp

An MCP server that judges whether articles match a user's query or clinical question (CQ) from their title and abstract, using TypeSafe Jev — a "System One" model that returns calibrated probabilities instead of generated text.

Give it a PubMed query (or PMIDs you already have); it runs the search, fetches the abstracts itself, asks Jev, and returns include / maybe / exclude with the underlying probabilities. No separate PubMed tool is required. Abstracts never enter the LLM conversation, so hundreds of records can be screened in seconds for a fraction of a cent.

日本語の説明は下にあります。

Tools

Tool

Input

Use

search_and_screen

PubMed query + research question

Search PubMed and judge every hit in one step (up to 5000 hits, relevance order)

screen_pmids

research question + PMIDs

PubMed articles you already have PMIDs for (title/abstract fetched server-side via E-utilities)

screen_records

research question + [{id, title, abstract}]

Anything outside PubMed (CiNii, arXiv, database exports)

Optional arguments for all tools:

  • inclusion_criteria, exclusion_criteria — lists of short English statements

  • include_threshold (default 0.7), exclude_threshold (default 0.3)

  • return_decisions — which groups to list in results; default ["include", "maybe", "error"]. counts always covers every article, so a 300-hit search does not flood the conversation with excluded records.

  • detailed — false (default) returns one line per article, grouped by decision; true returns every probability per article (much longer).

  • save_full_results_to — path of a JSON file that receives every result, including excluded articles in full detail (for PRISMA-style records or a spreadsheet). An existing file is never overwritten.

How an article is judged

One Jev request per article, several typed questions at once:

  • match (Noul) — probability that the article directly addresses the research question

  • relevance (Score, 0–2) — off-topic / related background / directly answers, with confidence

  • one Noul per inclusion / exclusion criterion

The decision rule is plain code (_decide in server.py), tuned for sensitivity:

  1. an exclusion criterion ≥ include threshold → exclude

  2. match ≤ exclude threshold → exclude (or maybe when the record has no abstract)

  3. match ≥ include threshold → include, demoted to maybe if any inclusion criterion < 0.5 (abstracts often omit such details, so an unmet inclusion criterion never excludes)

  4. otherwise → maybe

Results are sorted by match. The detailed output and the saved file keep every probability, so you can re-threshold later.

Related MCP server: jev-mcp

Setup

Requires uv and a TypeSafe API key (console).

git clone https://github.com/masa-med-ai/typesafe-screening-mcp.git
cd typesafe-screening-mcp
uv sync

Provide the key in one of two ways (never write it into a config file in plain text):

# macOS keychain (prompts for the key)
security add-generic-password -s typesafe-api-key -a "$USER" -w

# or an environment variable
export TYPESAFE_API_KEY=...

Optional environment variables: NCBI_API_KEY (higher E-utilities rate limit), TYPESAFE_MODEL (default jev-latest).

Claude Code

claude mcp add --scope user typesafe-screening -- \
  uv run --project /path/to/typesafe-screening-mcp python /path/to/typesafe-screening-mcp/server.py

Claude Desktop / other MCP clients

{
  "mcpServers": {
    "typesafe-screening": {
      "command": "uv",
      "args": ["run", "--project", "/path/to/typesafe-screening-mcp", "python", "/path/to/typesafe-screening-mcp/server.py"]
    }
  }
}

Optional: Claude Code skill

skills/literature-screening/ is a thin skill that tells Claude how to use this server well: turning a CQ into a literal-friendly English sentence, building a broad query, placing criteria on the inclusion or exclusion side, always saving the full results, and reporting maybe / exclude honestly. It also ships a script that turns a saved full-results file into an Excel workbook (Summary / Include / Maybe / Exclude sheets with author, year, journal, publication type, PubMed links and columns for human decisions). The skill text is in Japanese.

# install (symlink keeps it in sync with the repo)
ln -s /path/to/typesafe-screening-mcp/skills/literature-screening ~/.claude/skills/literature-screening

# Excel export on its own
uv run skills/literature-screening/scripts/results_to_xlsx.py full_results.json screening.xlsx

The MCP server works without the skill; the skill needs the MCP server.

Usage

Ask your assistant something like:

With typesafe-screening, search PubMed for prospective studies of CADx in colonoscopy from the last 5 years and judge them against the question "How well does CADx characterize colorectal polyps during colonoscopy?" Save the full results to ~/cadx_screening.json.

The assistant writes the PubMed query and the English research question; the server does the rest. search_and_screen also returns total_hits, screened and PubMed's query_translation, so you can see whether max_results cut anything off.

Default (slim) result — one line per article, PMID | match probability | title, with the reason appended when a criterion or a missing abstract drove the decision:

{
  "total_hits": 15,
  "screened": 15,
  "counts": {"include": 8, "maybe": 5, "exclude": 2, "error": 0},
  "results": {
    "include": [
      "41563136 | 0.98 | Real-Time Characterization of Colonic Polyps: A Multicenter Prospective Study Evaluating the CAD-EYE System in Screening"
    ],
    "maybe": [
      "40375757 | 0.96 | Computer-aided diagnosis for colorectal polyp in comparison with endoscopists: Systematic review and meta-analysis. | inclusion criterion #1 not evident in abstract"
    ],
    "error": []
  }
}

With detailed: true (and always in the saved file), each article carries every probability:

{
  "id": "32371116",
  "title": "Efficacy of Real-Time Computer-Aided Detection of Colorectal Neoplasia in a Randomized Trial.",
  "decision": "include",
  "reason": "high match",
  "match": 0.99,
  "relevance": 2.0,
  "relevance_confidence": 1.0,
  "inclusion": [0.99]
}

In one real run, 326 PubMed hits were screened in about 17 seconds using ~330k input tokens (about US$0.014 at the time of writing).

Writing good questions and criteria

Jev reads literally, so the wording decides the result.

  • Write the research question as one self-contained English sentence; spell out abbreviations. Non-English input works less well.

  • Phrase criteria as positive statements ("The study is a randomized controlled trial"). Avoid negations, "only", and double negatives.

  • A narrow question gives narrow matches: "How well does CADx characterize polyps?" will score safety or cost-effectiveness papers on CADx lower. Broaden the wording if you want them.

  • Do numeric and date limits (sample size, publication year) in the PubMed query, not in criteria — Jev is unreliable with numbers and dates.

Limitations

  • A screening aid, not a replacement for human review. The default thresholds are not calibrated on labelled data. For a systematic review, validate sensitivity against a known set of included studies, and have humans review at least the maybe group and a sample of exclude.

  • Judgement uses title and abstract only. Records without an abstract are never auto-excluded.

  • Titles/abstracts are sent to the TypeSafe API. Do not send patient data or other confidential text.

  • Abstract text is untrusted input; Jev does not defend against instructions embedded in it.

  • PubMed only for the built-in search; other databases go through screen_records.

This project is not affiliated with TypeSafe or NCBI. When using E-utilities, follow the NCBI usage guidelines.

日本語

文献検索のとき、ユーザーの検索意図や CQ に合う文献かどうかを、タイトルと抄録から TypeSafe の Jev で判定する MCP サーバーです。

  • search_and_screen: PubMed の検索式と CQ を渡すと、検索・抄録取得・判定までサーバー側で一括実行します（別途 PubMed 用のツールは不要）。抄録が LLM の会話に乗らないため、数百件でも数十秒・数円未満で処理できます。

  • screen_pmids: 手元に PMID がある場合はこちら。

  • screen_records: PubMed 以外（CiNii、arXiv など）の {id, title, abstract} を直接渡します。

  • 各文献に include / maybe / exclude と、根拠となる確率（CQ への一致、関連度、採択・除外基準ごとの確率）を返します。判定ルールはコードで固定されており、感度優先です（採択基準を満たさないだけでは除外せず maybe にします）。

  • 既定では include / maybe / error だけを、1 文献 1 行（PMID | match | タイトル）で返します（件数は全件分）。全確率が必要なら detailed: true。return_decisions で変更でき、save_full_results_to に JSON のパスを渡すと除外分を含む全結果をファイルに保存します。

  • skills/literature-screening/ は Claude Code 用の薄いスキルです（CQ の英文化、検索式と基準の作り方、結果報告の手順、全結果 JSON を Excel にするスクリプト）。~/.claude/skills/ にシンボリックリンクを張って使います。MCP 本体はスキルなしでも動きます。

  • API キーは環境変数 TYPESAFE_API_KEY か macOS キーチェーン（サービス名 typesafe-api-key）から読みます。

  • CQ と基準は英語の肯定文で渡してください（日本語で依頼すれば、呼び出し側の LLM が英訳して渡します）。数値・年の条件は PubMed の検索式側で絞るのが確実です。

  • 閾値は実データで較正していません。系統的レビューで使う場合は、既知の採択文献で感度を確認し、maybe と exclude の一部は人が確認してください。患者情報などの機密テキストは送らないでください。

License

MIT

Available Tools

3 tools
screen_pmidsA

Judge whether the given PubMed articles match the user's query / clinical question (CQ).

Use this when you already have PMIDs; use search_and_screen to search and judge in one step. Titles and abstracts are fetched server-side from PubMed, so abstracts stay out of the conversation. Each article gets include / maybe / exclude plus the underlying probabilities, sorted by match probability.

Args: research_question: The user's query or CQ, as ONE self-contained sentence in ENGLISH (Jev is most accurate in English and reads literally - translate Japanese input, spell out abbreviations, avoid negations/double negatives). pmids: PubMed IDs to screen. inclusion_criteria: Optional extra criteria, each a short positive English statement (e.g. "The study is a randomized controlled trial"). An unmet criterion demotes include to maybe; it never excludes, since abstracts often omit such details. exclusion_criteria: Optional; a confidently met criterion excludes the article (e.g. "The article is a case report"). include_threshold: match probability at or above which an article is included. exclude_threshold: match probability at or below which an article is excluded. return_decisions: Which groups to list in "results". Default ["include", "maybe", "error"]; "counts" always covers every article. Add "exclude" only for small batches. save_full_results_to: Optional file path (.json). Every result, including excluded articles, is written there in full detail. Fails if the file already exists. detailed: False (default) returns one line per article, grouped by decision: "PMID | match | title". True returns every probability per article (much longer).

ParametersJSON Schema
NameRequiredDescriptionDefault
pmidsYes
detailedNo
return_decisionsNo
exclude_thresholdNo
include_thresholdNo
research_questionYes
exclusion_criteriaNo
inclusion_criteriaNo
save_full_results_toNo

TDQS

A5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full responsibility and does so thoroughly: it discloses that titles/abstracts are fetched server-side so abstracts stay out of the conversation, explains how inclusion_criteria demote include to maybe but never exclude, notes exclusion_criteria behavior, states save_full_results_to fails if the file already exists, and describes sorting by match probability. This is rich behavioral detail beyond the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but every sentence earns its place. It front-loads purpose and usage, then organizes argument semantics in a scannable Args list. The length is justified because there are 9 undocumented parameters and no annotations, and the wording is tight rather than redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 9 parameters, no output schema, and no annotations, the description is remarkably complete. It covers all parameters, return groups, threshold behavior, criteria semantics, output verbosity, and the side effect of save_full_results_to. There are no critical gaps an agent would need to guess about.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must fully compensate. It does: every parameter is explained with constraints, defaults, examples, and behavioral consequences. For instance, research_question must be one self-contained English sentence, inclusion_criteria get a concrete example and demotion rule, and return_decisions explains default group membership and the 'counts' coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Judge whether the given PubMed articles match the user's query / clinical question.' It also distinguishes itself from the sibling search_and_screen by stating 'Use this when you already have PMIDs.' This leaves no ambiguity about what the tool does.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit usage guidance is provided: 'Use this when you already have PMIDs; use search_and_screen to search and judge in one step.' It also gives practical advice on thresholds, criteria behavior, and warns to add 'exclude' only for small batches. The description clearly tells the agent when to choose this tool over an alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

screen_recordsB

Same as screen_pmids, for articles that are not in PubMed (CiNii, arXiv, Embase exports...).

Args: research_question: The user's query or CQ as one self-contained English sentence. records: List of {"id": str, "title": str, "abstract": str}. English text works best. Other arguments: see screen_pmids.

ParametersJSON Schema
NameRequiredDescriptionDefault
recordsYes
detailedNo
return_decisionsNo
exclude_thresholdNo
include_thresholdNo
research_questionYes
exclusion_criteriaNo
inclusion_criteriaNo
save_full_results_toNo

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It describes the input record shape and the research_question but does not disclose thresholds, decision outputs, return formatting, completion behavior, or side effects, instead deferring to screen_pmids.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded, with the distinguishing non-PubMed purpose first. The Args block is compact and readable; the only inefficiency is relying on a sibling definition instead of providing key details inline.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a nine-parameter tool with no annotations and no output schema, the description is incomplete on its own. It gives selection and core input guidance but leaves thresholds, criteria, output behavior, and save destination to inference or to another tool's documentation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for all nine parameters. It explains research_question and records well, but the other seven parameters are only dismissed with "Other arguments: see screen_pmids," which is a pointer rather than actual semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description identifies a specific action (screen records) and narrows the target set to articles not in PubMed, with concrete source examples (CiNii, arXiv, Embase). It differentiates from screen_pmids by source coverage, though it depends on the sibling definition for the full action semantics.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Same as screen_pmids, for articles that are not in PubMed" explicitly states when this tool is appropriate (non-PubMed records) and implicitly routes PubMed records to the sibling screen_pmids. This is clear selection guidance without ambiguity.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_and_screenA

Run a PubMed search and judge every hit against the user's query / clinical question (CQ).

Search, abstract retrieval and judgement all happen server-side; only the decisions come back. Use a broad, sensitive query - screening is cheap (hundreds of articles in seconds) - and let the judgement do the narrowing.

Args: pubmed_query: PubMed search expression (field tags, MeSH, boolean operators). Put date, language and numeric limits here, e.g. ("2021/01/01"[dp] : "3000"[dp]); Jev is unreliable with numbers and dates. research_question: The user's query or CQ, as ONE self-contained sentence in ENGLISH (Jev is most accurate in English and reads literally - translate Japanese input, spell out abbreviations, avoid negations/double negatives). max_results: Screen at most this many hits, taken in PubMed relevance order (max 5000). Check "total_hits" against "screened" in the result to see whether hits were left out. inclusion_criteria: Optional extra criteria, each a short positive English statement (e.g. "The study is a randomized controlled trial"). An unmet criterion demotes include to maybe; it never excludes, since abstracts often omit such details. exclusion_criteria: Optional; a confidently met criterion excludes the article (e.g. "The article is a case report"). include_threshold: match probability at or above which an article is included. exclude_threshold: match probability at or below which an article is excluded. return_decisions: Which groups to list in "results". Default ["include", "maybe", "error"]; "counts" always covers every article. Add "exclude" only for small batches. save_full_results_to: Optional file path (.json). Every result, including excluded articles, is written there in full detail. Fails if the file already exists. detailed: False (default) returns one line per article, grouped by decision: "PMID | match | title". True returns every probability per article (much longer).

ParametersJSON Schema
NameRequiredDescriptionDefault
detailedNo
max_resultsNo
pubmed_queryYes
return_decisionsNo
exclude_thresholdNo
include_thresholdNo
research_questionYes
exclusion_criteriaNo
inclusion_criteriaNo
save_full_results_toNo

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations available, the description carries the full burden and does so thoroughly. It discloses server-side processing, the irrelevance of numerical limits in Jev, threshold behavior for include/exclude, the demotion of include to maybe when inclusion criteria are unmet, file-overwrite failure for save_full_results_to, and the exact output grouping behavior. This is far beyond a generic statement and gives an agent actionable expectations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long, but the tool has 10 parameters and complex behavior; every sentence adds operational value. It is front-loaded with the core purpose and key performance guidance, followed by a logically ordered parameter breakdown. No filler or repeated information is present.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 10 parameters, no output schema, and no annotations, the description provides all the information needed to call and interpret the tool correctly. It explains output groups ('include', 'maybe', 'error'), how to detect incomplete screening via 'total_hits' vs 'screened', the detailed vs concise output formats, and edge cases like overwrite failures. Nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 0% description coverage, so the description must compensate for all 10 parameters. The Args section documents every parameter with practical detail: examples for pubmed_query, the one-sentence English requirement for research_question, the meaning of thresholds, the effect of inclusion vs exclusion criteria, and the default return groups. This fully replaces missing schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Run a PubMed search and judge every hit against the user's query / clinical question'. This clearly differentiates search_and_screen from its siblings (screen_pmids, screen_records), which evaluate already-supplied PMIDs or records rather than performing their own PubMed search.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives strong contextual guidance: 'Use a broad, sensitive query - screening is cheap' and notes that 'search, abstract retrieval and judgement all happen server-side'. It does not explicitly name alternatives or state when not to use this tool, but the clear search-based scope makes the comparison to the screen-only siblings apparent. It stops short of an explicit either/or routing rule.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 3 tool updatesv0.1.0
    • First observedscreen_pmids
    • First observedscreen_records
    • First observedsearch_and_screen
Deploy Server
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A4.1/5.0

Scored across 3 tools

Disambiguation4/5

Each tool has a distinct input path: search-then-screen, screen specific PMIDs, and screen arbitrary non-PubMed records. screen_pmids and screen_records are similar in action but clearly separated by input type, so an agent can select correctly with the descriptions.

Naming Consistency4/5

Tools follow a mostly consistent verb-first snake_case pattern (screen_pmids, screen_records), with search_and_screen as the one oddity using a compound verb form. The naming is predictable and readable despite this minor deviation.

Tool Count5/5

Three tools is a well-scoped set for a narrow screening workflow: one search+screening tool and two variants for different input sources. Each tool earns its place and there is no bloat.

Completeness4/5

The tools cover the full screening pipeline: searching PubMed, screening known PMIDs, and screening external records. A minor gap is the lack of a raw article/metadata retrieval tool, but it is not essential for the stated screening purpose.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • biolit

    rachadele
    A
    license
    Not graded
    quality
    B
    maintenance
    Enables LLM-assisted biomedical literature screening and structured extraction from PubMed alerts, PMIDs, DOIs, and GEO accessions, with full-text retrieval and multi-provider LLM support.
    74 PyPI
    2
    MIT

  • jev-mcp

    arunav25
    A
    license
    Not graded
    quality
    C
    maintenance
    Enables MCP clients to call TypeSafe's JEV classifier and receive structured, typed judgments with probabilities for binary, choice, and scoring questions.
    MIT

  • daf-jev

    docxology
    A
    license
    Not graded
    quality
    B
    maintenance
    Enables natural-language interaction with TypeSafe's Jev decision API, supporting mixed question calls, batch evaluation, model listing, and confidence or composite-score gates over stdio.
    3
    MIT

  • Jev MCP

    chungsubeen0
    A
    license
    A
    quality
    B
    maintenance
    Enables frontier coding agents to delegate routine probabilistic judgments to TypeSafe Jev, providing calibrated triage signals for failures, attempts, completion, context ranking, findings, risk, and generic evidence-grounded questions.
    7
    MIT