Skip to main content
Glama

Evidenceline

A working concept of an assistant for PFAS groundwater monitoring in Western Australia: it checks report text against lab data and guideline values, and shows where every number came from. The site and every lab result are synthetic (a fictional site, FDS-01); the guidance is real and public; this is a personal project, not affiliated with any company.

Live site: https://evidenceline.autopilotyourworkflow.com (the website launches first; its live question box and the hosted connector say "coming soon" until the small API service behind them is switched on)

What it does and who it is for

It is for environmental scientists who screen lab results against guideline values and write reports that a person signs. Evidenceline is a set of nine read-only tools that run inside Claude through MCP (the Model Context Protocol), in Python:

  • Tidy messy lab files. It combines a lab results file, a chain of custody and a field sheet into one table, then lists numbered questions for the scientist (sample ids that do not match, unit mix-ups, duplicate differences, holding times, blanks, detection limits above a criterion), each with the rows it is based on and the rule it applied.

  • Screen results under both drinking-water rules, side by side. WA names PFAS NEMP 3.0 as adopted; the national values were updated in 2025. Evidenceline shows both and never picks one.

  • Answer guidance questions from the public documents, with the document, edition and page it relied on, or say "not covered" instead of guessing.

  • Write and check report text. The model writes placeholders; code fills in the exact numbers; a checker then traces every number, change and comparison in the paragraph back to its lab row or guideline table.

A guideline value is an investigation level. Evidenceline never says water is unsafe or a site is contaminated; those are a scientist's judgement.

Related MCP server: MCP_Documents

Choices I made

  • Deterministic checks in code, not in the model. Screening, sums, non-detects, unit conversion and the paragraph checker are plain Python with exact decimals. A model can suggest wording; it never decides whether a number is right.

  • Placeholders for numbers. The model writes {PFOS|MB2|Sep 2025} and fill_numbers puts the value in from the data, with a numbered source for each. If one placeholder is wrong, nothing is filled.

  • Two rules side by side. Every screening result is shown under PFAS NEMP 3.0 and under the current national values, with the arithmetic. Choosing between them stays with the scientist.

  • Redaction at the tool boundary. Client, site and people's names listed in a local identifier file, and street addresses, lot numbers, emails and phone numbers found by built-in patterns, are replaced with placeholders before any tool output reaches a model, in any letter case or spacing. If a configured identifier file is missing or broken, every tool fails closed. Without that file, names are not redacted. The hosted connector loads a built-in file that lists only the fictional site's client and address.

  • Lexical search plus a number verifier on AI answers. Guidance search is BM25 over page-sized chunks, so every passage has an exact page. The live question box only shows a written answer if code can find every number in it on the page it cites (or in the verified guideline values); otherwise it shows the passages alone.

What went wrong

  • The first paragraph checker let 15 false claims pass. An independent tester who had not written the code wrote adversarial cases from the lab data by hand; the first version failed 19 of them, 15 of which passed a false claim. All were fixed, and the cases stay in tests/test_adversarial.py.

  • Client names leaked through unusual spellings. An independent tester got the fictional client name through the redaction with capitals, line breaks, tabs, no-break spaces and similar tricks. Matching now ignores case and separators; those cases are in tests/test_adversarial_phase1.py.

  • A number after the word "Lot" was mangled. The redaction read "Lot" followed by a filled-in result as a land lot and turned "0.038" into "[LOT-1].038". Filled values are now put back between separately redacted stretches of text.

  • A guideline note misread its source. The first data file said the PFAS NEMP 3.0 value of 0.07 ug/L applied only to the sum of PFOS and PFHxS. Table 4, footnote a says it means "PFOS only, PFHxS only, and the sum of the two". Both automated verification passes caught it; the note, the screening, the checker and the website now follow the footnote. The same passes could not find the word "Total" in the arsenic soil note's source, so it was removed. Two later independent re-checks read both sources again and confirmed both new notes. One of them also found that lookup_limit, asked for PFOS alone under NEMP 3.0, still named the sum as the quantity to compare; it now names PFOS on its own, and so does the question box.

  • Casual questions were refused. The first held-out question set, written without seeing the search, found the right page first for only 6 of 24 questions: chatty words that no document contains outweighed the words that mattered. That set was then used to fix the search (17 of 24 after), so it became a tuning set. A second held-out set, never used for tuning, is the fair measure: the right page comes first for 12 of 24, and for 4 of its 12 casual questions, so everyday wording is still the search's main weakness. A stricter check later found seven of its questions close in wording to tuning questions; left out, the score is 9 of 20 (casual 3 of 10), and the Accuracy page shows both.

  • The checker misread a guideline value as a result. "below the current value of 0.03 ug/L" was read as a measured PFHxS result and flagged. The checker now recognises a rule name before the word "value"; the sentence is a test in tests/test_checker.py.

What I'd do differently

  • Write the held-out questions first, and keep writing new ones. The search was tuned on the same questions it was scored on, which flattered it. A held-out set written without seeing the search came later, and fixing the search on it used it up; a second one (evals/guidance_heldout2.json) is now the only fair score.

  • Add semantic search earlier. Lexical search is exact about pages but misses questions worded unlike the documents, such as how PFAS samples should be stored.

  • Trace each number to the passage its own sentence cites. The answer verifier ties every concentration to the guideline value cited in its own sentence, but a plain number is still accepted when it is found in any cited passage.

  • Read real lab export formats. The tidy step works on one packaged fictional site, not on a folder of files.

  • Get a practitioner to review the guideline values. Today they are checked by two independent automated passes against the source pages, and the site says exactly that.

Quick start

Python 3.12 or later. The server needs only mcp and pydantic.

git clone https://github.com/autopilotyourworkflow/evidenceline
cd evidenceline
python -m venv .venv
.venv/bin/python -m pip install --upgrade pip     # Windows: .venv\Scripts\python
.venv/bin/python -m pip install -e .

Guidance search reads a local index built from the public documents, which are downloaded, never committed:

.venv/bin/python -m pip install -e . --group corpus   # needs pip 25.1 or later
.venv/bin/python scripts/fetch_corpus.py
.venv/bin/python scripts/build_index.py

Add the server to Claude Code (stdio), using the absolute path of your checkout:

claude mcp add evidenceline -- /path/to/evidenceline/.venv/bin/evidenceline-mcp

Or in a Claude Desktop or project .mcp.json config:

{ "mcpServers": { "evidenceline": { "command": "/path/to/evidenceline/.venv/bin/evidenceline-mcp" } } }

Then ask Claude, for example: "Use the evidenceline tools to check this paragraph about well MB2: ...".

A read-only hosted copy will also be available as a remote connector (no install, rate-limited, same nine tools). Until its service is switched on, the website says "coming soon" and this address answers that it is not connected yet:

claude mcp add --transport http evidenceline-demo https://evidenceline.autopilotyourworkflow.com/mcp

The hosted copy loads a built-in identifier file for the fictional site only (EVIDENCELINE_REDACT=builtin:fds01-demo): its made-up client name and address become placeholders, but any other client or people's names are not redacted. Use it with the fictional site, not with client work.

Redaction reads an optional identifier file (~/.evidenceline/redact.toml; see examples/redact.example.toml). Client, site and people's names are redacted only when listed there. Without it, the built-in patterns for emails, WA lots, street addresses and Australian phone numbers still apply.

Tools

Tool

What it does

tidy_lab_files

Combines the lab file, chain of custody and field sheet for FDS-01 into 73 rows and six numbered review items, each with evidence rows, the rule, its quoted source and what the scientist decides. Rows on request (include_rows).

get_review_item

One review item with every evidence line quoted exactly from its file.

get_results

Well MB2's results over four rounds, each with its lab report, file and row.

lookup_limit

One drinking-water value with its document, table, page and WA status.

compare_rules

One monitoring round screened under both rules side by side, with the arithmetic.

check_paragraph

Every number, change, guideline and detection claim in a paragraph traced or flagged, and what was not checked.

fill_numbers

Replaces placeholders with exact values from code and returns a numbered source list. All or nothing.

search_guidelines

Passages from the public guidance with document, edition, WA status, page and link, or "not covered".

show_redactions

The placeholders in use and the patterns loaded, never the raw values.

Run the tests

.venv/bin/python -m pip install -e . --group dev --group api
.venv/bin/python -m pytest
.venv/bin/python -m ruff check src tests scripts
.venv/bin/python -m pyright                              # strict
.venv/bin/python -m evidenceline.guidance.evaluate            # search, tuning set 1 (golden)
.venv/bin/python -m evidenceline.guidance.evaluate --heldout   # tuning set 2 (the first held-out set)
.venv/bin/python -m evidenceline.guidance.evaluate --heldout2  # held-out set 2, never tuned on
.venv/bin/python scripts/prepublish_check.py             # nothing private in the repository
cd web && npm ci && npm run check && node tests/adversarial.mjs

Developer notes (layout, design decisions, known limits): DEVNOTES.md. Going live: DEPLOY.md.

Accuracy

Test results, search scores and the automated checks of every guideline value are on the site's Accuracy page. It leads with the held-out set the search was never tuned on, and labels the two tuning sets as such. Each guideline value was checked by two independent automated passes, and the notes reworded after them by two later re-checks. CI recomputes the page's data on every change and keeps the result as a download. No practitioner has reviewed the values.

Licence

Code: Apache License 2.0. The guidance documents are not in this repository; each one keeps its own terms, listed with its source in src/evidenceline/data/corpus_manifest.json, and the site quotes only short excerpts.

Photo credits

Photos on the site are from Unsplash and Pexels contributors, used under the Unsplash License and the Pexels License: Iain (@photoken123), oscabla, Sear Greyson, Nathan Hurst (Unsplash) and Alexey K. (Pexels). Details and photo pages: web/public/img/credits.md.

Available Tools

9 tools
check_paragraphCheck a report paragraphA
Read-onlyIdempotent

Check a draft report paragraph against the lab data and guideline values, without any language model.

It finds (a) every number with a unit (ug/L, µg/L, ng/L, mg/L) and traces it to a lab row, a detection limit, a computed sum or difference, or a guideline table, or flags it as untraced; (b) claims of change between two dates (increased, fell, lower than, stable) and tests them against the data; (c) claims that a result is above, below or within the guideline, evaluated under each rule; (d) claims that an analyte was or was not detected. Negation such as 'did not increase' is handled.

Every checked item has a status: consistent, inconsistent, untraced, depends_on_rule or needs_judgement. Anything it could not read confidently is listed under not_checked with the reason. Show the person every item that is not 'consistent' and every not_checked item; the person decides what to change.

ParametersJSON Schema
NameRequiredDescriptionDefault
textYesThe draft report paragraph to check, in plain English.
wellYesMonitoring well id, for example 'MB2'.

Output Schema

ParametersJSON Schema
NameRequiredDescription
wellYes
notesYes
scopeYesWhat kinds of statement this checker covers.
checkedYes
summaryYes
sentencesYes
not_checkedYes
needs_attentionYesChecked items whose status is not 'consistent'.

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint false, so the description adds value by explaining the tool is deterministic ('without any language model'), by describing the statuses it produces, and by noting that unclear items are reported under not_checked with reasons. This goes well beyond the annotation metadata and gives the agent a clear picture of behavior and output.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is longer than average, but it is organized with a clear opening sentence and a bulleted list of the main check categoriesholistic. Each sentence earns its place by explaining a distinct aspect: what the tool checks, how statuses are assigned, and how the output should be handled. It could be slightly trimmed, but the structure aids comprehension.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with only two required parameters and an output schema, the description covers all essential context: the types of checks performed, the statuses returned, the handling of unclear items, and the instruction to surface non-consistent results to the user. The presence of an output schema means the return values do not need to be explained, and the description is complete enough for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already describes both parameters fully, including that text is the draft report paragraph in plain English and well is a monitoring well id. With schema description coverage at 100%, the description does not need to add much; it does reinforce the purpose of the text parameter in the opening sentence, but no additional parameter-level detail is necessary.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Check a draft report paragraph against the lab data and guideline values,' which clearly identifies the tool's function. It further differentiates the tool by stating it operates 'without any language model' and enumerates the categories of checks performed, making it easy to distinguish from sibling tools like fill_numbers or lookup_limit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a clear usage context: checking draft report paragraphs against lab data and guidelines, with a deterministic process and an explicit instruction to show non-consistent and not_checked items to a human. It does not explicitly name alternatives or state when not to use this tool, but the context is strong enough for an agent to infer when it is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

compare_rulesCompare both rules for one roundA
Read-onlyIdempotent

Screen one monitoring round under BOTH guideline rules side by side, with the arithmetic shown.

For each rule, every limit is listed with the value compared (for a sum, for example '0.038 + 0.019 = 0.057'), the limit, and whether the result is above or not above it. A result equal to the limit is not above it. The tool never picks a rule: choosing one is the scientist's call. When the two rules disagree, the notes say so; the lab values are identical in both columns.

ParametersJSON Schema
NameRequiredDescriptionDefault
dateYesSample date of the round: '2025-09-16', '16 September 2025' or 'Sep 2025'.
wellYesMonitoring well id, for example 'MB2'.

Output Schema

ParametersJSON Schema
NameRequiredDescription
dateYes
fileYes
wellYes
notesYes
rulesYes
lab_reportYes
rules_agreeYes

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnly/idempotent annotations, the description discloses key behavior: a result equal to the limit is 'not above', the tool never chooses a rule, disagreements are flagged in the notes, and lab values appear identically in both columns. This is substantive, decision-relevant behavioral disclosure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core purpose and each subsequent sentence covers a necessary caveat: arithmetic display, equality handling, rule selection, and disagreement behavior. It is detailed without being bloated.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With only two well-documented parameters, an output schema, and annotations covering safety, the description needs only to supply the edge-case behavior and decision boundary. It does so thoroughly, making the tool actionable for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%: date and well each have their own format descriptions and examples. The prose does not add parameter-level semantics beyond the schema, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Screen one monitoring round under BOTH guideline rules side by side,' clearly naming the resource (a monitoring round) and the operation (comparison under both rules). The detail about per-rule limits and arithmetic separates it from sibling tools like lookup_limit or get_results.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'BOTH rules' phrasing implies this tool is for side-by-side comparison, but the description never explicitly says when to use it instead of alternatives such as get_results or lookup_limit. The note that the tool never picks a rule is useful context but is not a usage guideline.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fill_numbersFill in the numbersA
Read-onlyIdempotent

Fill in the numbers: replace each placeholder in a draft paragraph with the exact value and unit, by code.

Never type a concentration yourself. Write a placeholder where each number goes, then call this tool. Fields are separated by '|'; case and spaces around fields do not matter:

  • {ANALYTE|WELL|DATE}: measured result, e.g. {PFOS|MB2|Sep 2025} gives '0.038 ug/L'. A non-detect gives the result as reported, e.g. '<0.001 ug/L'; describe it as not detected.

  • {sum PFOS+PFHxS|WELL|DATE}: the sum of PFOS and PFHxS, e.g. {sum PFOS+PFHxS|MB2|2025-09-16} gives '0.057 ug/L'.

  • {lor|ANALYTE|WELL|DATE}: the detection limit, e.g. {lor|PFOA|MB2|Sep 2025} gives '0.001 ug/L'.

  • {limit|ANALYTE|RULE}: a guideline value. RULE is 'nemp-3.0' or 'current' and is required, e.g. {limit|PFOS|current} or {limit|PFOS+PFHxS|nemp-3.0}. Under nemp-3.0 use the PFOS+PFHxS form: its 0.07 ug/L applies to PFOS alone, PFHxS alone and the sum (Table 4, footnote a), so say which one the sentence compares.

  • {change|ANALYTE|WELL|FROM|TO}: change from the earlier round FROM to the later round TO, with direction and percent, e.g. {change|PFOS|MB2|Nov 2024|Sep 2025} gives 'a fall of 0.003 ug/L (7.3%)'. Write the sentence so the phrase fits, e.g. 'PFOS showed {change|...} between November 2024 and September 2025.' Dates: '2025-09-16', '16 September 2025' or 'Sep 2025'. One well per call.

Returns the filled text, the same text with [n] source markers, each value with its offsets and explanation, and a numbered source list (lab file and row, or guideline document, table and page). If any placeholder is unknown or ambiguous, nothing is filled and the error lists every problem and the valid forms. Afterwards, call check_paragraph on the filled text: it checks the words around the numbers.

ParametersJSON Schema
NameRequiredDescriptionDefault
textYesThe draft paragraph, with every number written as a placeholder such as {PFOS|MB2|Sep 2025}.
wellYesMonitoring well id, for example 'MB2'. Placeholders must name this well.

Output Schema

ParametersJSON Schema
NameRequiredDescription
textYesThe text with every placeholder replaced. Pass this to check_paragraph.
wellYes
notesYes
valuesYes
sourcesYes
next_stepYes
annotated_textYesThe same text with source numbers in brackets after each value.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnly/idempotent annotations, it discloses failure behavior (nothing is filled if a placeholder is unknown or ambiguous, with an error listing every problem and valid forms), return structure (filled text, markers, offsets, explanations, source list), and constraints such as RULE being required and nemp-3.0 semantics. No overreach or contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The definition is long but necessary; every section (instruction, placeholder grammar, date formats, error behavior, follow-up step) earns its place. The core purpose and the 'never type a concentration' instruction are front-loaded before the detailed grammar.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with this many placeholder variants and rules, the description covers all needed call-time knowledge: exact formats, unit conventions, the nemp-3.0/current rule, return payload, error handling, and the follow-up check_paragraph call. An output schema exists, but the description explains behavior rather than duplicating it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although the schema already documents text and well at 100%, the description enriches them: placeholders must name the well, well is a monitoring-well id, fields are '|'-separated, case/spacing do not matter, and each placeholder form is exemplified. This goes well beyond the schema's terse 'draft paragraph' and 'Monitoring well id'.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a precise statement: replace each placeholder in a draft paragraph with the exact value and unit, by code. It further explains the placeholder grammar and the 'Never type a concentration yourself' rule, making it distinct from sibling check/read tools. The purpose is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly tells the agent to write placeholders and then call this tool instead of typing values, and instructs calling check_paragraph afterwards. It defines one well per call, but it does not name alternative tools or situations when another sibling should be preferred, stopping short of full when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_resultsGet lab resultsA
Read-onlyIdempotent

Return the lab results for one monitoring well, oldest round first.

Each result has the sample date, analyte, the result exactly as the lab reported it, the exact value in ug/L (null when not detected), the detection limit, the lab report id, and the file and row it came from, so any number can be traced to its source. Use these values verbatim; do not round them.

ParametersJSON Schema
NameRequiredDescriptionDefault
wellYesMonitoring well id, for example 'MB2'.
analyteNoOptional analyte to filter by, for example 'PFOS'. Omit for all analytes.

Output Schema

ParametersJSON Schema
NameRequiredDescription
noteYes
siteYes
unitYes
wellYes
countYes
datesYesSample dates with results, oldest first.
matrixYes
resultsYes

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnly and idempotent annotations, the description discloses output contents, null handling for non-detects, traceability to source file/row, and a strict instruction to use values verbatim without rounding. No contradiction with annotations exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core purpose and ordering, followed by a concise list of return fields and a clear imperative not to round values. Every sentence earns its place, with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a full output schema and 100% parameter schema coverage, the description still adds valuable context: ordering, verbatim usage, and non-detect handling. It is complete enough for an agent to call the tool correctly without additional information.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents both parameters with examples ('MB2', 'PFOS') and the optionality of analyte, so the baseline is 3. The description adds no new parameter-level semantics beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Return the lab results for one monitoring well, oldest round first.' This clearly differentiates it from sibling tools like lookup_limit and search_guidelines, which serve distinct purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description makes the intended use clear: retrieve lab results for a single well, with optional analyte filtering. It does not explicitly name alternatives or when not to use this tool, but the context is sufficient for correct selection among the listed siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_review_itemGet one review itemA
Read-onlyIdempotent

Return one review item from tidy_lab_files in full, with every evidence row quoted exactly as it is in its file.

Use it to show the person the lab, chain-of-custody and field-sheet lines behind a finding, with their row numbers, the rule and its source, and the judgement left to the scientist. Numbers come from tidy_lab_files for the same site.

ParametersJSON Schema
NameRequiredDescriptionDefault
siteYesSite id. Only the synthetic site 'FDS-01' is available.
numberYesReview item number from tidy_lab_files, starting at 1.

Output Schema

ParametersJSON Schema
NameRequiredDescription
itemYes
noteYes
siteYes
of_totalYesHow many review items the site has.
syntheticYes
source_linesYesEvery evidence row, as written in its file.

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already communicate that the operation is read-only, idempotent, and non-destructive. The description adds meaningful behavioral detail beyond that: results quote evidence rows exactly, include row numbers and rule source, and leave judgement to the scientist. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded with the core behavior. The second sentence adds useful context about what the review item containsolaris. The final sentence is slightly redundant but does reinforce the data source relationship.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema and strong annotations, the description is sufficient for an agent to understand what the tool returns and when to use it. It could go further by explicitly naming sibling tools or exclusions, but nothing critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the description does not need to restate parameter meanings. It does add a small clarifying note that numbers come from tidy_lab_files for the same site, but most parameter semantics are already carried by the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'Return one review item from tidy_lab_files in full'. It further specifies the key behavior, 'every evidence row quoted exactly as it is in its file', which clearly distinguishes this from a generic list or summary tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a clear use case: 'Use it to show the person the lab, chain-of-custody and field-sheet lines behind a finding'. It explains what context the returned item provides, though it does not explicitly name alternatives or state when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

lookup_limitLook up a guideline limitA
Read-onlyIdempotent

Look up the drinking-water guideline value for one analyte under one rule.

Returns the value and unit, whether the rule sets it for one analyte or for the sum of PFOS and PFHxS (applies_to), what to compare with it for the analyte asked about (compared_quantity), the source document, table and page, the rule's WA adoption status, and a one-line note in plain English. Under 'nemp-3.0', 0.07 ug/L applies to PFOS alone, PFHxS alone and their sum (Table 4, footnote a): asking for either returns that value, with the footnote quoted. Under 'current', there is no sum value. Drinking water is the only scenario loaded.

ParametersJSON Schema
NameRequiredDescriptionDefault
ruleYes'nemp-3.0' (PFAS NEMP 3.0, March 2025, named as adopted on the WA government page) or 'current' (ADWG values updated 2025, as listed in PFAS NEMP 3.1).
analyteYes'PFOS', 'PFHxS', 'PFOA', 'PFBS', or 'PFOS+PFHxS' for the sum of PFOS and PFHxS.

Output Schema

ParametersJSON Schema
NameRequiredDescription
noteYesOne line in plain English.
pageYes
ruleYes
unitYes
tableYes
valueYes
analyteYesThe analyte you asked about.
caveatsYes
scenarioYes
rule_nameYes
wa_statusYes
applies_toYesWhat the rule sets the value for: 'sum' means PFOS and PFHxS added together (under NEMP 3.0, Table 4, footnote a, the same value also applies to PFOS on its own and PFHxS on its own); 'single' means one analyte.
page_basisYes'PDF page' or 'printed page'.
source_documentYes
compared_quantityYesWhat to compare with the value for the analyte asked about: 'sum of PFOS and PFHxS', or one analyte such as 'PFOS', or 'PFOS on its own' when a sum value also applies to PFOS alone.

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, so the description need not restate those. It adds useful context: the special behavior for 'nemp-3.0' where PFOS and PFHxS have a shared sum value, and that 'Drinking water is the only scenario loaded.' It also clarifies what the return includes (applies_to, compared_quantity, source, etc.), which goes beyond the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact, well-organized, and front-loads the core purpose. It uses paragraphs to separate general behavior from specific rule nuances and provides an example. No unnecessary repetition; every sentence adds information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given a simple two-parameter tool with a detailed output schema, the description covers the essential context: rule options, analyte values, specific edge cases (sum value, only drinking water), and return fields. It does not delve into the output schema's structure, but that is not required as the output schema is provided. Minor gaps: it doesn't mention what happens if an invalid analyte is passed, but the schema enforces allowed values.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already provides full descriptions for both parameters (100% coverage). The description adds value by explaining the implications of each rule and giving a concrete example of the sum behavior. It also clarifies the analyte's meaning in context (e.g., 'PFOS+PFHxS' as a sum), which enhances the schema's literal list.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a clear verb-object pair: 'Look up the drinking-water guideline value for one analyte under one rule.' It specifies the resource (guideline limits), the scope (one analyte, one rule), and what it returns (value, unit, applies_to, compared_quantity, source, adoption status, note). It distinguishes itself from siblings like compare_rules and check_paragraph by focusing on single-rule lookup.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly explains the two rule options ('nemp-3.0' and 'current') and the analyte values accepted, providing concrete examples. It notes that 'Under 'current', there is no sum value' and that 'Drinking water is the only scenario loaded,' which prevents incorrect usage. It does not explicitly list alternatives, but the sibling names (compare_rules, search_guidelines) suggest alternatives exist; however, it gives enough context for when to use this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_guidelinesAsk the guidelinesA
Read-onlyIdempotent

Search the indexed public guidance (ASC NEPM Schedule B1, PFAS NEMP 3.0, DWER guidelines on assessing and on reporting and classifying contaminated sites, and the ADWG PFAS fact sheet) and return the passages that match.

Each passage gives the document title, edition and date, its WA status, the PDF page and the printed page number, the nearest section heading and the headings above it, a short excerpt (shorter for documents whose licence allows only brief quotes, always with the notice) and the official link. At most one passage comes from any one page. When no passage covers enough of the question, the result is 'not covered' with the reason, instead of weak matches.

Never take a guideline value from an excerpt: tables extract badly. For PFAS drinking-water values use lookup_limit; for any other value, read it on the cited page. The tool never says which document or edition applies to a site. PFAS NEMP 3.1 is not indexed (its host was unreachable when the index was built).

ParametersJSON Schema
NameRequiredDescriptionDefault
kNoHow many passages to return, 1 to 10. Default 5.
questionYesA question in plain English, for example 'What must a detailed site investigation report include?' or 'Can PFAS-contaminated soil be reused?'. At most 500 characters.

Output Schema

ParametersJSON Schema
NameRequiredDescription
notesYes
statusYes
passagesYes
questionYes
explanationYesWhat the result means, in plain English.
searched_termsYesThe terms searched for, after synonym expansion.
indexed_documentsYes
unavailable_documentsYesDocuments listed in the corpus but not indexed, and why.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark the tool read-only and idempotent, but the description adds substantial behavioral detail beyond that: at most one passage per page, the 'not covered' outcome with reasons instead of weak matches, the fact that PFAS NEMP 3.1 is not indexed, and the warning about table extraction. This is rich, non-obvious behavior that structured fields alone would not convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is longer than average but every sentence carries a distinct piece of information: scope, return fields, deduplication, not-covered behavior, value-extraction caveat, and indexing limitation. It is front-loaded with purpose and return behavior before caveats. Slight redundancy with the schema's plain-English example prevents a perfect score.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a search tool with two simple parameters, an output schema, and read-only annotations, the description covers everything an agent needs: what corpus is searched, what each passage contains, how non-answers are reported, which sibling to use for values, and what the tool cannot tell the user. Nothing critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents 'question' and 'k' with examples and constraints. The description reinforces the plain-English question format but adds little new parameter-level meaning beyond what the schema provides. It compensates with usage-level guidance, not parameter semantics, so the baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Search the indexed public guidance... and return the passages that match.' It names the exact document set (ASC NEPM, PFAS NEMP 3.0, DWER guidelines, ADWG fact sheet), so an agent can identify the tool's scope. It also differentiates from lookup_limit, which is explicitly routed to for PFAS drinking-water values.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells the agent when not to trust the tool's excerpts for numeric values: 'Never take a guideline value from an excerpt: tables extract badly.' It names the alternative tool and condition: 'For PFAS drinking-water values use lookup_limit.' It also states a hard limitation, 'The tool never says which document or edition applies to a site,' giving clear when-not-to-use context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

show_redactionsShow redactionsA
Read-onlyIdempotent

Show what the redaction guard rail has replaced in this session: placeholder names and counts only.

Lists each placeholder in use (for example [CLIENT-1], [ADDRESS-1], [LOT-1], [EMAIL-1], [PHONE-1]) with how many times it replaced an identifier, the totals per type, how many identities the local identifier file defines per type (none when no identifier file is loaded, in which case names are not redacted), and the built-in patterns. It never shows the raw values: they stay in the memory of the machine running this server. Use the placeholders as they are in any text you write; the person's local export puts the real values back.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
audit_logYes
explanationYes
placeholdersYes
identifier_fileYes
built_in_patternsYes
replacements_by_typeYes
identities_configuredYesIdentities in the identifier file, per type.

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds valuable behavior beyond that: it never shows raw values, states values stay in server memory, and explains the conditional behavior when no identifier file is loaded. This is meaningful, non-redundant transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The first sentence front-loads the core purpose and limitation, followed by a compact, ordered explanation of output content and one usage tip. It is longer than minimal but every sentence contributes necessary detail about placeholders, counts, identities, patterns, and privacy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless tool with safe/idempotent annotations and an output schema, the description fully explains what is returned, what is not returned, and conditional behavior. An agent has everything needed to call and interpret the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the description needs to explain none. Baseline 4 applies because there is no parameter surface to document; the description correctly omits parameter details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description states a specific verb ('Show') and resource ('what the redaction guard rail has replaced in this session'), and explicitly limits output to placeholder names and counts only. Includes concrete placeholder examples and distinguishes itself from browsing/searching sibling tools by focusing on redaction state.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use the tool (when placeholders are needed for writing text) through 'Use the placeholders as they are in any text you write,' but it does not explicitly state when to use this tool versus sibling tools or when not to use it. Usage guidance is implied, not explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tidy_lab_filesTidy lab resultsA
Read-onlyIdempotent

Tidy one site's lab results, chain of custody and field sheet into one table, and list what needs review.

Converts units exactly (ug/L for water, mg/kg for soil; ng/L, mg/L and the spellings ug/L, µg/L, μg/L are understood), reads every date day first, matches sample ids across the three files, and runs six QA checks: sample ids, units, field duplicate RPD, holding times to extraction, blank detections linked through the field sheet's equipment column, and LOR against investigation levels (soil HIL A; both drinking-water rules for water, side by side).

Returns counts by matrix and analyte, numbered review items (what was found, the evidence rows, the rule and its source, and what the scientist decides), everything that was checked and not flagged, and what was not checked. The combined rows (one per lab result, with file and row numbers) are included only with include_rows=true; row_count always says how many there are. Show the person every review item and the checked list; never describe the result as "no issues". The tool never merges, rejects or corrects a result, and a guideline value is an investigation level, not a finding that a site is contaminated. The data is synthetic.

ParametersJSON Schema
NameRequiredDescriptionDefault
siteNoSite id. Only the synthetic site 'FDS-01' is available.FDS-01
include_rowsNoInclude the full combined table (73 rows for FDS-01, about 25 KB). Default false: review items, checks and counts only.

Output Schema

ParametersJSON Schema
NameRequiredDescription
rowsYesThe combined table, one row per lab result. Empty unless include_rows is true; row_count says how many rows there are.
siteYes
filesYes
notesYes
checksYesEvery check that ran, what it looked at and what it raised.
countsYesResults by matrix and analyte.
resultsYes
samplesYesLab samples with results.
summaryYes
settingsYes
row_countYesHow many combined rows the site has, whether or not they are included.
syntheticYes
not_checkedYes
review_itemsYes
rows_includedYes
checked_not_flaggedYes

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes far beyond the readOnly/idempotent annotations: it explains exact unit conversions, date parsing, QA checks, output contents, the include_rows behavior, and important caveats such as 'never merges, rejects or corrects a result' and 'never describe the result as no issues.' This gives the agent a detailed behavioral contract.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but well-structured, with the core purpose front-loaded and supporting details organized logically. Every sentence contributes meaningful information about behavior, output, limits, or usage, with no filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with this complexity, the description covers the transformation, QA rules, return semantics, parameter behavior, and safety limitations. The presence of an output schema means return values need not be spelled out further; an agent has everything needed to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents both parameters fully, so the baseline is 3. The description adds useful semantic detail beyond the schema, especially that include_rows=true returns the full combined table (73 rows, about 25 KB) and that site only supports the synthetic site FDS-01.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Tidy one site's lab results, chain of custody and field sheet into one table, and list what needs review.' This clearly distinguishes the tool from siblings like get_results or lookup_limit by stating the combining/tidying operation and the review-item output.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context for when to use the tool: whenever a consolidated lab table and review items are needed for one site. It does not explicitly name alternatives or state when not to use it, but the unique purpose is obvious enough that an agent should select it correctly without further guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 9 tool updatesv0.1.0
    • First observedcheck_paragraph
    • First observedcompare_rules
    • First observedfill_numbers
    • First observedget_results
    • First observedget_review_item
    • First observedlookup_limit
    • First observedsearch_guidelines
    • First observedshow_redactions
    • First observedtidy_lab_files

TDQS

A4.4/5.0

Scored across 9 tools

Disambiguation5/5

Each tool owns a distinct task: data tidy/QA, review-item drill-down, per-well retrieval, limit lookup, rule comparison, paragraph checking, number filling, document search, and redaction reporting. The closest pairs (get_results vs tidy_lab_files and lookup_limit vs search_guidelines) are clearly separated by workflow step and output type.

Naming Consistency5/5

All nine tool names follow the same lowercase snake_case verb_noun pattern, from tidy_lab_files to show_redactions. Minor plural/singular variation such as get_results vs get_review_item is natural and does not break the convention.

Tool Count5/5

Nine tools is well within the ideal range and each one maps to a distinct step in the evidence-to-report workflow. The count feels deliberately scoped rather than padded or thin.

Completeness4/5

The core workflow is well covered: data ingestion/QA, review item drill-down, results retrieval, guideline comparison, report number filling, paragraph verification, document search, and redaction reporting. The main gap is that soil guideline values such as HIL A are only searchable as document passages rather than available through a structured lookup like the drinking-water values, so some soil reporting tasks require manual reading of cited pages.

Related MCP Connectors

Related MCP Servers

  • F
    license
    Not graded
    quality
    A
    maintenance
    Provides tools to analyze local PDFs and CSVs (page count, text search, scoring, column stats) with strict refusal to guess ambiguous data. Requires a paid license.
    -
  • F
    license
    B
    quality
    A
    maintenance
    Enables local, offline document extraction and manipulation—PDF first but also HTML, DOCX, XLSX, PPTX, EML, EPUB, Markdown, and plain text—through tools for probing, locating, extracting, converting, assembling, OCR, protecting, and redacting documents, with nothing leaving the machine.
    7
    -
  • A
    license
    B
    quality
    C
    maintenance
    Enables auditable thermophysical data curation by validating molecules, predicting properties with JR-MPNN and UManSysProp, and normalizing, deduplicating, scoring, and triaging records with an audit trail.
    6
    GPL 3.0
  • F
    license
    Not graded
    quality
    C
    maintenance
    Enables manuscript researchers to conduct retrieval and scoring searches, inspect human-readable results, record inline reviews, rescore cached corpora, and export reports or ChatGPT-ready research packets.
    -