ReviewTrace

Review feedback should not disappear into a revised document.

ReviewTrace maps every reviewer comment to the change that addressed it and shows the evidence.

reviewed.docx  +  revised.docx   ──►   ReviewTrace   ──►   Revision Compliance Report

A supervisor, editor, peer reviewer or client asks for changes. You send back a revised document. ReviewTrace answers the question that follows: were those changes actually addressed, and where is the proof?

It is an auditor, not a Word editor. It reads your files, never modifies them, and runs entirely on your machine.

What it does

• Reads Word comments (author, date, threads, resolved flag, anchored text, section and neighbouring paragraphs), tracked insertions/deletions, tables, footnotes and reference lists.

• Extracts issues from a separate reviewer report (.docx, .md, .txt) or from a hand-written issues.yml.

• Compares the original and revised documents structurally (sections → paragraphs → words; moved text is recognised as moved, not as an unrelated delete and insert).

• Assesses every issue with explainable rules and reports a status, a separate confidence, and the evidence behind both, with logical locations (Section 2.1, paragraph 7; never page numbers, which DOCX does not store).

• Produces a terminal report, Markdown, JSON, CSV, a formal traceability matrix and a grounded draft response-to-reviewers letter.

• Exposes the same engine as an MCP server.

Related MCP server: zetesis

Why it exists

"The comment disappeared" is not evidence. Neither is "related words appear somewhere in the revised document", "the document changed nearby", or a confident-sounding paragraph from a language model. ReviewTrace never marks a comment RESOLVED on any of those grounds. Every status must be backed by evidence, and ambiguity becomes NEEDS_REVIEW instead of being forced into resolved/unresolved.

ReviewTrace also keeps three things apart that are easy to confuse:

Example output

The repository ships a small thesis chapter (examples/thesis/). The supervisor's four comments:

1. Explain the major constructs of the Health Belief Model.

2. Show how the model relates to the present study.

3. Add recent references.

4. Remove the repeated paragraph in Section 2.4.

The revision fully addresses comment 1, only loosely touches comment 2, adds one reference for comment 3, and leaves the duplicated paragraph in place. Running

reviewtrace audit examples/thesis/reviewed.docx examples/thesis/revised.docx

prints (abridged to two of the four issues):

REVIEWTRACE
word comments · semantic assessment: disabled

  Review issues:    4
  RESOLVED          1
  UNRESOLVED        1
  NEEDS REVIEW      2
────────────────────────────────────────────────────────────
┌─ RT-001 ────────────────────────────────────────────────────────────────────────────────┐
│ Reviewer: Prof. Adaeze Okafor                                                           │
│ “Explain the major constructs of the Health Belief Model.”                              │
│ Anchored text: “The Health Belief Model”                                                │
│                                                                                         │
│ RESOLVED   Confidence: MEDIUM   (human review recommended)                              │
│ Reviewer's tool: n/a  ·  Document: changed  ·  Basis: deterministic/lexical-coverage    │
│                                                                                         │
│ Evidence:                                                                               │
│ + 67 words of new text at Section 2.1 Theoretical Framework, paragraph 7; contains the  │
│ requested terms (constructs)                                                            │
│ + 2.1 Theoretical Framework grew from 9 to 90 words (+81)                               │
│ + tracked insertion by A. Student on 2024-05-12                                         │
│                                                                                         │
│ Revised location:                                                                       │
│ Section 2.1 Theoretical Framework, paragraph 7                                          │
└─────────────────────────────────────────────────────────────────────────────────────────┘
┌─ RT-004 ────────────────────────────────────────────────────────────────────────────────┐
│ Reviewer: Prof. Adaeze Okafor                                                           │
│ “Remove the repeated paragraph in Section 2.4.”                                         │
│ Anchored text: “Evidence from the region is mixed and the empirical base remains        │
│ limited, which motivates the present study.”                                            │
│                                                                                         │
│ UNRESOLVED   Confidence: HIGH                                                           │
│ Reviewer's tool: n/a  ·  Document: unchanged  ·  Basis: deterministic/duplicate-check   │
│                                                                                         │
│ Evidence:                                                                               │
│ - the duplicated paragraph (2 copies at Section 2.4 Summary of the Literature,          │
│ paragraphs 14–15) still appears 2 times in the revised document (Section 2.4 Summary of │
│ the Literature, paragraphs 16–17)                                                       │
│ - duplicate of paragraph 14 still present                                               │
│                                                                                         │
│ Revised location:                                                                       │
│ Section 2.4 Summary of the Literature, paragraphs 16–17                                 │
│                                                                                         │
│ Remaining action:                                                                       │
│ Remove the repeated paragraph.                                                          │
└─────────────────────────────────────────────────────────────────────────────────────────┘

The four results are RESOLVED, NEEDS_REVIEW, NEEDS_REVIEW and UNRESOLVED:

• RT-001 is RESOLVED with MEDIUM confidence, and says why. New text containing the requested concept ("constructs") was added in the section the comment points to. That is lexical evidence; whether the explanation is correct and sufficient has not been assessed, so the finding is flagged for human review.

• RT-002 is NEEDS_REVIEW. Text was added, but it never says how the model relates to the study. It may be worded differently, so ReviewTrace declines to decide.

• RT-003 is NEEDS_REVIEW. One 2024 reference was added. "Recent" is not an objective criterion, so adequacy is a human judgement. Given a criterion such as "at least two references published since 2020" the same comment is decided deterministically.

• RT-004 is UNRESOLVED with HIGH confidence. The identical paragraph still appears twice.

Installation

Requires Python 3.10 or newer.

git clone https://github.com/seun-john/reviewtrace.git
cd reviewtrace
pip install -e .            # the CLI and library
pip install -e ".[mcp]"     # adds the MCP server
pip install -e ".[dev]"     # adds pytest, Ruff, mypy and the MCP SDK for development

The project uses standard pyproject.toml metadata, so uv works too (uv pip install -e ".[dev]"); CI and the test suite use plain pip.

Check it works:

reviewtrace --help

Quick start

# what did the reviewer write, and where?
reviewtrace comments examples/thesis/reviewed.docx

# were the comments addressed?
reviewtrace audit examples/thesis/reviewed.docx examples/thesis/revised.docx

# keep everything: JSON, Markdown, CSV matrix and a draft response letter
reviewtrace audit examples/thesis/reviewed.docx examples/thesis/revised.docx --out-dir out/

The commands in this README run against the files in examples/; they are exercised by the test suite. python scripts/build_examples.py regenerates those files.

Commands

audit accepts -f terminal|markdown|json|csv, -o FILE, --out-dir DIR, -v, and --fail-on unresolved|not-resolved (exit status 3, for CI). It refuses to write over an input file.

Word-comment workflow

reviewed.docx contains the reviewer's comments; revised.docx is the response.

reviewtrace audit reviewed.docx revised.docx

For each comment ReviewTrace records the comment text, author, initials, date, the anchored text, the paragraph and section containing it, the neighbouring paragraphs, reply threads and Word's own resolved flag. A comment that just says "Explain this." is therefore assessed against the sentence it was attached to, not against nothing.

It reads word/comments.xml, commentsExtended.xml, commentsIds.xml, footnotes.xml, endnotes.xml and the comment-range and tracked-change markup in document.xml directly, because python-docx exposes none of that. python-docx is used to open and validate the package and to resolve styles.

Word's resolved flag is recorded as word_comment_resolved and shown next to the result. It is never used to decide the status: a reviewer can resolve a comment whose text was not changed, and an open comment may have been fixed in a later clean version.

Separate-review-report workflow

reviewtrace audit --original original.docx --review reviewer_comments.docx --revised revised.docx
reviewtrace audit --original original.docx --review reviewer_comments.md   --revised revised.docx

The report can use numbered lists, bullets, Comment 3: labels, a comments table, or plain prose (in which case only sentences that read as requests are extracted, and the rest are counted and reported). Headings such as "Reviewer 2" and "Major comments" set the reviewer and severity. A quoted passage or a section reference in an item (In Section 3.3, "…") is used to locate what it refers to.

Extract first, correct the issues by hand, then audit:

reviewtrace extract reviewer_comments.md -o issues.yml
reviewtrace audit --original original.docx --issues issues.yml --revised revised.docx

version: 1
issues:
  - id: RT-001
    reviewer: Supervisor
    comment: >
      Explain how the sample size was calculated.
    anchor: >
      The study included 396 respondents.
    location:
      section: Sample Size
    category: EXPLAIN
    severity: major

Unknown keys are rejected (a typo such as comments: is an error, not silently ignored). See docs/issues-file.md.

Compound and duplicate comments

"Define the theory, explain its assumptions and show how it relates to this study." is three requirements. It becomes RT-014 with sub-requirements RT-014.1–RT-014.3, each assessed on its own. The parent is RESOLVED only if every part is; if only one part was addressed it is PARTIALLY_RESOLVED or NEEDS_REVIEW. Probable duplicate comments are flagged (possible_duplicates) but never merged, so each keeps its provenance.

Traceability matrix

reviewtrace matrix out/audit.json                 # Markdown
reviewtrace matrix out/audit.json -f csv -o matrix.csv

Columns: ID, Reviewer, Comment, Anchored Context, Requested Action, Status, Confidence, Evidence, Revised Location, Remaining Issue. The CSV and JSON forms add Reviewer Marked Resolved, Document Changed, Assessment Source, Method and Needs Human Review. Excerpt for the demo:

CSV cells that begin with = + - @ are prefixed with an apostrophe so a reviewer's comment can never execute as a spreadsheet formula.

Response-to-reviewers generation

reviewtrace response out/audit.json -o response.md

Every sentence comes from a finding; nothing is invented. Only RESOLVED findings are answered "Addressed."; PARTIALLY_RESOLVED and UNRESOLVED say "Further revision required" and name what is outstanding. Excerpt:

### Comment 4

Remove the repeated paragraph in Section 2.4.

**Response:**

Further revision required.

**Not yet done:**

Remove the repeated paragraph.

The command also reports which responses need author attention before the letter is sent (any finding that is not a HIGH-confidence resolution).

Status definitions

Examples of what is decided deterministically: removal of anchored or duplicated text, a heading rename (Change 'A' to 'B'), a numeric or text replacement (should be 396, not 384, including in a named table), a paragraph moved to a named section, and reference counts against explicit criteria (at least two … since 2020). Requests that ask for new explanatory content are checked lexically and otherwise sent to NEEDS_REVIEW.

Confidence definitions

Confidence is separate from status. It reflects how strong the rule behind the status is and how firmly the request could be tied to a place in the document.

RESOLVED is never LOW: if the evidence would only support LOW confidence the status becomes NEEDS_REVIEW. Anything other than a HIGH-confidence RESOLVED/UNRESOLVED is marked requires_human_review. See docs/statuses-and-confidence.md.

Semantic assessment and its limits

Many comments are inherently semantic ("Use a more convincing argument"). ReviewTrace works without any language model and by default semantic assessment is disabled: such comments become NEEDS_REVIEW (when changes exist at the location) or NOT_ASSESSABLE (when there is nothing to examine).

The library defines the SemanticReviewer interface for an optional, explicitly enabled reviewer. Design rules, enforced by merge_assessment:

• The reviewer is never asked "was this resolved? yes/no". It receives the request, the original and revised context and the detected changes, and must return structured output: status recommendation, explanation, evidence references, missing requirements and confidence.

• A semantic RESOLVED/PARTIALLY_RESOLVED/UNRESOLVED must cite evidence that exists; otherwise it is downgraded to NEEDS_REVIEW.

• Semantic confidence is capped at MEDIUM, the result is always flagged for human review, and it cannot override a HIGH-confidence deterministic result.

• Reports mark the source (assessment_source: semantic vs deterministic) and keep the deterministic status alongside.

Version 0.1.0 ships the interface and the merge policy only. It contains no provider adapter (Anthropic, OpenAI, Gemini, local or OpenAI-compatible endpoints are planned), and the CLI has no option to enable one. See docs/semantic-review.md.

Even without a model, note what "lexical" evidence is: a check that the words the reviewer asked for appear in newly added text at the right place. It rules out unrelated edits; it does not prove the new text is good.

Privacy

ReviewTrace is local-first.

• Files stay on your machine. No document content is uploaded.

• No telemetry.

• No external LLM or network calls. No API key is needed or read. (The test suite asserts an audit runs with network connections blocked.)

• Source documents are opened read-only and never modified; every audit records each input's SHA-256 so you can verify that afterwards.

• Any future semantic integration must be configured explicitly.

MCP usage

pip install "reviewtrace[mcp]"
reviewtrace mcp            # or: python -m reviewtrace.mcp.server

Example client configuration:

{
  "mcpServers": {
    "reviewtrace": {
      "command": "reviewtrace",
      "args": ["mcp"],
      "env": { "REVIEWTRACE_MCP_ROOT": "/path/to/my/documents" }
    }
  }
}

Tools: extract_review_comments, extract_review_issues, compare_documents, audit_revision, inspect_issue, generate_traceability_matrix, generate_response_to_reviewers. They are thin wrappers over the same engine the CLI uses and return structured data. Set REVIEWTRACE_MCP_ROOT to confine file access to one directory. The server was developed against MCP Python SDK 2.x (MCPServer); a fallback for the 1.x FastMCP API exists but is not covered by the test suite.

Supported file types

Known limitations

• English-language heuristics. Request classification and term matching assume English.

• Lexical, not semantic, checking for content requests. A RESOLVED from lexical coverage means the requested words appear in new text at the right location, nothing more. It is capped at MEDIUM and flagged for review. A paraphrase that avoids the reviewer's words gives NEEDS_REVIEW, not a false UNRESOLVED, but a shallow sentence that uses the right words can give RESOLVED.

• Requests without a location. If a comment cannot be tied to a place (no anchor, section or distinctive topic words) it never reaches RESOLVED.

• No formatting or citation-style comparison (fonts, spacing, APA/Harvard conformity): reported as NOT_ASSESSABLE. Images are not compared; figure requests compare captions only.

• Not read: text boxes, headers/footers, comments in headers/footers, equations, PDFs.

• Logical locations, not pages. Paragraph numbers count non-empty body paragraphs in order; they are not shown by Word. Section numbers come from heading text.

• Matching is text-similarity based. A heavily rewritten paragraph can be reported as a removal plus an addition.

• Comment threads use Word's commentsExtended metadata; documents saved by tools that do not write it show no resolved state or replies.

• Testing. The suite was run on Python 3.14 (Windows). Python 3.10–3.13 are targeted by CI but were not run locally.

Roadmap

0.2 PDF reviewer-annotation import, better compound-comment decomposition, optional semantic providers, HTML report, GitHub Action.

0.3 Google Docs review support, Microsoft Word cloud integration, journal response-letter templates, configurable institutional review workflows.

Companion ecosystem. ReviewTrace should later delegate specialised checks. "Ensure all references use APA 7th" would go to a citation checker (for example CiteProof) instead of being re-implemented here; SpecGuard, DocGuard and CleanOutput are other candidates.

Deliberately out of scope for 0.1: a GUI, hosted service, accounts, PDF annotation extraction, automatic rewriting, accepting/rejecting tracked changes, and page-level visual comparison.

Contributing

See CONTRIBUTING.md. Security issues: SECURITY.md.

pip install -e ".[dev]"
pytest
ruff check . && ruff format --check .
mypy

Licence

MIT. See LICENSE.