위클리-검증 (weekly-verify-mcp)

Compares the written weekly work report (.docx) against the source data (.xlsx) to find wrong values, missing items, and descriptions that contradict the data, then returns them to a human. A personal MCP server.

It does not fix. It only points out. Fixing is done by a human.

See docs/검증프로세스.md for the full design.

Current Status — P0~P5 Complete · Registered with Codex

Related MCP server: phionyx-pipeline-mcp

The 4 Verification Layers

The entire design of this tool comes from this table.

• L1 alone is not enough. If you copy the value of L-18 into the L-19 row, both values exist in the source, so L1 passes. Only L2, which also checks the task code, catches it.

• L3 is the reason this tool exists. What people usually get wrong in weekly reports is not numbers but items they did not write. A wrong number is visible, but a missing row is not.

• L4 structurally cannot be an error. harness.Finding enforces this, and tests/test_harness.py locks it in. Giving blocking authority to a check you are not sure about would block normal reports, and people would turn the checker off.

The 6 Gates

What evals/weekly-verify.yaml requires of the engine. Scored with uv run python scripts/run_eval.py.

Coordinate requirements differ per layer (G5)

If you force-fill coordinates that cannot be filled, a person will open the wrong place and believe they have verified it.

N/A is not a pass

Gates of the "don't do bad things" kind (G2·G3·G5) are satisfied automatically if the engine produces nothing. If you mark that as PASS, an engine with no features at all would look like it passed 5 out of 6. So when there is no basis to evaluate, mark it N/A, and N/A does not count as a pass.

Current scoring results (as of P3 — all passed)

C01 수치 조작 [OK] 1/1    C05 지연 누락  [OK] 3/3    C09 반올림 [OK] 0/0
C02 날짜 오기 [OK] 1/1    C06 미제출 은폐 [OK] 3/3    C10 정상   [OK] 오탐 0
C03 값 오배치 [OK] 1/1    C07 역행 무시  [OK] 1/1
C04 없는 과제 [OK] 1/1    C08 서술 상충  [OK] 0/0

[PASS] G1_검출률  11/11 (100%)   [PASS] G4_속도   최장 0.1초 / 제한 60초
[PASS] G2_오탐   error 0건       [PASS] G5_근거   좌표 누락 0건 / error 12건
[PASS] G3_심각도  위반 0건        [PASS] G6_계층정확성  11/11 (100%)

G5 examined 12 errors, but the expectation is 11. This is because C01's contamination changed one cell in each of two tables, and both must be fixed, so it is not a duplicate.

Folders

weekly-verify-mcp/
├─ docs/검증프로세스.md      설계 문서 (13개 절)
├─ src/weekly_verify/
│  ├─ harness.py             도구 응답 규약 · Finding · 두 좌표계
│  ├─ sources.py             원본 xlsx 파서 + DATA_ROOT 가드
│  ├─ findings.py            판정 5종 (원본만 보고 계산)
│  ├─ report.py              보고서 docx 파서
│  ├─ claims.py              주장 추출 (규칙 기반, LLM 미사용)
│  ├─ values.py              L1 존재 · L2 정합 + 억제 규칙 3종
│  ├─ completeness.py        L3 누락 검사
│  ├─ narrative.py           L4 서술 상충 (warning 고정)
│  ├─ verify.py              진입점 — 네 계층 조립 + 계층 간 중복 제거
│  ├─ record.py              검증 결과 문서 생성·저장 + 승인 토큰
│  └─ server.py              MCP 도구 10개 · 리소스 2 · 프롬프트 1
├─ scripts/
│  ├─ make_fixtures.py       오염 시나리오 10종 생성
│  ├─ check_fixtures.py      픽스처 자기검증 (30개 검사)
│  ├─ run_eval.py            채점기 (게이트 6종)
│  ├─ smoke_stdio.py         stdio 기동 + 하네스 검사 15종
│  └─ verify_registration.py 설정 파일의 절대경로로 기동 확인
├─ tests/
│  ├─ test_harness.py        하네스 규약 잠금 (16개)
│  ├─ test_eval_contract.py  채점기 검증 — 가짜 엔진 7종 (20개)
│  ├─ test_findings.py       판정 엔진 ↔ 정답지 1건씩 대조 (27개)
│  ├─ test_completeness.py   L3 누락 검사 (31개)
│  ├─ test_values.py         L1·L2 + 억제 규칙 (53개)
│  ├─ test_narrative.py      L4 서술 상충 (43개)
│  └─ test_server_contract.py docstring ↔ 실제 규칙 잠금 (63개)
├─ data/
│  ├─ 원본/                  ← 서버가 읽는 유일한 곳
│  │  ├─ 마스터_주간보고_누적.xlsx
│  │  └─ 제출_2026-W35/ (담당자 6명)
│  ├─ 보고서/ C01~C10.docx   검증 대상
│  └─ 출력/                  검증 결과 기록 (저장 산출물)
├─ config/                   등록 설정 + README
├─ templates/                검증 결과 문서 템플릿
└─ evals/                    🚫 서버 접근 금지 — evals/README.md 참조
   ├─ 정답지_2026-W35.xlsx
   ├─ fixtures_manifest.json 픽스처가 담고 있는 사실
   └─ weekly-verify.yaml     엔진에게 요구하는 정책

The reason manifest and suite are separated is that their concerns differ — the former is a fact created by the generator, so it changes together with the docx, while the latter is a policy set by a human, so it is unrelated to the docx. If merged into one file, expectations would appear in two places and drift.

The 10 Contamination Scenarios

data/보고서/C01~C10.docx. The expected detection for each file is in evals/fixtures_manifest.json.

C05 · C06 · C07 are the core

These three maintain internal consistency within the contaminated report. While deleting the 3 delayed items, the summary table count was also changed from 6 to 3. So reading only the report shows no contradiction — it only becomes visible when compared against the source. The deleted L-02 · L-08 · L-19 are exactly the 3 items whose owners left the 이슈리스크 (issue/risk) field blank.

C09 · C10 are just as important

They are false-positive prevention tests. No one uses a checker that raises error on a normal report.

Suppression Rules — report one mistake as one finding

Even if contamination changes only one place, derived values cascade into errors. C02 only pushed 계획완료일 (planned completion date) back one day, but 경과일 (elapsed days) also became wrong; C03 only changed 금주진척률 (this week's progress rate), but 증감 (change) also became wrong. If you point those out as-is, one miswritten value becomes two findings, forcing a person to search two places and making it impossible to gauge severity by the number of findings.

Suppression is not unconditional. Each rule is tested as a pair — the suppressed side and the non-suppressed side. Suppression rules quietly collapse the moment you think "it would be nice if this were flagged too."

Columns not verified are declared

5 types are listed in claims.미검증_열 (unverified columns) with reasons (상태 (status) is a judgment label so it belongs to L3; the O (완료 2026-08-20) form in the regression table is free-form so comparison is unstable, etc.). Silently skipping would read as "everything was verified," but there are columns that were actually not examined.

Running

uv sync --extra dev

uv run python scripts/make_fixtures.py

uv run python scripts/check_fixtures.py

uv run python scripts/run_eval.py

uv run python -m pytest -q

uv run python scripts/smoke_stdio.py

uv run python scripts/verify_registration.py

The 10 MCP Tools

Fast path: tool 9 alone runs all four layers. The intermediate tools are used to show a person "why it was judged that way."

Resources template://verification · report://{보고서}, prompt verify_weekly_report.

Saving is a 'finding record' — error does not block it

The design draft said "refuse to save if there is even one error," but that was a wrong rule. What you save is a record of findings, and the time you most need a record is when there are errors. Blocking then would make the tool useless.

What should be blocked is something else.

The conclusion is one of 통과 · 보류 · 반송 (pass · hold · return). The approval token is computed from the findings, so if even one finding changes, the token mismatches and saving is rejected.

These two conditions are stated in the tool description of save_approved_verification, and tests/test_server_contract.py compares the description against the actual code to lock it in — since the model does not read code, a rule not in the description does not exist for the model.

Registration (P5)

Codex registration complete — [mcp_servers.weekly-verify] is in ~/.codex/config.toml (backup: config.toml.bak-2026-08-26). Restart Codex and the tools will appear.

Claude Desktop has %APPDATA%\Claude as a partially blocked path, so the tool cannot write into it. Merge the contents of config/claude_desktop_config.example.json manually — this is an intentional block and is not bypassed.

command must be an absolute path. Desktop apps do not inherit the login shell's PATH, so if you write only uv, it works in the terminal but the server will not start in the app.

uv run python scripts/verify_registration.py

It checks 3 things: 2 example configs and the actually registered live config. Even if the examples are correct, if the actual registration is wrong, the tools will not appear in the app.

About the data

All data in data/ is practice-purpose fictional (fabricated) data. The assignee names are characters from classical novels, and the tasks are non-existent logistics automation topics. It contains no real internal data or personal information whatsoever, and do not put real internal files in this folder.

The source data was taken from mx-agentic-ai-day1-prd.