nl-unity-suite-t.md•14.7 kB
# Unity T Editing Suite — Additive Test Design
You are running inside CI for the `unity-mcp` repo. Use only the tools allowed by the workflow. Work autonomously; do not prompt the user. Do NOT spawn subagents.
**Print this once, verbatim, early in the run:**
AllowedTools: Write,mcp__unity__manage_editor,mcp__unity__list_resources,mcp__unity__read_resource,mcp__unity__apply_text_edits,mcp__unity__script_apply_edits,mcp__unity__validate_script,mcp__unity__find_in_file,mcp__unity__read_console,mcp__unity__get_sha
---
## Mission
1) Pick target file (prefer):
- `unity://path/Assets/Scripts/LongUnityScriptClaudeTest.cs`
2) Execute T tests T-A..T-J in order using minimal, precise edits that build on the NL pass state.
3) Validate each edit with `mcp__unity__validate_script(level:"standard")`.
4) **Report**: write one `<testcase>` XML fragment per test to `reports/<TESTID>_results.xml`. Do **not** read or edit `$JUNIT_OUT`.
**CRITICAL XML FORMAT REQUIREMENTS:**
- Each file must contain EXACTLY one `<testcase>` root element
- NO prologue, epilogue, code fences, or extra characters
- NO markdown formatting or explanations outside the XML
- Use this exact format:
```xml
<testcase name="T-D — End-of-Class Helper" classname="UnityMCP.NL-T">
<system-out><![CDATA[
(evidence of what was accomplished)
]]></system-out>
</testcase>
```
- If test fails, include: `<failure message="reason"/>`
- TESTID must be one of: T-A, T-B, T-C, T-D, T-E, T-F, T-G, T-H, T-I, T-J
5) **NO RESTORATION** - tests build additively on previous state.
6) **STRICT FRAGMENT EMISSION** - After each test, immediately emit a clean XML file under `reports/<TESTID>_results.xml` with exactly one `<testcase>` whose `name` begins with the exact test id. No prologue/epilogue or fences. If the test fails, include a `<failure message="..."/>` and still emit.
---
## Environment & Paths (CI)
- Always pass: `project_root: "TestProjects/UnityMCPTests"` and `ctx: {}` on list/read/edit/validate.
- **Canonical URIs only**:
- Primary: `unity://path/Assets/...` (never embed `project_root` in the URI)
- Relative (when supported): `Assets/...`
CI provides:
- `$JUNIT_OUT=reports/junit-nl-suite.xml` (pre‑created; leave alone)
- `$MD_OUT=reports/junit-nl-suite.md` (synthesized from JUnit)
---
## Transcript Minimization Rules
- Do not restate tool JSON; summarize in ≤ 2 short lines.
- Never paste full file contents. For matches, include only the matched line and ±1 line.
- Prefer `mcp__unity__find_in_file` for targeting; avoid `mcp__unity__read_resource` unless strictly necessary. If needed, limit to `head_bytes ≤ 256` or `tail_lines ≤ 10`.
- Per‑test `system-out` ≤ 400 chars: brief status only (no SHA).
- Console evidence: fetch the last 10 lines with `include_stacktrace:false` and include ≤ 3 lines in the fragment.
- Avoid quoting multi‑line diffs; reference markers instead.
— Console scans: perform two reads — last 10 `log/info` lines and up to 3 `error` entries (use `include_stacktrace:false`); include ≤ 3 lines total in the fragment; if no errors, state "no errors".
— Final check is folded into T‑J: perform an errors‑only scan (with `include_stacktrace:false`) and include a single "no errors" line or up to 3 error lines within the T‑J fragment.
---
## Tool Mapping
- **Anchors/regex/structured**: `mcp__unity__script_apply_edits`
- Allowed ops: `anchor_insert`, `replace_method`, `insert_method`, `delete_method`, `regex_replace`
- For `anchor_insert`, always set `"position": "before"` or `"after"`.
- **Precise ranges / atomic batch**: `mcp__unity__apply_text_edits` (non‑overlapping ranges)
STRICT OP GUARDRAILS
- Do not use `anchor_replace`. Structured edits must be one of: `anchor_insert`, `replace_method`, `insert_method`, `delete_method`, `regex_replace`.
- For multi‑spot textual tweaks in one operation, compute non‑overlapping ranges with `mcp__unity__find_in_file` and use `mcp__unity__apply_text_edits`.
- **Hash-only**: `mcp__unity__get_sha` — returns `{sha256,lengthBytes,lastModifiedUtc}` without file body
- **Validation**: `mcp__unity__validate_script(level:"standard")`
- **Dynamic targeting**: Use `mcp__unity__find_in_file` to locate current positions of methods/markers
---
## Additive Test Design Principles
**Key Changes from Reset-Based:**
1. **Dynamic Targeting**: Use `find_in_file` to locate methods/content, never hardcode line numbers
2. **State Awareness**: Each test expects the file state left by the previous test
3. **Content-Based Operations**: Target methods by signature, classes by name, not coordinates
4. **Cumulative Validation**: Ensure the file remains structurally sound throughout the sequence
5. **Composability**: Tests demonstrate how operations work together in real workflows
**State Tracking:**
- Track file SHA after each test (`mcp__unity__get_sha`) and use it as a precondition
for `apply_text_edits` in T‑F/T‑G/T‑I to exercise `stale_file` semantics. Do not include SHA values in report fragments.
- Use content signatures (method names, comment markers) to verify expected state
- Validate structural integrity after each major change
---
### T-A. Temporary Helper Lifecycle (Returns to State C)
**Goal**: Test insert → verify → delete cycle for temporary code
**Actions**:
- Find current position of `GetCurrentTarget()` method (may have shifted from NL-2 comment)
- Insert temporary helper: `private int __TempHelper(int a, int b) => a + b;`
- Verify helper method exists and compiles
- Delete helper method via structured delete operation
- **Expected final state**: Return to State C (helper removed, other changes intact)
### Late-Test Editing Rule
- When modifying a method body, use `mcp__unity__script_apply_edits`. If the method is expression-bodied (`=>`), convert it to a block or replace the whole method definition. After the edit, run `mcp__unity__validate_script` and rollback on error. Use `//` comments in inserted code.
### T-B. Method Body Interior Edit (Additive State D)
**Goal**: Edit method interior without affecting structure, on modified file
**Actions**:
- Use `find_in_file` to locate current `HasTarget()` method (modified in NL-1)
- Edit method body interior: change return statement to `return true; /* test modification */`
- Validate with `mcp__unity__validate_script(level:"standard")` for consistency
- Verify edit succeeded and file remains balanced
- **Expected final state**: State C + modified HasTarget() body
### T-C. Different Method Interior Edit (Additive State E)
**Goal**: Edit a different method to show operations don't interfere
**Actions**:
- Locate `ApplyBlend()` method using content search
- Edit interior line to add null check: `if (animator == null) return; // safety check`
- Preserve method signature and structure
- **Expected final state**: State D + modified ApplyBlend() method
### T-D. End-of-Class Helper (Additive State F)
**Goal**: Add permanent helper method at class end
**Actions**:
- Use smart anchor matching to find current class-ending brace (after NL-3 tail comments)
- Insert permanent helper before class brace: `private void TestHelper() { /* placeholder */ }`
- Validate with `mcp__unity__validate_script(level:"standard")`
- **IMMEDIATELY** write clean XML fragment to `reports/T-D_results.xml` (no extra text). The `<testcase name>` must start with `T-D`. Include brief evidence in `system-out`.
- **Expected final state**: State E + TestHelper() method before class end
### T-E. Method Evolution Lifecycle (Additive State G)
**Goal**: Insert → modify → finalize a field + companion method
**Actions**:
- Insert field: `private int Counter = 0;`
- Update it: find and replace with `private int Counter = 42; // initialized`
- Add companion method: `private void IncrementCounter() { Counter++; }`
- **Expected final state**: State F + Counter field + IncrementCounter() method
### T-F. Atomic Multi-Edit (Additive State H)
**Goal**: Multiple coordinated edits in single atomic operation
**Actions**:
- Read current file state to compute precise ranges
- Atomic edit combining:
1. Add comment in `HasTarget()`: `// validated access`
2. Add comment in `ApplyBlend()`: `// safe animation`
3. Add final class comment: `// end of test modifications`
- All edits computed from same file snapshot, applied atomically
- **Expected final state**: State G + three coordinated comments
- After applying the atomic edits, run `validate_script(level:"standard")` and emit a clean fragment to `reports/T-F_results.xml` with a short summary.
### T-G. Path Normalization Test (No State Change)
**Goal**: Verify URI forms work equivalently on modified file
**Actions**:
- Make identical edit using `unity://path/Assets/Scripts/LongUnityScriptClaudeTest.cs`
- Then using `Assets/Scripts/LongUnityScriptClaudeTest.cs`
- Second should return `stale_file`, retry with updated SHA
- Verify both URI forms target same file
- **Expected final state**: State H (no content change, just path testing)
- Emit `reports/T-G_results.xml` showing evidence of stale SHA handling.
### T-H. Validation on Modified File (No State Change)
**Goal**: Ensure validation works correctly on heavily modified file
**Actions**:
- Run `validate_script(level:"standard")` on current state
- Verify no structural errors despite extensive modifications
- **Expected final state**: State H (validation only, no edits)
- Emit `reports/T-H_results.xml` confirming validation OK.
### T-I. Failure Surface Testing (No State Change)
**Goal**: Test error handling on real modified file
**Actions**:
- Attempt overlapping edits (should fail cleanly)
- Attempt edit with stale SHA (should fail cleanly)
- Verify error responses are informative
- **Expected final state**: State H (failed operations don't modify file)
- Emit `reports/T-I_results.xml` capturing error evidence; file must contain one `<testcase>`.
### T-J. Idempotency on Modified File (Additive State I)
**Goal**: Verify operations behave predictably when repeated
**Actions**:
- **Insert (structured)**: `mcp__unity__script_apply_edits` with:
`{"op":"anchor_insert","anchor":"// Tail test C","position":"after","text":"\n // idempotency test marker"}`
- **Insert again** (same op) → expect `no_op: true`.
- **Remove (structured)**: `{"op":"regex_replace","pattern":"(?m)^\\s*// idempotency test marker\\r?\\n?","text":""}`
- **Remove again** (same `regex_replace`) → expect `no_op: true`.
- `mcp__unity__validate_script(level:"standard")`
- Perform a final console scan for errors/exceptions (errors only, up to 3); include "no errors" if none
- **IMMEDIATELY** write clean XML fragment to `reports/T-J_results.xml` with evidence of both `no_op: true` outcomes and the console result. The `<testcase name>` must start with `T-J`.
- **Expected final state**: State H + verified idempotent behavior
---
## Dynamic Targeting Examples
**Instead of hardcoded coordinates:**
```json
{"startLine": 31, "startCol": 26, "endLine": 31, "endCol": 58}
```
**Use content-aware targeting:**
```json
# Find current method location
find_in_file(pattern: "public bool HasTarget\\(\\)")
# Then compute edit ranges from found position
```
**Method targeting by signature:**
```json
{"op": "replace_method", "className": "LongUnityScriptClaudeTest", "methodName": "HasTarget"}
```
**Anchor-based insertions:**
```json
{"op": "anchor_insert", "anchor": "private void Update\\(\\)", "position": "before", "text": "// comment"}
```
---
## State Verification Patterns
**After each test:**
1. Verify expected content exists: `find_in_file` for key markers
2. Check structural integrity: `validate_script(level:"standard")`
3. Update SHA tracking for next test's preconditions
4. Emit a per‑test fragment to `reports/<TESTID>_results.xml` immediately. If the test failed, still write a single `<testcase>` with a `<failure message="..."/>` and evidence in `system-out`.
5. Log cumulative changes in test evidence (keep concise per Transcript Minimization Rules; never paste raw tool JSON)
**Error Recovery:**
- If test fails, log current state but continue (don't restore)
- Next test adapts to actual current state, not expected state
- Demonstrates resilience of operations on varied file conditions
---
## Benefits of Additive Design
1. **Realistic Workflows**: Tests mirror actual development patterns
2. **Robust Operations**: Proves edits work on evolving files, not just pristine baselines
3. **Composability Validation**: Shows operations coordinate well together
4. **Simplified Infrastructure**: No restore scripts or snapshots needed
5. **Better Failure Analysis**: Failures don't cascade - each test adapts to current reality
6. **State Evolution Testing**: Validates SDK handles cumulative file modifications correctly
This additive approach produces a more realistic and maintainable test suite that better represents actual SDK usage patterns.
---
BAN ON EXTRA TOOLS AND DIRS
- Do not use any tools outside `AllowedTools`. Do not create directories; assume `reports/` exists.
---
## XML Fragment Templates (T-F .. T-J)
Use these skeletons verbatim as a starting point. Replace the bracketed placeholders with your evidence. Ensure each file contains exactly one `<testcase>` element and that the `name` begins with the exact test id.
```xml
<testcase name="T-F — Atomic Multi-Edit" classname="UnityMCP.NL-T">
<system-out><![CDATA[
Applied 3 non-overlapping edits in one atomic call:
- HasTarget(): added "// validated access"
- ApplyBlend(): added "// safe animation"
- End-of-class: added "// end of test modifications"
validate_script: OK
]]></system-out>
</testcase>
```
```xml
<testcase name="T-G — Path Normalization Test" classname="UnityMCP.NL-T">
<system-out><![CDATA[
Edit via unity://path/... succeeded.
Same edit via Assets/... returned stale_file, retried with updated hash: OK.
]]></system-out>
</testcase>
```
```xml
<testcase name="T-H — Validation on Modified File" classname="UnityMCP.NL-T">
<system-out><![CDATA[
validate_script(level:"standard"): OK on the modified file.
]]></system-out>
</testcase>
```
```xml
<testcase name="T-I — Failure Surface Testing" classname="UnityMCP.NL-T">
<system-out><![CDATA[
Overlapping edit: failed cleanly (error captured).
Stale hash edit: failed cleanly (error captured).
File unchanged.
]]></system-out>
</testcase>
```
```xml
<testcase name="T-J — Idempotency on Modified File" classname="UnityMCP.NL-T">
<system-out><![CDATA[
Insert marker after "// Tail test C": OK.
Insert same marker again: no_op: true.
regex_remove marker: OK.
regex_remove again: no_op: true.
validate_script: OK.
]]></system-out>
</testcase>