spec.md•8.42 kB
# Test Suite Specification: `test-filesystem::regex_search_content`
## 1. Objective
This document defines the specification for creating a robust, automated test suite for the `regex_search_content` tool provided by the `test-filesystem` MCP server. The goal is to ensure the tool functions correctly across various scenarios, including those identified during prior interactive testing.
## 2. Testing Framework
**Recommendation:** **Bun's built-in test runner**
* **Rationale:** Bun's test runner is built-in, provides a Jest-compatible API, and runs quickly with native ESM support.
## 3. Directory Structure
The test suite files and related artifacts will be organized within the main project repository (`mcp-filesystem/`) as follows:
```
mcp-filesystem/
├── src/
├── test/
│ ├── suites/
│ │ ├── regex_search_content/
│ │ │ ├── spec.md # This specification document
│ │ │ ├── basic_search.test.ts # Example test file
│ │ │ ├── depth_limiting.test.ts
│ │ │ ├── error_handling.test.ts
│ │ │ └── ... (other test files)
│ │ └── ... (other tool suites)
│ └── ... (other test helpers, fixtures)
└── ... (package.json, tsconfig.json, etc.)
```
**Test Data Directory (Managed by Tests via MCP):**
All test files and directories manipulated *during test execution* will reside within the `test-filesystem` MCP server's designated root under a dedicated subdirectory:
* `test/fs_root/regex_search_content/`
This ensures test isolation and utilizes the MCP server's file operations for setup and teardown.
## 4. Test File Conventions
* **Naming:** Test files should follow the pattern `[feature_or_scenario].test.ts`. Examples:
* `basic_search.test.ts`
* `file_patterns.test.ts`
* `depth_limiting.test.ts`
* `max_results.test.ts`
* `error_handling.test.ts`
* `edge_cases.test.ts`
* **Structure:** Utilize standard structure using Bun's test API (similar to Jest):
* Use `describe()` blocks to group tests related to a specific feature or aspect (e.g., `describe('maxDepth parameter', () => { ... })`).
* Use `it()` or `test()` for individual test cases with descriptive names (e.g., `it('should return matches only from files matching the filePattern')`).
* Employ `beforeAll`, `afterAll`, `beforeEach`, `afterEach` for setup and teardown logic as described in Section 7.
## 5. MCP Tool Call Handling
* **Method:** Tests will directly invoke the `test-filesystem::regex_search_content` tool using the project's established MCP client/communication mechanism.
* **Invocation:** Calls should mirror how the tool would be used in a real scenario, passing parameters like `path`, `regex`, `filePattern`, `maxDepth`, `maxFileSize`, and `maxResults`.
* **Example (Conceptual):**
```typescript
import { mcpClient } from '../path/to/mcp/client'; // Assuming an MCP client instance
it('should find a simple pattern in the root test directory', async () => {
const result = await mcpClient.useTool('test-filesystem', 'regex_search_content', {
path: 'regex_search_content/', // Relative to MCP server root
regex: 'unique_pattern_123',
// maxDepth, maxResults etc. with defaults or specific values
});
// Assertions on the result structure and content
expect(result.success).toBe(true);
expect(result.data).toEqual(expect.arrayContaining([
expect.objectContaining({
file: 'regex_search_content/file1.txt',
matches: expect.arrayContaining([
expect.objectContaining({ line: 5, text: expect.stringContaining('unique_pattern_123') })
])
})
]));
});
```
* **Abstraction (Optional Future Improvement):** A helper function or class could be created later to wrap these MCP calls, simplifying test code and potentially enabling easier mocking if needed for unit testing components that *use* the MCP client. For this initial suite, direct calls are sufficient.
## 6. Test Case Implementation
* **Coverage:** Implement test cases derived from the original list (RCS-001 to RCS-083, skipping noted exclusions) and specifically address the findings from interactive testing.
* **Findings Integration:**
* **Path:** Always use `regex_search_content/` as the base `path` parameter in tests targeting the prepared test data area.
* **Recursion/Depth:**
* Test default depth behavior.
* Test `maxDepth: 1`, `maxDepth: 2`, `maxDepth: N` (where N > 2).
* Test interactions between `maxDepth` and `filePattern` (e.g., pattern matches file beyond `maxDepth`).
* Explicitly test `maxDepth: 0` and assert that it throws an appropriate validation error (addressing RCS-040 failure).
* **Regex Flags:**
* Confirm `(?i)` and `(?m)` are *not* supported and likely cause an invalid regex error.
* Test case-insensitive matching using character sets (e.g., `[Ss]earch`, `[Tt]his`).
* **`maxResults`:**
* Create scenarios with more files containing matches than `maxResults`.
* Assert that the number of *files* in the result array equals `maxResults`.
* Assert that the matches *within* the returned files are not truncated by `maxResults`.
* **Glob Negation:** Test `filePattern: '!(*.log)'` or similar and assert it likely fails or doesn't exclude as expected, confirming lack of support.
* **Error Handling:**
* Test with syntactically invalid regex patterns (confirming RCS-070 pass).
* Test with a `path` that does not exist within the MCP server's scope (assert specific error, not "No matches" - addressing RCS-071 failure).
* Test using a file path as the `path` parameter (assert specific error, not "No matches" - addressing RCS-072 failure).
* **Assertions:** Use `expect` assertions to validate:
* Success/failure status of the MCP call.
* Presence or absence of expected files in the results.
* Correct file paths (relative to the MCP server root).
* Correct line numbers for matches.
* Correct matching text snippets.
* Correct error messages/types for invalid inputs or scenarios.
## 7. Setup & Teardown
Test setup and teardown are crucial for creating a controlled environment within the `test-filesystem` MCP server's accessible directory (`test/fs_root/`).
* **Mechanism:** Use `beforeAll`, `afterAll` (and potentially `beforeEach`/`afterEach` if needed) within the test files.
* **Actions:** These hooks will use the `test-filesystem` MCP server's *own tools* (`create_directory`, `create_file`, `delete_directory`) to manage the test environment under `regex_search_content/`. The directory `test/fs_root/regex_search_content/` is created during setup and removed after the tests complete.
* **`beforeAll` (Executed once per test file):**
1. **Ensure Base Directory:** Call `test-filesystem::create_directory` with `path: 'regex_search_content/'`. This is idempotent.
2. **Create Test Files/Dirs:** Call `test-filesystem::create_file` multiple times to populate `regex_search_content/` with a variety of files and subdirectories needed for the tests in that specific file. Examples:
* `regex_search_content/file1.txt` (contains pattern A)
* `regex_search_content/subdir1/file2.log` (contains pattern A, pattern B)
* `regex_search_content/subdir1/nested/file3.txt` (contains pattern C)
* `regex_search_content/empty.txt`
* Files designed to test `maxResults`, `maxDepth`, `filePattern`, etc.
* **`afterAll` (Executed once per test file):**
1. **Clean Up:** Call `test-filesystem::delete_directory` with `path: 'regex_search_content/'` and `recursive: true`. This removes all test-specific files and directories created by the corresponding `beforeAll`.
* **`beforeEach`/`afterEach`:** Use sparingly only if a specific test requires a pristine state different from the `beforeAll` setup or needs to clean up uniquely created artifacts.
## 8. Execution
* Tests will be executed using Bun with the configured test runner:
* **Bun test:** `bun test test/suites/regex_search_content/` (or specific file)
* Integration with CI/CD pipelines should execute the Bun command.