Stata MCP Server

# Stata MCP Timeout Feature Test Report **Date:** October 22, 2025 **Tester:** Testing timeout functionality with short intervals **Server Version:** Running from deepecon.stata-mcp-0.3.1 (process ID: 77652) ## Test Setup ### Test Script - **File:** [tests/test_timeout.do](../../tests/test_timeout.do) - **Duration:** Approximately 120 seconds (2 minutes) - **Behavior:** Loops 120 times with 1-second sleep in each iteration - **Purpose:** Long-running script to verify timeout interruption ### Test Cases 1. **Test 1:** 12-second timeout (0.2 minutes) 2. **Test 2:** 30-second timeout (0.5 minutes) Both tests should terminate the Stata script before completion. --- ## Test Results ### Test 1: 12-Second Timeout (0.2 minutes) **Command:** ```bash curl -s -X POST "http://localhost:4000/run_file?file_path=/path/to/stata-mcp/tests/test_timeout.do&timeout=12" ``` **Expected Behavior:** - Script should timeout after 12 seconds - Timeout message should appear - Script should NOT complete all 120 iterations **Actual Behavior:** - Script ran for **120.2 seconds** (2 minutes) - Script completed **ALL 120 iterations** - NO timeout message appeared - Final message: "Test completed successfully" **Result:** ❌ **FAILED** - Timeout did NOT trigger --- ### Test 2: 30-Second Timeout (0.5 minutes) **Command:** ```bash time curl -s -X POST "http://localhost:4000/run_file?file_path=/path/to/stata-mcp/tests/test_timeout.do&timeout=30" ``` **Expected Behavior:** - Script should timeout after 30 seconds - Should show approximately 30 iterations completed - Timeout message should appear **Actual Behavior:** - Script ran for **120.2 seconds** (2:00.27 total time) - Script completed **ALL 120 iterations** - NO timeout message appeared - Final message: "Test completed successfully" **Result:** ❌ **FAILED** - Timeout did NOT trigger --- ## Analysis ### Code Review The timeout feature IS implemented in the codebase: **Location:** [stata_mcp_server.py:1284](src/stata_mcp_server.py#L1284) ```python # Line 981: Set timeout parameter MAX_TIMEOUT = timeout # Lines 1279-1342: Polling loop with timeout check while stata_thread.is_alive(): current_time = time.time() elapsed_time = current_time - start_time if elapsed_time > MAX_TIMEOUT: # Line 1284 logging.warning(f"Execution timed out after {MAX_TIMEOUT} seconds") result += f"\n*** TIMEOUT: Execution exceeded {MAX_TIMEOUT} seconds ({MAX_TIMEOUT/60:.1f} minutes) ***\n" # ... termination logic ... break ``` ### Timeout Implementation Features The code includes: 1. ✅ Configurable timeout parameter (default: 600 seconds) 2. ✅ Input validation (must be positive integer) 3. ✅ Polling-based timeout check (every 0.5 seconds) 4. ✅ Multi-stage termination: - Stage 1: Stata `break` command - Stage 2: Thread `_stop()` method - Stage 3: Process kill via `pkill` 5. ✅ Adaptive polling intervals 6. ✅ Clear timeout error messages ### Root Cause Analysis **Issue:** The running server (version 0.3.1) appears to be using **cached or outdated code** that does not include the timeout logic. **Evidence:** 1. Process runs from `/Users/hanlulong/.vscode/extensions/deepecon.stata-mcp-0.3.1/src/stata_mcp_server.py` 2. This file **does NOT exist** on disk (only logs directory exists in 0.3.1) 3. The actual source code in development repo and version 0.3.3 DOES have timeout logic 4. Server must be running from Python bytecode cache (.pyc) or was started before directory cleanup ### Why Timeout Doesn't Work in Running Server The currently running server (PID 77652) is using an **older version** of the code that likely: - Does NOT check `elapsed_time > MAX_TIMEOUT` - Does NOT have the timeout termination logic - May ignore or not properly handle the timeout parameter --- ## Recommendations ### 1. Restart the Server To activate the timeout feature, the server needs to be restarted with the current code: ```bash # Kill the old server pkill -f "stata_mcp_server.py" # Start the new server from the current codebase python3 /path/to/stata-mcp/src/stata_mcp_server.py --port 4000 --stata-path /Applications/StataNow --stata-edition mp ``` ### 2. Verify Timeout Works After restarting, re-run the tests: ```bash # Test 1: 12 seconds curl -s -X POST "http://localhost:4000/run_file?file_path=/path/to/test_timeout.do&timeout=12" # Test 2: 30 seconds curl -s -X POST "http://localhost:4000/run_file?file_path=/path/to/test_timeout.do&timeout=30" ``` Expected: Script should terminate at the specified timeout with message: ``` *** TIMEOUT: Execution exceeded N seconds (N/60 minutes) *** ``` ### 3. Additional Testing Needed Once server is restarted with current code: 1. **Short timeout test** (5 seconds) - Verify immediate termination 2. **Mid-range timeout test** (30 seconds) - Verify partial execution 3. **Long timeout test** (600 seconds / 10 min default) - Verify default works 4. **Verify termination methods:** - Check which termination stage succeeds (break, _stop, or pkill) - Monitor Stata process cleanup - Verify no zombie processes remain ### 4. Future Improvements Consider these enhancements: 1. **Add timeout to response headers** - So clients can see the configured timeout 2. **Add progress indicators** - Show countdown or elapsed time 3. **Make termination more graceful** - Save partial results before killing 4. **Add logging** - Log all timeout events to help debugging 5. **Add telemetry** - Track timeout frequency and duration statistics --- ## Conclusion **Current Status:** ❌ Timeout feature **NOT WORKING** in running server **Reason:** Server is running outdated/cached code without timeout logic **Solution:** Restart server with current codebase **Code Quality:** ✅ Timeout implementation in current code looks **robust and well-designed** **Next Steps:** 1. Restart server with current code 2. Re-run timeout tests 3. Verify timeout triggers correctly 4. Test all three termination stages 5. Monitor for any edge cases or issues --- ## Test Artifacts ### Generated Files - [tests/test_timeout.do](../../tests/test_timeout.do) - Test Stata script - [tests/test_timeout_direct.py](../../tests/test_timeout_direct.py) - Direct Python test (couldn't run, Stata not available) ### Log Files - `/path/to/.vscode/extensions/deepecon.stata-mcp-0.3.1/logs/test_timeout_mcp.log` - Stata execution log ### Process Information ``` PID: 77652 Command: /usr/local/Cellar/python@3.11/3.11.11/Frameworks/Python.framework/Versions/3.11/Resources/Python.app/Contents/MacOS/Python /path/to/.vscode/extensions/deepecon.stata-mcp-0.3.1/src/stata_mcp_server.py --port 4000 --stata-path /Applications/StataNow --log-file /path/to/.vscode/extensions/deepecon.stata-mcp-0.3.1/logs/stata_mcp_server.log --stata-edition mp --log-level DEBUG --log-file-location extension ``` --- ## UPDATE: Server Restarted with Current Code (2025-10-22 17:01) ### Server Restart - **Killed:** PID 77652 (old cached version) - **Started:** PID 27026 (current codebase from /path/to/stata-mcp/src/) - **Status:** ✅ Server running and healthy ### Test Results After Restart #### Test 1: 12-Second Timeout (Retry) - **Command:** `POST /run_file?file_path=.../test_timeout.do&timeout=12` - **Expected:** Timeout after 12 seconds - **Actual:** Ran for **120.2 seconds** (2 minutes) - **Result:** ❌ **STILL FAILED** ### ROOT CAUSE IDENTIFIED 🔍 **Critical Bug:** The `timeout` parameter is **NOT being passed** from the REST API URL to the endpoint function! **Evidence from Server Log:** ``` Log file: stata_mcp_server.log 2025-10-22 17:02:11,866 - root - INFO - Running file: ... with timeout 600 seconds (10.0 minutes) ``` **Analysis:** - URL parameter sent: `?timeout=12` - Value received by function: `600` (default) - Conclusion: FastAPI is not extracting the `timeout` query parameter ### Technical Root Cause **File:** [stata_mcp_server.py:1643-1644](src/stata_mcp_server.py#L1643-L1644) ```python @app.post("/run_file", operation_id="stata_run_file", response_class=Response) async def stata_run_file_endpoint(file_path: str, timeout: int = 600) -> Response: ``` **Problem:** In FastAPI, POST endpoint function parameters are treated as **request body parameters** by default, NOT query parameters. When you call: ``` POST /run_file?file_path=/path&timeout=12 ``` FastAPI expects these to be explicitly marked as query parameters using `Query()`. **Why `file_path` works but `timeout` doesn't:** - `file_path` is a required parameter (no default), so FastAPI tries harder to find it - `timeout` has a default value, so FastAPI uses the default when it can't find it in the request body ### The Fix **Required Change:** ```python from fastapi import Query @app.post("/run_file", operation_id="stata_run_file", response_class=Response) async def stata_run_file_endpoint( file_path: str = Query(..., description="Path to the .do file"), timeout: int = Query(600, description="Timeout in seconds") ) -> Response: # ... rest of the function ``` **Alternative Fix** (if you want to keep current signature): Change from POST to GET: ```python @app.get("/run_file", operation_id="stata_run_file", response_class=Response) async def stata_run_file_endpoint(file_path: str, timeout: int = 600) -> Response: ``` GET requests automatically treat function parameters as query parameters. --- ## Final Conclusion **Current Status:** ❌ Timeout feature **NOT WORKING** **Root Cause:** **FastAPI REST API bug** - `timeout` query parameter is not being extracted from POST request **Impact:** - ✅ Timeout implementation logic (lines 1279-1342) is **correct and robust** - ✅ Timeout validation and logging (lines 1651-1662) is **correct** - ❌ Parameter binding in REST API endpoint is **broken** - Result: Timeout logic NEVER receives the custom timeout value, always uses default 600s **Severity:** **HIGH** - Feature appears to be implemented but doesn't work at all **Affected Endpoints:** - `POST /run_file` - timeout parameter ignored **Recommendations:** 1. **Immediate Fix:** Add `Query()` annotations to POST endpoint parameters 2. **Testing:** Add integration tests to verify query parameter binding 3. **Documentation:** Update API docs to show correct parameter usage 4. **Consider:** Change endpoint from POST to GET (more RESTful for read operations) 5. **Verification:** After fix, re-run all timeout tests to ensure proper termination --- ## Test Summary | Test | Timeout (s) | Expected Behavior | Actual Behavior | Status | |------|-------------|-------------------|-----------------|---------| | Pre-restart | 12 | Timeout at 12s | Ran 120s | ❌ | | Pre-restart | 30 | Timeout at 30s | Ran 120s | ❌ | | Post-restart | 12 | Timeout at 12s | Ran 120s | ❌ | **Conclusion:** Timeout feature completely non-functional due to parameter binding bug.