actors-mcp-server

# Apify MCP server development instructions ## Overview This is an MCP (Model Context Protocol) server that exposes [Apify Actors](https://apify.com/store) as tools. It enables AI assistants to discover and use thousands of web scraping and automation tools from the Apify Store. The codebase is built with TypeScript using ES modules and follows a modular architecture with clear separation of concerns. The server can run in multiple modes: - **Standard Input/Output (stdio)**: For local integrations and command-line tools like Claude Desktop - **HTTP Streamable**: For hosted deployments and web-based MCP clients - **Legacy SSE over HTTP**: Legacy version of the protocol for hosted deployments and web-based clients (deprecated and will be removed in the future) ### Key concepts - **MCP**: Model Context Protocol defines tools, resources, and prompts that AI agents can use - **Apify Actors**: Reusable automation tools (web scrapers, data extractors) available on Apify Store - **Tool discovery**: Actors are dynamically converted to MCP tools based on their input schemas - **Two-step Actor execution**: `call-actor` requires `step="info"` to get schema, then `step="call"` to execute ### Root directories - `src/`: Main TypeScript source code - `tests/`: Unit and integration tests - `dist/`: Compiled JavaScript output (generated during build) - `evals/`: Evaluation scripts and test cases for AI agent interactions ### Core architecture (`src/` directory) The codebase is organized into logical modules: - `src/mcp/` - Core MCP protocol implementation - `src/tools/` - MCP tool implementations - `src/utils/` - Shared utility modules - `src/actor/` - Actor-specific implementation (for Apify platform deployment) (only used for testing) - Entry points: - `src/index.ts` - Main library export (`ActorsMcpServer` class) - `src/index-internals.ts` - Internal exports for testing and advanced usage - `src/stdio.ts` - Standard input/output entry point (CLI, used for Docker) - `src/main.ts` - Actor entry point (for Apify platform) - `src/input.ts` - Input processing and validation ## Validating TypeScript changes MANDATORY: Always check for TypeScript compilation errors before running tests or declaring work complete. ### TypeScript compilation steps - Run `npm run type-check` to check for TypeScript errors without building - Run `npm run build` to compile TypeScript files and check for errors - Fix all compilation errors before running tests or committing changes ## Testing ### Running tests - **Unit tests**: `npm run test:unit` (runs `vitest run tests/unit`) - **Integration tests**: `npm run test:integration` (requires build first, requires `APIFY_TOKEN`) ### Important: Integration tests require APIFY_TOKEN **⚠️ DO NOT attempt to run integration tests as an agent.** Integration tests require a valid `APIFY_TOKEN` environment variable, which only humans have access to. As an agent, you should: - Run `npm run type-check` and `npm run build` to validate TypeScript changes - Run `npm run test:unit` for unit tests which don't require authentication - Skip integration tests - these must be run by humans with valid Apify credentials ### Test structure - `tests/unit/` - Unit tests for individual modules - `tests/integration/` - Integration tests for MCP server functionality - `tests/helpers.ts` - Shared test utilities - `tests/const.ts` - Test constants ### Test guidelines - Write tests for new features and bug fixes - Use descriptive test names that explain what is being tested - Follow existing test patterns in the codebase - Ensure all tests pass before submitting a PR ### Manual testing as an MCP client To test the MCP server, a human must first configure the MCP server. Once configured, the server exposes tools that become available to the coding agent. #### 1. Human setup (required before testing) 1. **Configure the MCP server** in your environment (e.g., Claude code, VS Code, Cursor) 2. **Verify connection**: The client should connect and list available tools automatically 3. **Tools are now available**: Once connected, all MCP tools are exposed and ready to use #### 2. Coding agent testing workflow **Note**: Only execute this testing workflow when explicitly requested by the user. Once the server is configured, test the MCP tools by: 1. **Invoke each tool** through the MCP client (e.g., ask the AI agent to "search for actors" or "fetch actor details for apify/rag-web-browser") 2. **Test with valid inputs** (happy path) – verify outputs match expected formats 3. **Test with invalid inputs** (edge cases) – verify error messages are clear and helpful 4. **Verify key behaviors**: - **call-actor** requires two-step workflow (`step="info"` then `step="call"`) - All tools return helpful error messages with suggestions - **get-actor-output** supports field filtering using dot notation - Search tools support pagination with `limit` and `offset` **Tools to test:** - **search-actors** - Search Apify Store (test: valid keywords, empty keywords, non-existent platforms) - **fetch-actor-details** - Get Actor info (test: valid actor, non-existent actor) - **call-actor** - Execute Actor in two steps: `step="info"` to get schema, then `step="call"` to run - **get-actor-output** - Retrieve Actor results (test: valid datasetId, field filtering, non-existent dataset) - **search-apify-docs** - Search documentation (test: relevant terms, non-existent topics) - **fetch-apify-docs** - Fetch doc page (test: valid URL, non-existent page) ## Coding guidelines ### Indentation We use **4 spaces** for indentation (configured in `.editorconfig`). ### Naming conventions - **Constants**: Use uppercase `SNAKE_CASE` for global, immutable constants (e.g., `ACTOR_MAX_MEMORY_MBYTES`, `SERVER_NAME`) - **Functions & Variables**: Use `camelCase` format (e.g., `fetchActorDetails`, `actorClient`) - **Classes, Types, Interfaces**: Use `PascalCase` format (e.g., `ActorsMcpServer`, `ActorDetailsResult`) - **Files & Folders**: Use lowercase `snake_case` format (e.g., `actor_details.ts`, `key_value_store.ts`) - **Booleans**: Prefix with `is`, `has`, or `should` (e.g., `isValid`, `hasFinished`, `shouldRetry`) - **Units**: Suffix with the unit of measure (e.g., `intervalMillis`, `maxMemoryBytes`) - **Date/Time**: Suffix with `At` (e.g., `createdAt`, `updatedAt`) - **Zod Validators**: Suffix with `Validator` (e.g., `InputValidator`) ### Types and interfaces - Prefer `type` for flexibility. - Use `interface` only when it's required for class implementations (`implements`). ### Comments - Use JSDoc style comments (`/** */`) for functions, interfaces, enums, and classes - Use `//` for generic inline comments - Avoid `/* */` multiline comments (single asterisk) - Use proper English (spelling, grammar, punctuation, capitalization) ### Code structure - **Avoid `else`**: Return early to reduce indentation and keep logic flat - **Keep functions small**: Small, focused functions are easier to understand and test - **Minimal parameters**: Functions should only accept what they actually use - Use comma-separated parameters for up to three parameters - Use a single object parameter for more than three parameters - **Declare variables close to use**: Variables should be declared near their first use - **Extract reusable logic**: Extract complex or reusable logic into named helper functions ### Async functions - Use `async` and `await` over `Promise` and `then` calls - Use `await` when you care about the Promise result or exceptions - Use `void` when you don't need to wait for the Promise (fire-and-forget) - Use `return await` when returning Promises to preserve accurate stack traces ### Imports and import ordering - Imports are automatically ordered and grouped by ESLint: - Groups: builtin → external → parent/sibling → index → object - Alphabetized within groups - Newlines between groups - Use `import type` for type-only imports - Do not duplicate imports – always reuse existing imports if present - Do not use dynamic imports unless explicitly told to do so ### Error handling - **User Errors**: Use appropriate error codes (4xx for client errors), log as `softFail` - **Internal Errors**: Use appropriate error codes (5xx for server errors), log with `log.exception` or `log.error` - Always handle and propagate errors clearly - Use custom error classes from `src/errors.ts` when appropriate ### Code quality - All files must follow ESLint rules (run `npm run lint` before committing) - Prefer readability over micro-optimizations - Avoid mutating function parameters (use immutability when possible) - If mutation is necessary, clearly document and explain it with a comment - Clean up temporary files, scripts, or helper files created during development ### Common patterns - **Tool implementation**: Tools are defined in `src/tools/` using Zod schemas for validation - **Actor interaction**: Use `src/utils/apify-client.ts` for Apify API calls, never call Apify API directly - **Error responses**: Return user-friendly error messages with suggestions - **Input validation**: Always validate tool inputs with Zod before processing - **Caching**: Use TTL-based caching for Actor schemas and details (see `src/utils/ttl-lru.ts`) ### Anti-patterns - **Don't** call Apify API directly – always use the Apify client utilities - **Don't** mutate function parameters without clear documentation - **Don't** skip input validation – all tool inputs must be validated with Zod - **Don't** use `Promise.then()` - prefer `async/await` - **Don't** create tools without proper error handling and user-friendly messages ## External dependencies ### Important relationship: apify-mcp-server-internal **IMPORTANT**: This package (`@apify/actors-mcp-server`) is imported and used in the private repository `~/apify/apify-mcp-server-internal` for the hosted server implementation. **Key points:** - Changes to this repository may affect the hosted server - Breaking changes must be coordinated between both repositories - The hosted server uses this package as a dependency - Canary releases can be created using the `beta` tag on PR branches (see README.md) **Before making changes:** - Consider the impact on the hosted server - Test changes locally before submitting PRs - Coordinate breaking changes with the team - Check if changes require updates in `apify-mcp-server-internal` ## Development workflow ### Setup 1. Clone the repository 2. Install dependencies: `npm install` 3. Build the project: `npm run build` ### Development commands - `npm run start:dev` - Start development server (uses `tsx` for direct TypeScript execution) - `npm run build` - Build TypeScript to JavaScript - `npm run lint` - Run ESLint - `npm run lint:fix` - Fix ESLint issues automatically - `npm run type-check` - Type check without building - `npm run test` - Run unit tests - `npm run test:integration` - Run integration tests (requires build) - `npm run clean` - Clean build artifacts