jcrawl4ai-mcp-server

Java implementation of MCP Server for interacting with Crawl4ai API.

Project Overview

jcrawl4ai-mcp-server is a Spring Boot-based MCP server that interacts with the Crawl4ai API to perform web crawling. The main functionalities include:

Crawling specified URLs using a given strategy, maximum depth, and output format.
Getting the crawl result by a given task ID.

Configuration

application.properties

Configure the following properties in the src/main/resources/application.properties file:

cawl4ai.base-url: Base URL of the Crawl4ai server.
cawl4ai.api-token: API token for the Crawl4ai server.

Example configuration:

cawl4ai.base-url=http://your-cral4ai-server-url:11235
cawl4ai.api-token=your-api-token

Dependencies

The project depends on the following libraries:

Spring AI MCP Server
Spring Boot
Hutool

Running the Project

Build and run the project using Maven:

mvn clean install
java -jar target/jcawl4ai-mcp-server-1.0.0.jar

You can download the jar file from this link directly.

APIs

Crawl4aiApi

`crawl` Method

Description: Call the Crawl4ai API to crawl the specified URLs.
Parameters:
- urls: Array of target website URLs.
- strategy: Crawl strategy.
- max_depth: Maximum depth.
- output_format: Output format.
Return Value: JSON string of the crawl result.

`task` Method

Description: Get the crawl result by a given task ID.
Parameters:
- taskId: Task ID.
Return Value: JSON string of the crawl result.

Logging

Log file path: ./target/mcp-stdio-server.log.

MCP Server Configuration

{
  "mcpServers": {
    "jcawl4ai-mcp-server": {
      "autoApprove": [
        "crawl",
        "task"
      ],
      "disabled": false,
      "timeout": 60,
      "command": "java",
      "args": [
        "-jar",
        "/path/to/your/jar/file/jcawl4ai-mcp-server-1.0.0.jar"
      ],
      "transportType": "stdio"
    }
  }
}

Contact

If you have any questions or suggestions, please contact Ken Ye.