🔭 EOSC Data Commons Search server

A server for the EOSC Data Commons project MatchMaker service, providing natural language search over open-access datasets. It exposes an HTTP POST endpoint and supports the Model Context Protocol (MCP) to help users discover datasets and tools via a Large Language Model–assisted search.

🧩 Endpoints

The HTTP API comprises 2 main endpoints:

• /mcp: MCP server that searches for relevant data to answer a user question using the EOSC Data Commons OpenSearch service

  • Uses Streamable HTTP transport

  • Available tools:

    • Search datasets

    • Get metadata for the files in a dataset (name, description, type of files)

    • Search tools

    • Search citations related to datasets or tools

• /chat: HTTP POST endpoint (JSON) for chatting with the MCP server tools via an LLM provider (API key provided through env variable at deployment)

  • Streams Server-Sent Events (SSE) response complying with the AG-UI protocol.

🔌 Connect client to MCP server

The system can be used directly as a MCP server using either STDIO, or Streamable HTTP transport.

Follow the instructions of your client, and use the /mcp URL of your deployed server (e.g. http://localhost:8000/mcp)

To add a new MCP server to VSCode GitHub Copilot:

• Open the Command Palette (ctrl+shift+p or cmd+shift+p)

• Search for MCP: Add Server...

• Choose HTTP, and provide the MCP server URL http://localhost:8000/mcp

Your VSCode mcp.json should look like:

{
    "servers": {
        "data-commons-search-http": {
            "url": "http://localhost:8000/mcp",
            "type": "http"
        }
    },
    "inputs": []
}

Or with STDIO transport:

{
   "servers": {
      "data-commons-search": {
         "type": "stdio",
         "command": "uvx",
         "args": ["data-commons-search"],
         "env": {
            "OPENSEARCH_URL": "OPENSEARCH_URL"
         }
      }
   }
}

Or using local folder for development:

{
   "servers": {
      "data-commons-search": {
         "type": "stdio",
         "cwd": "~/dev/data-commons-search",
         "env": {
            "OPENSEARCH_URL": "OPENSEARCH_URL"
         },
         "command": "uv",
         "args": ["run", "data-commons-search"]
      }
   }
}

🛠️ Development

📥 Install dev dependencies

uv sync --extra agent

Install pre-commit hooks:

uv run pre-commit install

Create a keys.env file with your LLM provider API key(s):

EINFRACZ_API_KEY=YOUR_API_KEY
MISTRAL_API_KEY=YOUR_API_KEY
OPENROUTER_API_KEY=YOUR_API_KEY

⚡️ Start dev server

Start the server in dev at http://localhost:8000, with MCP endpoint at http://localhost:8000/mcp

uv run uvicorn src.data_commons_search.main:app --log-config logging.yml --reload

Default OPENSEARCH_URL=http://localhost:9200

Customize server configuration through environment variables:

SERVER_PORT=8001 OPENSEARCH_URL=http://localhost:9200 uv run uvicorn src.data_commons_search.main:app --host 0.0.0.0 --port 8001 --log-config logging.yml --reload

curl -X POST http://localhost:8000/chat \
	-H "Content-Type: application/json" -H "Authorization: SECRET_KEY" \
	-d '{"messages": [{"role": "user", "content": "Educational datasets from Switzerland covering student assessments, language competencies, and learning outcomes, including experimental or longitudinal studies on pupils or students."}], "model": "einfracz/qwen3-coder"}'

npm run build && rm -rf ../data-commons-search/src/data_commons_search/webapp/ && cp -R dist/spa/ ../data-commons-search/src/data_commons_search/webapp/

📦 Build for production

Build binary in dist/

uv build

🐳 Deploy with Docker

Create a keys.env file with the API keys:

EINFRACZ_API_KEY=YOUR_API_KEY
MISTRAL_API_KEY=YOUR_API_KEY
OPENROUTER_API_KEY=YOUR_API_KEY
SEARCH_API_KEY=SECRET_KEY_YOU_CAN_USE_IN_FRONTEND_TO_AVOID_SPAM

You can use the prebuilt docker image ghcr.io/eosc-data-commons/data-commons-search:main

Example compose.yml:

services:
  mcp:
    image: ghcr.io/eosc-data-commons/data-commons-search:main
    ports:
      - "127.0.0.1:8000:8000"
    environment:
      OPENSEARCH_URL: "http://opensearch:9200"
      EINFRACZ_API_KEY: "${EINFRACZ_API_KEY}"

Build and deploy the service:

docker compose up

✅ Run tests

uv run pytest

To display all logs when debugging:

uv run pytest -s

🧹 Format code and type check

uvx ruff format
uvx ruff check --fix
uv run mypy

♻️ Reset the environment

Upgrade uv:

uv self update

Clean uv cache:

uv cache clean

🏷️ Release process

Run the release script providing the version bump: fix, minor, or major

.github/release.sh fix

export UV_PUBLISH_TOKEN=YOUR_TOKEN

🤝 Acknowledments

The LLM provider einfracz is a service provided by e-INFRA CZ and operated by CERIT-SC Masaryk University

Computational resources were provided by the e-INFRA CZ project (ID:90254), supported by the Ministry of Education, Youth and Sports of the Czech Republic.