SharePoint Search MCP

A self-hosted semantic search service for PPTX, DOCX, and PDF files stored in SharePoint. A local indexer reads a synchronized OneDrive folder, extracts and chunks text, creates 384-dimensional MiniLM embeddings, and writes them to SQLite or regular PostgreSQL. A separate read-only FastMCP server exposes search and retrieval tools over Streamable HTTP.

The PostgreSQL backend does not require pgvector. Embeddings are stored as float32 BYTEA values and loaded into an in-memory normalized NumPy matrix for cosine search. This keeps deployment possible on managed PostgreSQL offerings without extension support, but memory use grows with the number of chunks.

Features

• Incremental indexing: unchanged files are skipped and changed files are replaced.

• Per-file transactions and rollback so one corrupt document does not stop a run.

• Exact chunk order and page or slide numbers.

• Clickable SharePoint file links and optional folder links.

• Best-effort embedded Office metadata for PPTX and DOCX.

• SAP IAS or another OIDC discovery endpoint through FastMCP OAuthProxy.

• Privacy-conscious aggregate usage monitoring in PostgreSQL.

• Local SQLite fallback for development and small installations.

Related MCP server: Enterprise Knowledge MCP Server

Requirements

• Python 3.11

• A locally synchronized SharePoint or OneDrive folder

• PostgreSQL 16 for the recommended shared deployment, or SQLite for local use

• Cloud Foundry only when using the supplied deployment and tunnel helpers

• An OIDC application for authenticated deployments

Quick start

Create a virtual environment and install both components:

python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install -r requirements-dev.txt
Copy-Item indexer\.env.example indexer\.env

Edit indexer/.env so LOCAL_ROOT, LOCAL_PREFIX, SHAREPOINT_PREFIX, and SITE_URL describe the same synchronized library. Plan runs are read-only:

python -m indexer.indexer
python -m indexer.indexer --apply

For a simple local SQLite setup, set these values in indexer/.env:

INDEX_BACKEND=sqlite
INDEX_DB_PATH=mcp_server/index.db

Then start the local MCP server without OIDC:

$env:MCP_AUTH_DISABLED = "1"
$env:INDEX_BACKEND = "sqlite"
$env:INDEX_DB_PATH = "mcp_server/index.db"
python -m mcp_server.main

The Streamable HTTP endpoint is http://localhost:8000/mcp.

PostgreSQL

Start an optional local PostgreSQL 16 instance:

docker compose up -d
$env:DATABASE_URL = "postgresql://sharepoint_mcp:local-development-only@localhost:5432/sharepoint_mcp"
$env:INDEX_DATABASE_URL = $env:DATABASE_URL

Set INDEX_BACKEND=postgres and POSTGRES_SCHEMA=sharepoint_mcp for the indexer. The server uses DATABASE_SCHEMA for the same value. Schema and tables are created by the application user, which therefore needs the corresponding permissions.

The server preloads all vectors. A 384-dimensional vector occupies 1,536 bytes before NumPy and process overhead. Size memory from the real chunk count and do not assume the example 1,800 MB limit fits every corpus.

Model files

By default sentence-transformers downloads all-MiniLM-L6-v2 on first use. For a restricted or reproducible deployment, download it before packaging:

python tools/download_model.py

The generated mcp_server/models directory is intentionally ignored by Git but is not excluded from cf push. Set EMBEDDING_MODEL to mcp_server/models/all-MiniLM-L6-v2 in that case.

Cloud Foundry deployment

1. Create a PostgreSQL service instance and ensure its application user may create the configured schema and tables.

2. Copy vars.example.yml to vars.yml and replace every placeholder. vars.yml is ignored by Git because it contains the OIDC secret and monitoring salt.

3. Register the MCP_BASE_URL callback URLs required by FastMCP in your OIDC provider.

4. Download the model if the staging/runtime network cannot reach Hugging Face.

5. Deploy and verify:

cf push --vars-file vars.yml
cf logs sharepoint-search-mcp --recent

The service binding is discovered through VCAP_SERVICES. Outside Cloud Foundry, use DATABASE_URL. The supplied route-less helper in tools/db_tunnel_app and the PowerShell runner can reach a private CF database from a trusted workstation:

cf push -f tools/db_tunnel_app/manifest.yml --vars-file tools/db_tunnel_app/vars.yml
.\tools\run_local_onedrive_indexer.ps1
.\tools\run_local_onedrive_indexer.ps1 -Apply

Optionally set CF_API, CF_ORG, and CF_SPACE before running the scripts. When provided, the scripts refuse to run against a different target. Never pass -Apply until the plan and generated SharePoint paths have been reviewed.

Runtime configuration

MCP_AUTH_DISABLED is rejected on Cloud Foundry unless MCP_ALLOW_INSECURE_CF=1 is also set. Do not use that override for an internet-accessible deployment.

Authentication limitation

With FastMCP 2.12.5, the PostgreSQL key/value adapter persists dynamic client registrations. Authorization transactions, authorization codes, and some refresh-token bookkeeping still live in process memory. Existing clients must therefore be tested across restarts; complete restart-transparent OAuth state is not guaranteed by this version.

Usage monitoring and privacy

Monitoring stores daily aggregate tool and HTTP counts plus hashed OIDC subject IDs. It does not store prompts, query text, file paths, e-mail addresses, IP addresses, or tokens. Reports are deliberately not MCP tools. CF operators can retrieve them through:

.\tools\get_usage_metrics.ps1 -Days 7

Use a secret, installation-specific MCP_USAGE_HASH_SALT and define an appropriate retention period for your organization.

Testing

The default suite is offline and never connects to PostgreSQL or downloads a model:

python -m ruff check .
python -m compileall -q indexer mcp_server tools tests
python -m pytest

The included suite uses test doubles and temporary SQLite databases; it does not connect to PostgreSQL. Run database integration checks only against an explicitly disposable database, never production. Model quality is outside the unit-test boundary; run a manual smoke test after downloading the configured model.

Backup and recovery

Back up PostgreSQL according to your provider's policy. The search index is derived data and can be rebuilt from the synchronized source folder. Keep source mapping and embedding configuration with operational documentation, but never store secrets there.

License

Licensed under the Apache License, Version 2.0. See LICENSE.

SharePoint, Microsoft, SAP, SAP BTP, Joule, and other product names are trademarks of their respective owners. Their mention describes compatible systems and does not imply endorsement.