Skip to main content
Glama
MCP Spark Documentation Server

MCP Spark Documentation Server

by martoc

License: MIT Python 3.12 MCP

MCP Spark Documentation Server

An MCP (Model Context Protocol) server that provides search and retrieval tools for Apache Spark documentation. This server enables AI assistants like Claude to search and read Spark documentation directly.

Features

  • Full-text search using SQLite FTS5 with BM25 ranking and Porter stemming

  • Section filtering to narrow search results by documentation category

  • Sparse checkout for efficient cloning of only the docs directory from apache/spark

  • Docker support for portable deployment across projects

  • STDIO transport for seamless MCP client integration

Related MCP server: anydocs

Quick Start

# Build the Docker image (includes pre-indexed documentation)
make docker-build

# Test the server
make docker-run

Using uv (Local Development)

# Initialise the environment
make init

# Build the documentation index
make index

# Run the server
make run

Configuration

Claude Code / Claude Desktop

Add to your .mcp.json or global settings:

{
  "mcpServers": {
    "spark-documentation": {
      "command": "docker",
      "args": ["run", "-i", "--rm", "martoc/mcp-spark-documentation:latest"]
    }
  }
}

For a locally built Docker image:

{
  "mcpServers": {
    "spark-documentation": {
      "command": "docker",
      "args": ["run", "-i", "--rm", "mcp-spark-documentation"]
    }
  }
}

For local development without Docker:

{
  "mcpServers": {
    "spark-documentation": {
      "command": "uv",
      "args": ["run", "mcp-spark-documentation"],
      "cwd": "/path/to/mcp-spark-documentation"
    }
  }
}

MCP Tools

Tool

Description

search_documentation

Search Spark documentation by keyword query with optional section filtering

read_documentation

Retrieve the full content of a specific documentation page

search_documentation

Search Apache Spark documentation using full-text search with stemming support.

Parameter

Type

Required

Default

Description

query

string

Yes

-

Search terms (supports stemming)

section

string

No

None

Filter by section (e.g., sql-ref, streaming, mllib)

limit

integer

No

10

Maximum results (1-50)

Common Sections: sql-ref, api, streaming, mllib, graphx, structured-streaming, configuration, tuning

read_documentation

Retrieve the full content of a documentation page.

Parameter

Type

Required

Description

path

string

Yes

Relative path to document (from search results)

CLI Commands

# Build/rebuild the documentation index
uv run spark-docs-index index
uv run spark-docs-index index --rebuild
uv run spark-docs-index index --branch master

# Show index statistics
uv run spark-docs-index stats

Development

make init       # Initialise development environment
make build      # Run full build (lint, typecheck, test)
make test       # Run tests with coverage
make format     # Format code
make lint       # Run linter
make typecheck  # Run type checker

Documentation

Licence

This project is licensed under the MIT Licence - see the LICENSE file for details.

Available Tools

2 tools
read_documentationA

Read the full content of a specific Spark documentation page.

NameRequiredDescriptionDefault
pathYesThe relative path to the documentation file (e.g., 'sql-ref/sql-syntax.md' or 'api/python/index.md'). This path is returned in search results.

Output Schema

NameRequiredDescription
resultYes

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it does disclose one useful trait: it returns the FULL page content rather than a snippet. It does not address page size or truncation, whether an invalid path errors or returns empty, or any cost/latency characteristics of fetching whole documents.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One sentence, front-loaded with the verb and resource, with zero redundant wording. Nothing to trim and nothing buried.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter read tool with an output schema that covers return values, the description is nearly sufficient; the remaining gap is the missing explicit hand-off from search_documentation to this tool. The schema compensates for path semantics, so completeness is high.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single `path` parameter is documented with concrete examples and its provenance (returned in search results), so the schema already does the work. The description adds nothing about path format or validation beyond that, which is the correct baseline for a fully described single parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

"Read the full content of a specific Spark documentation page" gives a specific verb (read) and resource (documentation page) and scopes it to a single page, which implicitly separates it from search_documentation. It stops short of naming the sibling outright, so an agent must infer the contrast.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the phrase "a specific page" plus the schema note that paths are returned in search results suggests this is the follow-up call after a search. There is no explicit statement of when to use this versus search_documentation, nor any precondition about obtaining a path first.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_documentationC

Search Apache Spark documentation by keyword query.

NameRequiredDescriptionDefault
limitNoMaximum number of results to return (default: 10, max: 50).
queryYesSearch terms to find in the documentation. Supports full-text search with stemming (e.g., "stream" matches "streaming", "streams").
sectionNoOptional section to filter results. Common sections include: 'sql-ref', 'api', 'streaming', 'mllib', 'graphx', 'structured-streaming', etc.

Output Schema

NameRequiredDescription
resultYes

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations supplied, the description carries the full behavioral burden and discloses almost nothing: no ranking behavior, result shape, pagination, or scope limits. The only behavioral signal is 'keyword query', which is thin for a tool with no annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler, so it is efficient and easy to scan. It is arguably too terse for the tool's role, but that gap is captured under other dimensions rather than as verbosity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple: three fully documented parameters and an output schema, so return values need no explanation here. The remaining gap is routing guidance against read_documentation and any note on search behavior, which the description omits.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the query, limit, and section parameters are already fully documented, including stemming behavior and common section values. The description adds no parameter meaning beyond that, which meets the baseline for fully covered schemas.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb and resource ('Search Apache Spark documentation by keyword query'), so an agent knows exactly what the tool does. It does not differentiate from the sibling read_documentation, leaving the search-vs-read choice to the name alone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no statement of when to use this tool versus read_documentation, nor any prerequisite or exclusion. The only hint is the phrase 'keyword query', which implies lookup-by-term usage but is never made explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 2 tool updatesv0.3.7
    • Changedread_documentation1 field changed
      • addedInput schema / properties / path / description
        Added value: +"The relative path to the documentation file (e.g.,\n  'sql-ref/sql-syntax.md' or 'api/python/index.md'). This path\n  is returned in search results."
    • Changedsearch_documentation3 fields changed
      • addedInput schema / properties / limit / description
        Added value: +"Maximum number of results to return (default: 10, max: 50)."
      • addedInput schema / properties / query / description
        Added value: +"Search terms to find in the documentation. Supports\n   full-text search with stemming (e.g., \"stream\" matches\n   \"streaming\", \"streams\")."
      • addedInput schema / properties / section / description
        Added value: +"Optional section to filter results. Common sections include:\n     'sql-ref', 'api', 'streaming', 'mllib', 'graphx',\n     'structured-streaming', etc."
  2. 2 tool updatesv0.2.6
    • Changedread_documentation1 field changed
      • addedInput schema / additionalProperties
        Added value: +false
    • Changedsearch_documentation1 field changed
      • addedInput schema / additionalProperties
        Added value: +false
  3. 2 tool updatesv0.1.0
    • First observedread_documentation
    • First observedsearch_documentation

TDQS

A3.5/5.0

Scored across 2 tools

Disambiguation5/5

search_documentation and read_documentation have clearly complementary, non-overlapping purposes: one finds pages, the other fetches a specific page's content. An agent can trivially pick the right one based on whether it has a page identifier or only a query.

Naming Consistency5/5

Both tools follow the same snake_case verb_noun pattern (search_documentation, read_documentation), with consistent verb style and a shared noun suffix.

Tool Count4/5

Two tools is a tight, purposeful surface for a documentation-lookup server, with no redundancy. It is slightly thin — discovery aids like listing pages or versions are absent — but nothing feels gratuitous.

Completeness4/5

Search-then-read covers the core documentation workflow end to end, with no dead ends for locating and consuming content. Minor gaps exist around browsing structure (listing available pages, versions, or sections), which the search tool partially compensates for.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    A
    quality
    B
    maintenance
    Provides fast, token-efficient search over coding agent documentation (e.g., Claude Code, Cursor) using local SQLite FTS5 indexing, with tools for searching snippets, reading pages, and grepping markdown.
    5
    MIT
  • A
    license
    Not graded
    quality
    B
    maintenance
    Enables AI assistants to run incremental SQLite FTS5 full-text search over explicitly configured local workspace roots, covering Markdown, code, PDF, DOCX, XLSX, CSV and HTML with BM25 ranking, Chinese bigrams, and phrase, prefix, path, type and date filters. Exposes search, safe root-confined reads with line/page/sheet locations, recent-file listing, and reindexing as MCP tools so assistants can locate and quote source content without network or account access.
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Enables AI agents to search a local folder of Markdown, AsciiDoc, and text files through a SQLite FTS5 keyword index, returning the most relevant sections with file paths, line ranges, and heading breadcrumbs instead of grepping through a repo. It also exposes tools to list indexed documents and refresh the index incrementally, all offline with no dependencies, network calls, or embeddings.
    3
    MIT