Skip to main content
Glama
deenesjakoruzh
shantanu-deshmukh

ChunkTuner

by shantanu-deshmukh
OverviewSchemaRelated ServersScoreDiscussions
Python
Local

chunktuner

PyPI version Python versions License: MIT CI Docs

Auto chunking tuner and MCP server for RAG pipelines.

Give it your documents. It tries multiple chunking strategies, measures which setup supports retrieval best, and recommends a configuration for your corpus and use case. Zero API cost to start — run estimate for a dry-run before any paid calls.

Full documentation: shantanu-deshmukh.github.io/chunktuner

flowchart TD
    Lib["Python library"] --> Ingest
    CLI["CLI (chunk-tune)"] --> Ingest
    MCP["MCP server"] --> Ingest

    Ingest["Ingest your documents<br/>files, URLs, repos"] --> Tune

    subgraph Tune ["AutoTuner: for every strategy and param set"]
        direction LR
        Chunk["Chunk document"] --> Embed["Embed chunks<br/>and queries"] --> Score["Score retrieval<br/>recall, MRR, NDCG"]
    end

    Tune --> Rank["Rank all configs<br/>against baseline"] --> Best(["Recommended config<br/>.autochunk.yaml"])

What it does

When building a RAG pipeline, how you split documents into chunks directly impacts retrieval quality. chunktuner automates the process of finding the optimal chunking strategy for your specific corpus, embedding model, and use case.

It benchmarks strategies like fixed-token windows, recursive character splitting, semantic splitting, PDF structural chunking, and AST-based code chunking — then scores each one against real retrieval metrics (token recall, MRR, NDCG) and optional generation metrics (RAGAS faithfulness, answer relevancy).

Related MCP server: golden-dataset-mcp

Interfaces

  • Python library — programmatic integration into your pipeline

  • CLI (chunk-tune) — human-driven tuning from the terminal

  • MCP server — use directly from Claude Desktop or any MCP host

Quickstart

# Install (pick one)
uv tool install chunktuner
pip install chunktuner

# Initialize workspace (embedding_model defaults to null — no API calls)
chunk-tune init

# See cost estimate before running anything
chunk-tune estimate ./my_docs --use-case rag_qa

# Get a recommendation (dummy embeddings by default; add --embedding-model for real ones)
chunk-tune recommend ./my_docs --use-case rag_qa

Python API:

from pathlib import Path
from chunktuner import FileIngestor, DummyEmbeddingFunction, LiteLLMEmbeddingFunction, AutoTuner
from chunktuner import default_registry, Evaluator, ScoreCalculator

docs = FileIngestor().ingest_dir(Path("./my_docs"))

# Free/offline: use dummy embeddings for quick strategy comparison.
# Swap in LiteLLMEmbeddingFunction for real embeddings with any provider:
#   LiteLLMEmbeddingFunction("text-embedding-3-small")          # OpenAI
#   LiteLLMEmbeddingFunction("gemini/gemini-embedding-001")     # Google
#   LiteLLMEmbeddingFunction("openai/<id>", api_base="http://localhost:1234/v1")  # local
embedding_fn = DummyEmbeddingFunction()

tuner = AutoTuner(
    strategies=default_registry,
    evaluator=Evaluator(embedding_fn),
    scorer=ScoreCalculator(use_case="rag_qa"),
)
result = tuner.recommend(docs, use_case="rag_qa")
print(result.best.config)

Example output

After running recommend, you get a ranked table with the winning config and how much it beats the baseline:

  Rank   Strategy              Params                  Score   Recall   MRR    IOU   AvgTok
 ────────────────────────────────────────────────────────────────────────────────────────
   1 ★   recursive_character   1024 chr / 154 ov        0.821    0.950  0.880  0.062      212
     2   fixed_tokens          512 tok / 51 ov           0.764    0.920  0.840  0.059      444
   ...
  Baseline  fixed_tokens  512 tok / 0 ov  →  score 0.682
  Winner beats baseline by +0.139  (+20.4%)

Real-world example

See examples/financial_analysis for a full benchmark on S&P 500 earnings call transcripts — a corpus where separator choice and chunk size make a measurable difference in retrieval quality.

Run it offline with zero API cost:

cd examples/financial_analysis
uv sync
uv run python run_benchmark.py --fixture --num-transcripts 2

Supported strategies

Strategy

Best for

fixed_tokens

Baseline; uniform token windows

recursive_character

General prose and documentation

semantic

Theme-heavy articles

markdown_semantic

Structured Markdown docs

pdf_structural

PDFs with layout regions and tables

structural_semantic

PDF/DOCX with mixed layout and text

late_chunking

Long docs with dense cross-references

agentic

High-value narrative documents

code_ast

Code repos (Python, JavaScript)

code_window

Code baseline (sliding window)

MCP server (Claude Desktop)

Python FastMCP (chunk-tune-mcp, stdio). No Node.js build. See docs/mcp_setup.md.

Add to your .mcp.json:

{
  "mcpServers": {
    "chunktuner": {
      "command": "uvx",
      "args": ["--from", "chunktuner[mcp]", "chunk-tune-mcp"],
      "env": {
        "CHUNK_TUNER_BASE_DIR": "/path/to/your/corpus"
      }
    }
  }
}

Tools available: list_strategies, preview_chunks, evaluate_chunking, recommend_config.

CLI reference

chunk-tune init       Bootstrap workspace config
chunk-tune analyze    Quick structural scan (no API cost)
chunk-tune estimate   Dry-run cost/token estimate
chunk-tune evaluate   Full evaluation across strategies
chunk-tune recommend  Evaluation + best config recommendation
chunk-tune compare    Side-by-side comparison of specific strategies
chunk-tune preview    Inspect how a strategy splits a document
chunk-tune cache      Manage embedding and chunk cache

Installation options

pip install chunktuner                 # CLI + library
uv add chunktuner                      # library
uv tool install chunktuner             # global CLI
uvx --from chunktuner chunk-tune …     # ephemeral CLI (no install)

# With optional extras
pip install "chunktuner[docling]"      # PDF/DOCX support
uv add "chunktuner[docling]"           # PDF/DOCX support
uv add "chunktuner[ragas]"             # generation metrics
uv add "chunktuner[semantic]"          # semantic chunking
uv add "chunktuner[code]"              # AST code chunking
uv add "chunktuner[all]"               # everything

Contributing

See CONTRIBUTING.md.

Author

Shantanu Deshmukh — full stack developer building E2E AI applications.

LinkedIn / Twitter

Available Tools

4 tools
evaluate_chunkingB

Dry-run cost estimate or full evaluation (DummyEmbeddingFunction if no model).

ParametersJSON Schema
NameRequiredDescriptionDefault
pathYes
top_kNo
dry_runNo
max_docsNo
use_caseNorag_qa
strategiesNo
content_typeNo
embedding_modelNo

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the burden. It discloses the use of a DummyEmbeddingFunction when no model is given, which is a key behavior. However, it does not mention whether the tool is read-only, side effects, or other important traits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence, concise and front-loaded. It could be more structured (e.g., listing modes separately), but it is not unnecessarily verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 8 parameters, no output schema, and no parameter descriptions, the description is far too minimal. It fails to explain return values, how results are presented, or how parameters interrelate, making it incomplete for effective use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%—all 8 parameters have only titles. The description adds no explanation of any parameter, leaving their meaning entirely to the schema with no additional context.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool can perform a dry-run cost estimate or a full evaluation, and mentions using a DummyEmbeddingFunction if no model is provided. This differentiates it from siblings like list_strategies and preview_chunks.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use each mode (dry_run vs full evaluation), but provides no explicit guidance on when to choose this tool over its siblings. No exclusions or alternatives are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_strategiesB

List registered chunking strategies, optionally filtered by content type.

ParametersJSON Schema
NameRequiredDescriptionDefault
content_typeNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It does not state that the tool is read-only (safe), nor does it mention any side effects, authentication requirements, or rate limits. The agent can infer that listing is non-destructive, but this is not explicit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence of 10 words. It is front-loaded with the action and resource, and every word contributes meaning. There is no fluff or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simplicity of the tool (one optional parameter, output schema exists), the description provides the essential functionality. However, it lacks context about what 'chunking strategies' are and how they are registered. For a standalone tool, this might be insufficient without additional documentation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description adds minimal value beyond the input schema. It merely paraphrases 'optionally filtered by content type', which mirrors the schema property. Since schema description coverage is 0%, the description should compensate with more detail (e.g., allowed values, behavior when null), but it does not.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('List'), the resource ('registered chunking strategies'), and includes a filtering condition ('optionally filtered by content type'). It effectively differentiates from sibling tools like evaluate_chunking, preview_chunks, and recommend_config by focusing on listing rather than evaluation, preview, or recommendation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no explicit guidance on when to use this tool versus its siblings. There are no 'when-to-use' or 'when-not-to-use' statements, and no mention of alternatives. The agent is left to infer usage context from the tool name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

preview_chunksC

Chunk inline text with one strategy + params (no embeddings).

ParametersJSON Schema
NameRequiredDescriptionDefault
textYes
configNo
strategy_nameYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

C2.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must bear the full burden. It implies no embeddings and inline processing, but does not explicitly state that no data is persisted or that the operation is idempotent. The output schema exists but behavioral side effects are not addressed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence, very concise. However, it sacrifices useful details that could be added without becoming verbose, such as examples or config structure.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given three parameters, zero schema descriptions, and a sibling set, the description is insufficient. While the output schema covers return values, the lack of parameter guidance and usage context makes it incomplete for an AI agent to use correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%. The description only hints that 'strategy_name' selects a strategy and 'config' holds parameters, but does not explain the format or constraints for 'config' (anyOf object/null). No enum values or examples are provided.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('chunk'), the resource ('inline text'), and includes a distinguishing detail ('no embeddings'). However, it could better differentiate from sibling tool 'evaluate_chunking' which likely involves evaluation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given on when to use this tool versus siblings like 'evaluate_chunking', 'list_strategies', or 'recommend_config'. The description does not specify prerequisites or contexts.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

recommend_configC

Run tuner and return ranked Recommendation (uses dummy embeddings by default).

ParametersJSON Schema
NameRequiredDescriptionDefault
pathYes
top_kNo
max_docsNo
use_caseNorag_qa
strategiesNo
content_typeNo
embedding_modelNo

TDQS

C2.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry the full burden. It discloses the use of dummy embeddings by default, but does not mention side effects, idempotency, or other behavioral traits like whether it modifies state or runs asynchronously.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise at one sentence, but at the cost of missing critical information. It is structured well enough for a simple statement, but could be expanded to include essential details without becoming verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 7 parameters, no output schema, and no annotations, the description is severely incomplete. An agent cannot determine what the tool returns, how to set parameters correctly, or what the expected behavior is beyond a vague 'tuner' operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description should explain parameters. It does not mention path, top_k, max_docs, use_case, strategies, content_type, or embedding_model, leaving all semantics to be inferred from parameter names alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description says 'Run tuner and return ranked Recommendation', which vaguely states a tuning action but does not define what a 'tuner' or 'Recommendation' is in this context. The phrase 'uses dummy embeddings by default' adds some specificity but the core purpose remains unclear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus its siblings (evaluate_chunking, list_strategies, preview_chunks). The description does not mention alternatives or conditions for use, leaving the agent to infer.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Install Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

B3.2/5.0
Disambiguation5/5

Each tool has a distinct purpose: evaluating, listing, previewing, and recommending. No overlap in functionality.

Naming Consistency5/5

All tools follow a consistent verb_noun pattern (evaluate_chunking, list_strategies, preview_chunks, recommend_config).

Tool Count5/5

Four tools is appropriate for a chunking tuner server, covering the core workflow without being too sparse or overwhelming.

Completeness4/5

The set covers listing, previewing, evaluating, and recommending chunking configurations. A minor gap is the lack of explicit strategy management, but it may be predefined.

Maintenance

ActivityStale
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • ppb-mcp

    paulplee
    A
    license
    Not graded
    quality
    D
    maintenance
    Exposes queryable GPU inference benchmark data (quantization, throughput, VRAM, concurrent users) as tools for LLM clients.
    MIT

  • golden-dataset-mcp

    nipunkhanderia
    A
    license
    A
    quality
    B
    maintenance
    Enables version-controlled golden dataset management and semantic evaluation for RAG/LLM pipelines using TF-IDF cosine similarity, without requiring an LLM API key.
    9
    MIT

  • LLM Evaluation Harness MCP

    SaiArja
    A
    license
    Not graded
    quality
    C
    maintenance
    Evaluates RAG outputs on faithfulness, answer relevancy, and context precision using an LLM-as-a-Judge backend. Exposes tools for running evaluations, scoring individual samples, and checking thresholds, enabling CI gating and on-demand assessment via MCP.
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/shantanu-deshmukh/chunktuner'

If you have feedback or need assistance with the MCP directory API, please join our Discord server