Token Optimization System
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
Token Optimization System is a small MCP server for Claude Desktop. It indexes folders you choose on your own computer. When Claude needs something from those files, it asks the server, and the server returns the few sections that answer the question instead of whole documents. Claude reads less, so a conversation uses less of its context window and of your plan's usage.
Everything runs on your machine. The index is one SQLite file in the install folder. There is no account, no telemetry and no network connection.
Example
You ask Claude: "What acceptance criteria did we set for the chunking experiment?"
Claude calls optimizer_context with the question. The server returns two sections, about 1,500 tokens, each under a short file label, instead of whole files. How much that saves depends on how Claude would otherwise find the answer; see the measured result below.
Related MCP server: MCP-RAG
Measured result
One controlled test so far, run in Claude Code (not in Claude Desktop). It asked 20 questions about this project's own 39 Markdown notes and compared three setups: no server, the server available, and the server available with a one-line instruction to use it. Each question was answered once per setup, and Claude Code's own cost estimate was compared question by question. The rule for the verdict was written before the first run.
With the server and the instruction, the estimated cost of the answers was 41% lower in total (0.24 against 0.41 dollars for the 20 questions). It was lower in 17 of the 19 questions that both setups answered correctly, and the median question cost 47% less.
The median run put 31% fewer tokens into the context (about 11,000 against 16,000).
Answers were correct in 19 of 20 runs with the server and in 20 of 20 without it. The one miss was a wording mismatch in the automatic check, not a missing section.
Without the instruction, Claude still called the server in 18 of 20 runs.
What this does not show: the notes are 39 small files and the questions were written from them, so their wording overlaps closely with the text, which suits keyword search. Without the server, Claude Code searched with grep and read parts of files, which is a cheap baseline on small files. Results on large documents, in other clients or with your own questions can differ, in either direction. The raw data is not published.
After the test, in two weeks of real work in this project's folder (2026-10-28 to 2026-11-10), Claude Code did not call the server on its own in any of 21 sessions; in normal sessions Claude Code shows MCP tools by name only until it searches for them, while the test runs had the tool descriptions loaded from the start.
Tools Claude gets
Tool | What it does |
| The main tool. Returns the most relevant sections for a question, at most |
| Lists the best-matching sections with their file and heading. |
| Returns one indexed section, chosen by file and heading. |
| Shows what is indexed and how many tokens were delivered compared with the full files. The percentage it prints is a local estimate of what whole files would have cost. It is not what Claude spends without the server (in the test above, Claude Code searched and read parts of files), so it is higher than the saving measured there. |
Supported files: Markdown, Word (.docx), PDF, PowerPoint (.pptx), Excel (.xlsx), plain text, CSV, JSON, YAML, HTML, CSS, Python, JavaScript and TypeScript. Markdown and Word files are split by heading, PDFs by page, presentations by slide and spreadsheets by sheet.
Requirements
Windows 10 or 11
Python 3.10 or newer, from python.org
Install
Each step shows the command to run and what you should see. If a step does not end as shown, stop there and read Troubleshooting.
1. Download this repository. Either click Code > Download ZIP, then right-click the file and choose Extract All. Or, with Git installed:
git clone https://github.com/HEX-1000/token-optimization-system.git2. Open PowerShell in the downloaded folder.
Open the folder in File Explorer (the one that contains README.md), click the address bar, type powershell and press Enter. Check that you are in the right place:
dir README.mdYou should see README.md listed.
3. Install.
powershell -ExecutionPolicy Bypass -File scripts\install.ps1This copies the program to %USERPROFILE%\Programs\TokenOptimizationSystem, creates a private Python environment there and installs its packages. It ends with Installed. Next steps:.
4. Index a folder. Replace the path with a folder of your documents:
powershell -ExecutionPolicy Bypass -File scripts\index.ps1 -Folder "C:\Users\you\Documents\Notes"It lists the files it reads and ends with a summary in which Failed files: is 0. Repeat this step for every folder you want Claude to search.
5. Quit Claude Desktop completely. Right-click its icon in the system tray and choose Quit. Closing the window is not enough.
6. Connect Claude Desktop.
powershell -ExecutionPolicy Bypass -File scripts\connect-claude-desktop.ps1It makes a backup of Claude Desktop's config file, adds one entry named token-optimization-system, and ends with Done.
7. Start Claude Desktop and test. Under Settings > Developer, token-optimization-system should show as running. In a new chat, ask: "Run optimizer_stats." The answer starts with Token Optimization System v0.25.2 and shows your file count.
Tip: Claude uses the tools when it thinks they help. To make that the default, add this to your project instructions in Claude: "Before reading local files, call optimizer_context with the question. Read whole files only if the returned sections are not enough."
Keeping the index current
Run step 4 again after files change. Only new or changed files are read again. Add -Rebuild to index a folder from scratch.
Update
Download the new version (or run git pull if you cloned it), quit Claude Desktop from the tray icon so the old server stops, and run step 3 again. The program files are replaced and your index is kept. Then start Claude Desktop; the first line of the optimizer_stats answer shows the version, and CHANGELOG.md lists what changed. Nothing checks for updates for you, because the server makes no network connections.
Uninstall
Quit Claude Desktop and run
powershell -ExecutionPolicy Bypass -File scripts\connect-claude-desktop.ps1 -Undo.Delete
%USERPROFILE%\Programs\TokenOptimizationSystem. This also deletes the index.
Troubleshooting
Message or symptom | What to do |
| Run the commands exactly as shown, including |
| Install Python from python.org with Add python.exe to PATH ticked, open a new PowerShell window and repeat step 3. |
| Quit it from the tray icon, not just the window. |
| Run step 6 again with |
| In Claude Desktop open Settings > Developer > Edit Config once, quit Claude Desktop, and repeat step 6. |
A tool answers with | Nothing is indexed yet. Run step 4, then ask again. |
The server shows as failed in Claude Desktop | Python from the Microsoft Store sometimes fails when another app starts it. Install Python from python.org and repeat step 3 with |
Limitations
Search is keyword based (SQLite FTS5 with BM25). It finds sections that share words with the question, not paraphrases. There are no embeddings yet.
Queries are reduced to the letters a to z and digits, so words with other letters (ö, é, ß) do not match. English documents work best.
Text files other than Markdown (.txt, .json, .py and so on) are stored as one piece, which can be large. Folders of Markdown, Word, PDF, PowerPoint or Excel files work best.
Tables and speaker notes in Word and PowerPoint files are not extracted yet.
The scripts are for Windows. The server is plain Python and may run elsewhere, but that is untested.
Retrieval quality was measured on a small benchmark: 54 sections and 89 questions. Results on your files will differ.
Privacy
Indexing reads only the folders you pass to
index.ps1.The index is stored in
%USERPROFILE%\Programs\TokenOptimizationSystem\data\optimizer.db.When Claude asks a question, the server reads only its own index. It does not open files on disk and makes no network connections.
The sections the server returns become part of your conversation with Claude, like anything else you share with Claude. Index only what you would share.
How it works
Index. Files are split into sections by structure and stored in a SQLite full-text index.
Retrieve. The question is matched against the index in several passes (all words, any word, word prefixes).
Rank. Candidates are scored on how much of the question they cover, heading matches, phrase matches and BM25 relevance.
Select. The best section is returned only if its score clears a fixed threshold; otherwise the server says nothing matched. Further sections are added only when they add new information.
Return. Selected sections go back to Claude in a compact format.
Details and the evaluation method: docs/how-it-works.md.
Where this is going
This is free, open-source software, and it stays usable without contributing anything back. Every installation works alone, with its own index.
The long-term plan is a privacy-safe way to learn from many installations. Users who opt in would share experiment results, such as a method, its settings and aggregate scores, never their documents, queries or file names. Improvements found on one machine could then be tested and released for everyone. None of this exists yet. The current stage is a reliable local release; a release process, an experiment format and opt-in contribution come next, in that order.
Versions
Current: 0.25.2, retrieval V0.25.0 with compact output. See CHANGELOG.md.
Development and experiments happen in a separate repository. Changes reach this one after they are measured on the evaluation suite. Bug reports are welcome as issues, questions and ideas in Discussions. Both are public, so do not paste document text, file names or folder paths there. If you tried the server on your own files, the "Report a result" form asks for numbers and a general description. It never asks for document text, file names or your questions.
License
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
Maintenance
Related MCP Connectors
Token-efficient search for coding agents over public and private documentation.
Ingest, manage, and retrieve documents for RAG-powered AI applications
Provide your AI coding tools with token-efficient access to up-to-date technical documentation for…
Private, multimodal knowledge store for agents: hybrid search with cited, provenance-backed results.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables real-time indexing and semantic search of local documents (PDF, Word, text, Markdown, RTF) using vector embeddings and local LLMs. Monitors folders for changes and provides natural language search capabilities through Claude Desktop integration.22MIT
- AlicenseNot gradedqualityNot gradedmaintenanceTurns Claude Desktop into a personal document question-answering system using local vector search. Index PDF, TXT, and Markdown documents into collections and get answers based strictly on your documents with zero hallucination.10 npm-
- AlicenseNot gradedqualityDmaintenanceEnables Claude Desktop to search and query personal document collections (PDF, Word, Markdown, text) using semantic search and conversational AI with full context preservation across exchanges.MIT
- AlicenseAqualityAmaintenanceEnables local semantic search over documents and code for Claude Code and Claude Desktop, running entirely offline with local embeddings and vector storage.14424 PyPI7MIT