Allows searching and fetching documentation from LangGraph's llms.txt index, providing ranked BM25 search results and on-demand content retrieval for LangGraph developer guides.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@llmstxt-doc-search search for 'MCP transport' across all sources"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Live, ranked search across any number of
llms.txtdocumentation sites - Strands, Kiro, the AWS guides, and whatever you add at runtime.
llmstxt-doc-search is a Model Context Protocol (MCP) server that turns the llms.txt index a documentation site publishes into a fast, ranked search tool your agent can call. It indexes titles at startup, ranks queries with BM25, and fetches the full document only when you open a result - so you get current docs with almost no local storage. Built on the search engine from @praveenc/mcp-docs-server, generalized to a runtime registry of sources.
An llms.txt file is a curated index of a doc site's pages, published for tools like this one to consume. They can be large - AWS Bedrock's lists roughly a thousand documents - so downloading everything is wasteful and goes stale fast.
This server takes a leaner approach:
Title-only index, built lazily. On first search of a source, only the page titles are indexed. That is fast to build and tiny to hold in memory.
Ranked with BM25. Queries are scored with BM25 plus Porter stemming, bigrams, and markdown-aware weighting (headers, code, and links count for more). Technical terms like mcp, json, and stdio are preserved rather than stemmed.
Content on demand. The full markdown or HTML of a result is fetched only when you call fetch_doc.
The result is a good fit for broad, fast-moving reference material - the opposite tradeoff to snapshotting docs into a local vault.
Related MCP server: MCP Docs Server
Add the server to your MCP client configuration (Claude Desktop, Kiro, and others). It is downloaded and run on demand via npx - no manual build:
{
"mcpServers": {
"llmstxt-doc-search": {
"command": "npx",
"args": ["-y", "@praveenc/llmstxt-doc-search"]
}
}
}npm install -g @praveenc/llmstxt-doc-searchThen point your MCP client at the installed binary:
{
"mcpServers": {
"llmstxt-doc-search": {
"command": "llmstxt-doc-search"
}
}
}Once the server is connected, the typical flow is three calls:
docs_home() - orient yourself: see the registered sources and how to search and fetch.
search_docs("prompt caching", "aws-bedrock-userguide") - rank matching docs. Omit the source to search everything.
fetch_doc(url) - read the full content of a result you like.
Add your own source at any time and it is indexed immediately and persisted for future runs:
add_doc_source("langgraph", "https://langchain-ai.github.io/langgraph/llms.txt")Tool | Purpose |
| Orientation: registered sources plus how to search and fetch. Call this first. |
| List sources with their |
| BM25 search. Omit |
| Fetch the full content of a result URL. The URL must belong to a registered source. |
| Register and index a new |
| Remove a registered source. |
| Re-index a source to pick up new or changed docs. |
Seeded into the registry on first run:
strands, kiro, aws-bedrock-userguide, aws-agentic-ai-lens, aws-bedrock-agentcore-devguide, mcp.
The registry is persisted at ~/.config/llmstxt-doc-search/sources.json (override with LLMSTXT_REGISTRY_PATH). Anything you add, remove, or refresh at runtime is saved there.
All configuration is via environment variables; none are required.
Variable | Default | Meaning |
|
| Where the source registry is persisted. |
|
| How many top hits to fetch when building result snippets. |
|
| Log verbosity: |
npx @modelcontextprotocol/inspector npx -y @praveenc/llmstxt-doc-searchClone the repository for local work:
git clone https://github.com/praveenc/llmstxt-doc-search.git
cd llmstxt-doc-search
npm installnpm run dev # run from source with tsx (no build)
npm test # offline unit tests
npm run typecheck # type-check without emitting
npm run build # compile to dist/
npm run inspect:dev # MCP Inspector against the sourcePoint your client at a source checkout instead of the published package:
{
"mcpServers": {
"llmstxt-doc-search": {
"command": "npx",
"args": ["tsx", "/ABS/PATH/llmstxt-doc-search/src/index.ts"]
}
}
}Or, after npm run build, at the compiled entry point:
{
"mcpServers": {
"llmstxt-doc-search": {
"command": "node",
"args": ["/ABS/PATH/llmstxt-doc-search/dist/index.js"]
}
}
}src/
├── index.ts # MCP server entry point and tool registration
├── config.ts # Defaults and environment configuration
├── tools/
│ └── docs.ts # search_docs, fetch_doc, and source management
└── utils/
├── doc-fetcher.ts # HTTP fetching, redirect handling, HTML parsing
├── indexer.ts # BM25 search index
├── registry.ts # Persisted source registry
├── store.ts # In-memory document store
├── text-processor.ts # Tokenization and snippet helpers
├── url-validator.ts # SSRF guard and URL validation
├── stopwords.ts # Stop-word list
└── logger.ts # Logging utilitiesRanking uses BM25 (Best Matching 25) with several enhancements:
Porter stemming matches word variants (for example, running and run).
Bigrams capture phrase matches (for example, prompt caching).
Weighted scoring boosts title matches (3-8x), headers (4x), code blocks (2x), and link text (2x).
Domain-term preservation keeps technical terms like mcp, json, and stdio unstemmed so they match exactly.
This server fetches user-supplied URLs at runtime, so its SSRF surface is guarded in depth:
Scoped fetches. fetch_doc only retrieves URLs under a registered source's origin and path prefix, matched on a path boundary rather than a raw string prefix. There is no arbitrary fetch.
Scheme allow-list. Non-http(s) schemes are rejected.
Range-based address blocking. Private and reserved destinations are blocked using IP range classification (ipaddr.js), covering decimal, octal, and hex IPv4, IPv4-mapped IPv6, loopback, link-local, unique-local, carrier-grade NAT, and other reserved ranges - not just a hostname regex.
Connection-time validation. The resolved IP is checked at connection time via a custom DNS lookup, closing DNS-rebinding, and every redirect hop is re-validated.
Bounded responses. Response bodies are capped at 10 MB to limit memory and regular-expression (ReDoS) exposure.
Runtime dependencies report zero known vulnerabilities.
MIT - Copyright (c) 2026 Praveen Chamarthi
Contributions are welcome. If you find a bug or have an idea:
Open an issue describing the problem or proposal.
For code changes, fork the repo and create a feature branch.
Keep changes focused, add or update tests, and make sure npm test, npm run typecheck, and npm run build all pass.
Open a pull request against main with a clear description of what changed and why.
Commit messages follow the Conventional Commits style.
Questions and ideas: open a GitHub issue.
Bugs: please include your MCP client, the tool call you made, and any relevant logs (set LLMSTXT_LOG_LEVEL=debug for more detail).
Security issues: open an issue marked as security-sensitive, or contact the maintainer directly rather than posting exploit details publicly.
Register a new llms.txt source at runtime and index it. Persisted for future runs.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Short id, e.g. 'langgraph' | |
| llms_txt_url | Yes | URL of the source's llms.txt (https) |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses key behaviors: runtime registration, indexing, and persistence for future runs. However, it does not address potential side effects like duplicate name handling, idempotency, or error conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences that are front-loaded with the primary action and followed by the persistence behavior. Every word earns its place; no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given only two simple parameters and no output schema, the description is adequately complete for a registration tool. It explains what happens (indexing) and persistence across runs, which covers the main context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, as both `name` and `llms_txt_url` have descriptions. The tool description adds no further semantic detail beyond the schema, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool's action: 'Register a new llms.txt source at runtime and index it.' It uses a specific verb (register) and resource (llms.txt source), and the persistence note distinguishes it from sibling tools like remove_doc_source and refresh_doc_source.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The context is clear: this is for adding a new source at runtime, as opposed to listing, searching, fetching, removing, or refreshing. However, it does not explicitly mention when not to use it (e.g., for an existing source that needs update) or explicitly name alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Orientation: registered llms.txt sources + how to search/fetch. Call this first.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of behavioral disclosure, but it only states what the tool is, not what it does or its safety profile. It doesn't mention whether it's read-only, what output to expect, or any limitations, leaving the agent to infer safe behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, front-loading 'Orientation' and using a second sentence for critical usage guidance. It wastes no words, though the terse style sacrifices some clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, no-output-schema orientation tool, the description is minimally adequate. It tells the agent what the tool covers and to call it first, but doesn't specify the format or detail of the orientation content, which could lead to uncertainty.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline is 4 per the rubric. The description correctly doesn't add parameter details, as there are none to explain.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies this as an orientation tool for registered llms.txt sources and how to search/fetch, which distinguishes it from sibling action-oriented tools. However, it lacks a strong verb (e.g., 'provides orientation') and is somewhat noun-phrase based.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The explicit instruction 'Call this first' provides clear when-to-use guidance as an entry point, and the mention of covering search/fetch implies its role relative to siblings. It doesn't explicitly name alternatives or exclusions, but the imperative is strong.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Fetch full content of a doc url. The url must belong to a registered source (use search_docs first). Content is fetched live.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Document URL from a search_docs result |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden. It discloses that content is 'fetched live' and full content is returned, providing useful behavioral context. It doesn't explicitly state read-only or error behavior, but the fetch semantics are clear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the main verb and resource. Every word earns its place: no filler, fluff, or repetition of the schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter tool with no output schema, the description fully covers what the tool does, how the input is constrained, and a key behavioral trait (live fetch). No further context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents the single parameter well (100% coverage), including its format and origin. The description adds a constraint ('must belong to a registered source') but does not significantly augment the schema's meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific action ('Fetch full content of a doc url') and the resource type. It distinguishes from sibling tools like search_docs, which search for docs, and list_doc_sources, which list sources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly specifies the prerequisite 'use search_docs first' and the constraint that the url 'must belong to a registered source', providing clear when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List the registered documentation sources (name, llms.txt url, index status).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of disclosing safety and behavior. 'List' implies non-destructive behavior, but no additional context is given about ordering, pagination, or whether sources are from a local or remote registry. The output fields provide some transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, focused sentence that front-loads the key information (action, object, output details) without any wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with no parameters, the description covers the essential purpose and expected output fields. It lacks details like whether the list is exhaustive or if there are additional filters, but these are not critical for a basic listing operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool accepts zero parameters, so the baseline is 4. The description does not need to add parameter-level detail since there are none, and the schema is empty.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List') and the resource ('registered documentation sources'), and specifies the output fields (name, llms.txt url, index status). This distinguishes it from sibling tools like add_doc_source or remove_doc_source, which perform mutations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage as a read-only listing tool, but it does not explicitly state when to use it instead of alternatives. Sibling names suggest it complements add/remove/refresh, but no direct guidance or exclusions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Re-index a source from its llms.txt (pick up new/changed docs).
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Source name to refresh |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden of behavioral disclosure. It communicates the action ('Re-index') and benefit ('pick up new/changed docs') but does not disclose potential side effects (like whether absent docs are removed), requirements (source existence), or whether this is a destructive operation. This is a significant gap for a mutation-like tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that clearly states the action and benefit. Every word earns its place; there is no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter tool, the description is minimally adequate: it names the action, source of truth, and expected outcome. However, with no annotations or output schema, it lacks details on failure modes, idempotency, and whether the refresh is incremental or full. Given the low complexity, this is acceptable but not outstanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with parameter 'name' described as 'Source name to refresh', which aligns with the tool's description. The description adds no extra parameter semantics beyond what the schema already provides, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Re-index' with a clear resource 'a source from its llms.txt', and the parenthetical 'pick up new/changed docs' further clarifies the exact purpose. This distinguishes it well from sibling tools like add_doc_source, remove_doc_source, and search_docs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool (when new or changed docs need to be picked up from a source's llms.txt) but does not explicitly state when not to use it or mention alternatives (e.g., re-adding the source). Usage context is clear but exclusionary guidance is missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Remove a registered source.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Source name to remove |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description carries full burden. 'Remove' clearly indicates a deletion action, but it does not disclose whether the removal is reversible, if it cascades to related data, or if any confirmation is involved. This is acceptable for a simple operation but lacks extra context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that immediately conveys the action. There is zero redundancy, and it is front-loaded with the verb. For a tool with one parameter, this level of conciseness is optimal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple removal tool with a well-documented schema and no output schema, the description is sufficiently complete to convey core functionality. However, it lacks any note about side effects or post-conditions, which would make it more robust, especially as a mutation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one required parameter 'name' with the description 'Source name to remove', providing 100% coverage. The tool description adds no additional meaning to the parameter beyond what the schema already specifies, so it receives the baseline score for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action (Remove) and target (a registered source), using a specific verb and resource pair. This distinguishes it from siblings like add_doc_source (adds), refresh_doc_source (refreshes), and list_doc_sources (lists). No ambiguity remains about the tool's purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when a source needs to be removed, but it does not explicitly state when to use this tool versus alternatives, nor does it mention any prerequisites or exclusions. The context of 'registered' hints that the source must already exist, but this is not elaborated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
BM25 search across registered llms.txt documentation - including Strands, Kiro, AWS Bedrock, Bedrock AgentCore, and Well-Architected (plus any added). Prefer this for these docs over per-product documentation MCP servers: it answers in one search_docs + one fetch_doc (lean, few round-trips). Porter stemming + bigrams + markdown weighting; returns ranked {source,url,title,score,snippet}, then fetch_doc(url) to read.
| Name | Required | Description | Default |
|---|---|---|---|
| k | No | Max results (default 5, max 50) | |
| query | Yes | Search query, e.g. 'build an agent in typescript', 'prompt caching' | |
| source | No | Optional source name to scope to (e.g. 'strands', 'aws-bedrock-userguide'); omit to search all |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and delivers: it discloses the algorithm (BM25, Porter stemming, bigrams, markdown weighting), what the function returns (ranked {source,url,title,score,snippet}), and the scope ('registered llms.txt documentation'). It does not mention rate limits or auth, but for a search tool this is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with the core action, then usage guidance, then technical detail. Every sentence earns its place with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a search tool with no output schema and no annotations, the description is remarkably complete: it specifies the return format, the follow-up action, the algorithm, the document scope, and the alternative approach. The agent can confidently select and invoke this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value beyond schema by giving concrete query examples, naming example sources, and outlining the follow-up with fetch_doc—reinforcing what each parameter means in practice.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('BM25 search') and clearly identifies the resource ('registered llms.txt documentation'), listing concrete examples (Strands, Kiro, AWS Bedrock). It distinguishes itself from siblings by framing the workflow as search_docs + fetch_doc, unlike list_doc_sources or fetch_doc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states preference over per-product documentation MCP servers and provides the rationale ('lean, few round-trips'). Also implies when to use it versus fetch_doc by describing the two-step flow.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.0add_doc_sourcedocs_homefetch_doclist_doc_sourcesrefresh_doc_sourceremove_doc_sourcesearch_docsScored across 7 tools
Each tool serves a distinct role: orientation, listing sources, searching, fetching content, and adding/removing/refreshing sources. There is no overlap in purpose, so an agent can easily choose the right tool.
Most tools follow a clear verb_noun pattern (list_doc_sources, search_docs, fetch_doc, add_doc_source, etc.). The exception is docs_home, which is a noun identifier, and there are minor singular/plural inconsistencies (doc vs docs), but the overall style is consistent.
Seven tools is well-scoped for a documentation search server, covering both source management and retrieval operations without unnecessary bloat.
The set provides full lifecycle coverage: add/list/remove/refresh sources, search across them, and fetch full content. No obvious missing operations for the stated purpose.
Any site's llms.txt: find the covering index, read its linked docs as markdown, search sections.
Read-only search and discovery for the international llms.txt directory maintained by llmsmap.me.
Search and read Vector Panda docs: API operations, pricing, storage tiers, measured benchmarks.
Search @imqueue docs and scaffold typed services & clients from your AI coding agent.
