solid-knowledge-ai
Solid Knowledge AI
一个由自反思 LangGraph 智能体驱动的多源文档知识助手。它将 PDF + Markdown + 网页 摄取到同一个向量库中,然后通过一个对自身检索进行评分并验证自身答案是否有据可依的智能体来回答问题——当任一检查失败时,使用重写的查询重试;当无法为答案提供依据时,拒绝编造。使用 Langfuse 进行追踪,使用 DeepEval 进行质量测试,并通过 MCP 暴露。
旨在展示智能体开发:LangGraph · LiteLLM · ChromaDB · MCP · Langfuse · DeepEval。
为什么这不是"单纯的 RAG"
该智能体是一个纠正性 / 自反思 RAG 循环,而不是线性的 retrieve → generate 链:
question
│
▼
route ──chitchat/out_of_scope──▶ generate ──▶ END
│ kb
▼
retrieve ──▶ grade_docs ──irrelevant (rewrite query, retry)──▶ retrieve
│ relevant
▼
generate ──▶ self_check ──ungrounded (retry)──▶ retrieve
│ grounded / budget spent
▼
answer + citations (or an honest "I don't know")route — 对闲聊 / 超出范围的问题跳过检索。
grade_docs — 一个 LLM 相关性门控;失败时重写查询并重试。
self_check — 验证草拟的答案是否由检索到的上下文所蕴含;如果不是,则重试或给出保守回答,而不是产生幻觉。
共享的重试预算(
max_retries,默认 2)约束两个循环。Memory — 一个 SQLite 检查点器按
thread_id维护多轮对话状态。
Related MCP server: PDF MCP Server
快速开始
# 1. Install (Python 3.11+, uv)
uv sync
# 2. Configure — only ANTHROPIC_API_KEY is required
cp .env.example .env # then edit .env
# 3. Ingest the sample corpus (2 Markdown + 1 PDF + 1 Wikipedia article)
uv run skai ingest # -> builds ./.chroma (local MiniLM embeddings, no API)
# 4. Ask (defaults to Haiku 4.5; switch per-call with --model)
uv run skai ask "What do orcas eat?"
uv run skai ask "How do orcas communicate?" --source md
uv run skai ask "Summarize orca threats" --model sonnet # haiku | sonnet (Opus blocked)
# 5. Multi-turn chat (remembers the conversation)
uv run skai chat
# 6. Web UI (chat + feedback + live data ingestion)
uv run skai ui # http://localhost:7860
# 7. Serve over MCP (stdio) for Claude Desktop / an IDE
uv run skai mcpWeb UI
skai ui 启动一个 Gradio 应用,具备实时演示所需的功能:
聊天,带每会话记忆;每个答案都显示其来源、路由和模型。
每次响应后的反馈 — 👍/👎 + 可选评论,存储到 SQLite(
.skai/feedback.sqlite)并且在开启追踪时作为该轮对话 trace 上的 Langfuse 评分推送。这就是闭环:真实使用成为评估信号。示例提示,引导首次交互。
实时扩展知识库 — 上传
.md/.txt/.pdf或粘贴 URL,即刻被摄取到 Chroma 中,因此演示不限于种子语料。模型(haiku/sonnet)和来源过滤器(all/pdf/md/web)选择器。
反馈可通过 skai.feedback.export_jsonl 导出为 JSONL 评估种子。
命令
命令 | 作用 |
| 加载 → 分块 → 嵌入 → 持久化到 Chroma |
| 带引用的单次提问 |
| 带记忆的交互式多轮聊天 |
| Gradio Web UI:聊天、反馈、实时摄取 |
| 运行暴露 |
| 运行 DeepEval 质量套件(需要 |
MCP 客户端配置
服务器暴露两个工具 — search_kb(query, source_type?)(原始检索)和 ask(question)(完整智能体)。将 MCP 客户端指向它:
{
"mcpServers": {
"solid-knowledge-ai": {
"command": "uv",
"args": ["run", "skai", "mcp"],
"cwd": "/absolute/path/to/solid-knowledge-ai"
}
}
}模型选择
默认是 Haiku 4.5(快速、便宜 — 适合 Q&A 路由+评分+生成循环)。通过 --model 按调用切换,或通过 .env 中的 SKAI_MODEL 全局切换:
值 | 解析为 |
|
|
|
|
任意 LiteLLM id | 透传(例如 |
Opus 被有意阻止(resolve_model 会抛出异常),因此助手不会意外指向最昂贵的层级。
可观测性
在 .env 中设置 LANGFUSE_PUBLIC_KEY / LANGFUSE_SECRET_KEY(以及可选的 LANGFUSE_HOST)。每次图运行随后产生一个 trace,每个节点和每次 LLM 调用各有一个 span。没有密钥时,追踪是一个干净的空操作 — 其他一切不变。
质量评估(DeepEval)
uv sync --group eval
export ANTHROPIC_API_KEY=...
uv run skai ingest
uv run --group eval pytest evals -v # or: skai eval评判者是 通过 LiteLLM 的 Claude,因此不需要 OpenAI 密钥。指标:faithfulness、answer relevancy、contextual relevancy — 外加一个廉价的关键词门控。
测试
uv run pytest # 39 tests, fully offline: no network, no API keysLLM 是依赖注入的,因此整个图在测试中针对一个确定性的桩运行,Chroma 使用确定性的进程内嵌入函数。
结构组成
src/skai/
config.py settings (.env) agent/llm.py ChatLiteLLM -> Claude
models.py Document/Chunk/AgentState agent/nodes.py route/retrieve/grade/generate/self_check
ingest/loaders.py pdf | md | web -> Document agent/prompts.py node prompts
ingest/chunk.py source-aware splitting agent/graph.py StateGraph + SQLite memory
ingest/store.py Chroma add/query observability.py Langfuse handler (or no-op)
cli.py ingest|ask|chat|mcp|eval mcp_server.py search_kb / ask as MCP tools
evals/ DeepEval suite tests/ offline unit + graph tests智能体图与组件图(Mermaid): 参见 docs/ARCHITECTURE.md。
设计理由与技术权衡: 参见 docs/DECISIONS.md。
状态
已验证:uv run skai ingest 加载所有三种来源类型(2 个 md + 1 个 pdf + 1 个 web → 170 个块),真实的语义检索返回相关段落。39 个离线测试通过。ask/chat/eval 需要 ANTHROPIC_API_KEY。
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables intelligent search and question-answering over PDF documents using semantic similarity and keyword search. Supports OCR for scanned PDFs, persistent vector storage with ChromaDB, and maintains source tracking with page numbers.5MIT
- FlicenseNot gradedqualityDmaintenanceEnables AI-powered querying of PDF documents using hybrid retrieval (BM25 + vector search) and retrieval-augmented generation, returning structured answers with source citations and confidence scores.
- AlicenseAqualityAmaintenanceEnables AI agents to read and analyze PDF documents for natural language Q\&A. Supports multiple LLM providers including Google Gemini, Anthropic Claude, and OpenAI.1244Apache 2.0
- AlicenseNot gradedqualityCmaintenanceConvert PDF documents to Markdown and query them using AI with source attribution and confidence scoring, supporting multiple LLM providers.MIT
Related MCP Connectors
Governed, auditable knowledge your team curates for its AI assistants, self-hostable
Persistent memory and knowledge management for AI agents with semantic search and 50+ tools.
Your company's brain for AI agents. Cited, permission-aware knowledge across every system.
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/hailampy123/solid-knowledge-ai'
If you have feedback or need assistance with the MCP directory API, please join our Discord server