devbench
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@devbenchRun the tests for the current project and summarize failures"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
從 MCP 到 Multi-Agent:用一台本機 GPU 走完 30 天
iThome 鐵人賽 2026 · 全程零 API 成本,所有程式碼都在本機模型上真的跑過。
這個 repo 是整個系列的程式碼與實測數據。文章本身發表在 iThome, 不放在這裡(同一份內容放兩個地方會變成重複內容)。
outputs/ 底下那 11 份 .txt 是關鍵:文章裡每一段「執行結果」都不是手打的,
是用 scripts/run_day.sh 從真機抓回來、再原封不動貼上去的。
你可以拿它們跟自己機器上跑出來的數字對照。
這個系列在做什麼
30 天疊出一個叫 devbench 的東西:一個本機開發者工作台。 它從一個 MCP Server 開始,長成一群會分工的 Agent。
Day 1-5 Python 地基 型別 / async / 裝飾器 —— 全部是 MCP SDK 的語法基石
Day 6-10 MCP 協定 三大原語 / JSON-RPC / 傳輸層 / 寫出第一個 Server
Day 11-20 Agent 框架 安全治理 / ADK / LangGraph / ReAct / Plan-and-Execute
Day 21-30 Multi-Agent Supervisor / 記憶 / 評估 / OpenClaw / NVIDIA PAIR兩條伏筆
① 一個 endpoint 走到底。
從 Day 5 起,全專案只有 src/ironman/config.py 一個地方知道推論端點在哪。
Day 29 裝上 NVIDIA PAIR 之後,同一個 port 會變成整個區網的推論叢集,
而前面 28 天的程式碼一行都不用改。
tests/test_config.py 有一條測試在守這件事——誰把端點寫死,CI 就擋誰。
② Day 1-4 不是無關的 Python 教學。
Day | 語法主題 | 在 Day 10 的回收處 |
2 | 型別提示 / dataclass / Pydantic | tool 的 input schema 就是 |
3 | async / await / asyncio | MCP 協定全非同步 |
4 | 裝飾器 / context manager / generator |
|
Related MCP server: agentic-mcp-server
環境
這個系列跑在一台 NVIDIA GB10(DGX Spark 級) 上:
Ubuntu 24.04.4 LTS / aarch64 · 20 核 · 121 GB 統一記憶體
NVIDIA GB10, driver 580.142
Ollama 0.32.14但你不需要同款機器。除了 Day 28-29 的多節點實測之外, 任何跑得動 Ollama 的機器都能完整重現——把模型換小一點就行。
模型選型
不是拍腦袋決定的,是 Day 1 實測六個模型跑出來的結果
(完整數據見 outputs/day01.txt):
角色 | 模型 | 為什麼 |
主力 |
| 86.4 tok/s,而且是唯一答對「MCP 是什麼」的模型 |
小模型 |
| 59.7 tok/s,夠小才開得起高並行(Day 3 / 22 / 29) |
對照組 |
| 交叉驗證 tool schema 沒有綁死單一模型 |
開始
# 1. 裝 uv(user-level,不碰系統 Python)
curl -LsSf https://astral.sh/uv/install.sh | sh
# 2. 建環境
git clone git@github.com:jrlinjr/local-mcp-agent-30days.git
cd local-mcp-agent-30days
uv sync
# 3. 確認 Ollama 通了、模型都在
uv run ironman doctor
# 4. 隨便問一句
uv run ironman chat "用一句話說明 MCP" --stream跑每一天
uv run python days/day01_bench/main.py # 模型選型實測
uv run python days/day02_typing/main.py # dataclass vs Pydantic
uv run python days/day03_async/main.py # 並行度掃描
uv run python days/day04_decorators/main.py # 手刻 @tool 裝飾器
uv run python days/day05_first_llm/main.py # 第一支 LLM 呼叫
uv run python days/day07_primitives/main.py # MCP 三大原語
uv run python days/day08_jsonrpc/raw_client.py # 手刻 JSON-RPC(不用 SDK)
uv run python days/day08_jsonrpc/raw_client.py --sdk # 手刻 client 打官方 server
uv run python days/day09_transport/main.py # stdio vs Streamable HTTP
uv run python days/day10_mcp_server/client_demo.py # devbench MCP Server
uv run pytest -q # 全部測試把 devbench 掛進 Claude Code
repo 根目錄已經有一份 .mcp.json:
{
"mcpServers": {
"devbench": {
"command": "uv",
"args": ["run", "python", "days/day10_mcp_server/server.py"]
}
}
}在專案目錄開 Claude Code 就會自動載入,devbench 的四個工具直接可用。
遠端開發:本機編輯,遠端執行
如果你的 GPU 機器不是你打字的那台(我就是),
scripts/ 底下兩支腳本讓你不用在筆電裝任何東西:
cp .env.example .env.local # 填入你的遠端主機(.env.local 不進版控)
bash scripts/sync.sh # 本機 → 遠端(rsync,本機是唯一權威來源)
bash scripts/run_day.sh 03 # 在遠端跑 day03,輸出抓回 outputs/day03.txt主機名刻意不寫死在版控裡——它是內部基礎設施資訊,不該出現在公開 repo。
run_day.sh 抓回來的輸出會先過 scripts/scrub.py,把絕對路徑與主機名換成
<專案根目錄> 之類的中性字樣,因為那些輸出會被原封不動貼進文章。
把關機制
uv run pytest -q # 36 項端到端測試,不 mock
python3 scripts/verify_outputs.py # 文章數字 vs outputs/ 逐字比對verify_outputs.py 掃過每篇文章的程式碼區塊,把有數字的實測宣稱(tok/s、秒數、byte 數、pid、JSON-RPC 訊息框)挑出來,到 outputs/ 裡逐字比對,找不到就失敗。程式重跑之後數字會變,這條檢查會逼我把文章一起更新,而不是留著舊值。
文章不在這個 repo 裡,所以 clone 下來跑它會說「沒有文章可查核」,那是正常的。留著它是為了說清楚 outputs/ 裡那些數字是怎麼被把關的。
tests/test_config.py 裡有一條測試在守伏筆一:全專案只有 config.py 可以寫死推論端點,誰違反誰就掛。
scripts/scrub.py 在擷取輸出時自動抹掉絕對路徑與主機名——那些輸出會被原封不動貼進文章,不該帶著我的使用者名稱跑出去。
每日索引
文章發表於 iThome 鐵人賽 2026。這裡列出對應的程式碼與實測輸出。
Day | 主題 | 程式碼 | 實測輸出 |
1 | 開賽宣言:模型選型實測 | ||
2 | 型別提示、dataclass 與 Pydantic | ||
3 | async / await 與 asyncio | ||
4 | 裝飾器、Context Manager、Generator | ||
5 | 用 uv 建專案 + 呼叫第一支 LLM API | ||
6 | MCP 是什麼:Host / Client / Server | 觀念篇 | — |
7 | MCP 三大原語:Tools、Resources、Prompts | ||
8 | JSON-RPC 2.0 與 MCP 訊息生命週期 | ||
9 | 傳輸協定:stdio vs Streamable HTTP | ||
10 | 實作:用 Python SDK 寫一個 MCP Server |
目錄
src/ironman/ 共用地基
config.py ★ 唯一的設定來源(Day 29 的伏筆)
models.py Day 2 的資料模型,後面一路沿用
llm.py ★ 全系列唯一打 Ollama 的地方
cli.py uv run ironman ...
days/ 每一天的可執行範例
outputs/ 真實執行輸出(文章裡每個數字的來源)
tests/ 端到端測試,不 mock
scripts/ 遠端執行、輸出清洗、數字查核版本注意
MCP Python SDK 用的是 2.x。v1 的 from mcp.server.fastmcp import FastMCP
已改名為 from mcp.server.mcpserver import MCPServer,
且回應物件的欄位從 camelCase 改成 snake_case
(protocolVersion → protocol_version)。
網路上多數教學還停在 v1,照抄會直接爆——Day 10 有完整說明。
授權
程式碼採用 MIT License,隨你使用、修改、散布。
文章本身不在這個 repo 裡(發表於 iThome),著作權另計。 若你在自己的文章或簡報引用這裡的實測數據,附上出處連結就好。
Available Tools
4 toolsgit_logARead-onlyIdempotent
查看專案最近的 git commit 紀錄。
| Name | Required | Description | Default |
|---|---|---|---|
| count | No | 要看最近幾筆 commit |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is fully covered. The description adds the scoping constraint that it reads the project's recent commit history, which aligns with the annotations but does not disclose additional behavioral details such as sorting, output format, or side effects. Since the description does not contradict annotations and adds only scoping context, a 3 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that states the tool's purpose without wasted words. The key information (viewing recent commit history) is front-loaded, and there is no extraneous repetition of schema or annotation content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one optional, well-documented parameter, a read-only annotation profile, and an output schema, the description covers the essential purpose and scope. It does not include when-to-use guidance, which would enhance completeness, but the low complexity and rich schema make the missing guidance a minor gap rather than a critical one.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with the count parameter fully described as '要看最近幾筆 commit'. The tool description doesn't add information about parameters beyond what the schema provides, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('查看' / view) and a specific resource ('專案最近的 git commit 紀錄' / recent git commit history of the project). It clearly distinguishes git_log from sibling tools like list_files, run_tests, and read_file, which handle different concerns. An agent can tell this is the tool for commit history at a glance.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no explicit guidance on when to use this tool versus alternatives. There is no mention of 'use when you need to inspect commit history' or any exclusion/alternative statements. The context signals only list sibling tool names, but the description itself does not state any selection criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_filesARead-onlyIdempotent
依照 glob 樣式列出專案內的檔案。
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | 最多回傳幾筆 | |
| pattern | No | glob 樣式,例如 '**/*.py' 或 'src/**/*' | **/*.py |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds no additional behavioral context such as recursion behavior, sorting, or hidden files, but it also does not contradict the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that immediately communicates the tool's purpose. Every word contributes value, and the glob-style detail is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only listing tool with full schema coverage and an output schema, the description is adequate for an agent to select and invoke the tool. It could mention scope nuances like whether subdirectories are included by default, but the default glob pattern already implies recursive behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the parameters are already fully documented in the schema. The description only restates the glob concept already present in the schema, adding no extra semantic meaning beyond what the structured input already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific verb and resource: listing files in the project using a glob pattern. It is distinct enough from siblings like read_file or run_tests, though it does not explicitly differentiate itself from alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for enumerating project files via glob patterns, but it offers no explicit guidance on when to choose it over siblings or when not to use it. There are no exclusion criteria or alternative tool references.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
read_fileARead-onlyIdempotent
讀取專案內的一個檔案並回傳內容(附行號)。
| Name | Required | Description | Default |
|---|---|---|---|
| path | Yes | 相對於專案根目錄的檔案路徑,例如 src/ironman/llm.py | |
| max_lines | No | 最多讀幾行 |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (readOnlyHint=true, idempotentHint=true, destructiveHint=false), so the bar is lower. The description adds one useful behavioral detail: content is returned with line numbers, which clarifies the output format. It doesn't mention edge cases, but the read-only nature is already disclosed by annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-formed sentence that immediately states the action and expected result. It front-loads the core purpose and includes the line-number caveat with no fluff or redundant phrasing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only tool, the description, annotations, and full schema coverage collectively provide almost all necessary context. The output format is at least partially described (line numbers), and the schema covers parameters and constraints. A minor gap is the lack of any note on error behavior, but with annotations and output schema present, this does not undermine correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%; both path and max_lines already have rich documentation in the input schema (relative path example, limits, default). The description adds nothing about parameters beyond what the schema provides, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('讀取' / 'read'), a specific resource ('專案內的一個檔案' / 'a file in the project'), and the output ('回傳內容(附行號)' / returns content with line numbers). This makes the tool's purpose unmistakable and clearly distinguishes it from sibling tools like list_files, run_tests, and git_log.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It simply states what it does, with no mention of when to prefer read_file over list_files, run_tests, or git_log, nor any exclusions or conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
run_testsA
在專案內執行 pytest 並回傳結果摘要。
| Name | Required | Description | Default |
|---|---|---|---|
| target | No | 要測的路徑,預設整個 tests/ | tests |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations are all false, giving no indication of side effects. The description does not mention any potential effects (e.g., creating cache files, modifying the environment). No extra behavioral detail is provided.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that conveys the purpose without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple with one parameter and a clear purpose. The description mentions 'in the project' providing context. An output schema exists, so return values are not required to be described. Minor gap: no explicit statement about the working directory, but it is implied.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The parameter 'target' has a description in the schema, so the schema coverage is high. The tool description itself adds no additional parameter semantics beyond what the schema already provides, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (run pytest) and the result (return a summary). It distinguishes the tool from sibling tools like list_files or git_log.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool (when tests need to be run) but does not provide explicit guidance on alternatives or when not to use it. The parameter description gives some context but no comparative guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
4 tool updates
v0.1.0- First observed
git_log - First observed
list_files - First observed
read_file - First observed
run_tests
TDQS
Scored across 4 tools
Each tool performs a distinct operation: listing files, running tests, viewing git history, and reading file contents. There is no overlap or ambiguity between any pair of tools.
The verb_noun pattern is mostly consistent (list_files, run_tests, read_file), but git_log deviates slightly as it reads more like a noun compound than a verb phrase. Overall, naming is clear and predictable.
Four tools is a well-scoped set for a development inspection environment. Each tool serves a clear purpose without redundancy, and the count feels neither too thin nor bloated.
The tool set covers the core read-only workflows: enumerating, reading, testing, and version tracking. Missing write/edit capabilities could be a gap in some augmentation contexts, but for a benchmark or inspection tool, this surface is mostly sufficient.
Maintenance
Related MCP Connectors
AI-native git hosting — repos, PRs, issues, CI gates, and AI code review over MCP (60 tools).
MCP server for building and testing AI agents with multi-model experimentation and insights.
MCP server for AI agents to plan, verify, and deploy Cloudflare-native apps.
Self-hosted AI prompt library: prompts, collections, tags, teams, chains. 29 MCP tools for agents.
Related MCP Servers
- AlicenseCqualityDmaintenanceA privacy-first MCP server that provides local LLM-enhanced tools for code analysis, security scanning, and automated task execution using backends like Ollama and LM Studio. It enables symbol-aware code reviews and workspace exploration while ensuring that all code and analysis remain strictly on your local machine.36ISC
- FlicenseNot gradedqualityBmaintenanceEnables a fully local multi-agent AI system with 45 specialized agents and 50 predefined workflows via MCP, powered by Ollama and requiring no internet.1-
- AlicenseNot gradedqualityBmaintenanceSelf-hosted MCP server using Ollama for local AI-powered code analysis, refactoring, and optimization. Integrates with VS Code via Continue or Roo.MIT
- FlicenseNot gradedqualityBmaintenanceMCP server for AI DevTool workflow, exposing tools and resources for code review, repository chat, and repository operations.1-