Skip to main content
Glama
jcyeom

pubdata-mcp

by jcyeom
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

pubdata-mcp

한국 공공데이터를 수집·정규화해 DuckDB 데이터마트에 적재하고, 이를 MCP 서버로 노출한 뒤 Ollama 로컬 LLM 에이전트가 자연어 질문에 답하는 end-to-end 파이프라인입니다.

데이터 엔지니어링(수집·정규화·안전한 쿼리 서빙)과 AI 오케스트레이션(MCP + LLM 툴콜)을 하나의 프로젝트로 보여주는 백엔드 포트폴리오입니다.

아키텍처

data.go.kr ──ingest──> DuckDB 데이터마트 ──> MCP Server ──stdio──> Ollama Agent Host
 (실거래가/기상/대기질)   (정규화 테이블)      (툴/리소스)          (자연어 → 툴콜 → 답변)

  • ingest: 커넥터 인터페이스 + 3개 커넥터 (아파트 실거래가, 일별 기상, 일별 대기질). API 키 없으면 mock 폴백

  • store: DuckDB 적재 및 read-only 쿼리 (지역 partition 기반 idempotent 적재)

  • server: MCP 서버 (list_datasets, describe_schema, query_sql, price_stats, correlate + pubdata://catalog 리소스)

  • host: MCP 서버를 subprocess로 띄우고 Ollama 툴콜 루프로 질의에 답하는 에이전트

데이터셋

  • apt_trade — 아파트 매매 실거래가 (지역/월)

  • weather — 일별 기온·강수·풍속 (지역/일)

  • air_quality — 일별 PM10·PM2.5·오존 (지역/일)

weather와 air_quality는 (obs_date, region_code)를 공유해 correlate 툴로 크로스 도메인 상관분석이 가능합니다 (예: 풍속 vs 미세먼지 → corr ≈ -0.95, 바람이 강한 날 미세먼지가 낮음).

핵심 백엔드 포인트는 server/safe_sql.py의 read-only SQL 가드(단일문 강제, DDL/DML 차단, LIMIT 강제)와 DuckDB read-only 커넥션 이중 방어입니다.

Related MCP server: korean-public-data-mcp

빠른 시작

# 1. 의존성 설치 (uv)
uv sync

# 2. 데이터 적재 (키 없으면 mock 데이터로 동작)
for r in 11680 11650 11440; do
  uv run pubdata-ingest apt_trade   --region $r --ym 202406
  uv run pubdata-ingest weather     --region $r --ym 202406
  uv run pubdata-ingest air_quality --region $r --ym 202406
done

# 3. MCP 서버 단독 실행 (Claude Desktop / MCP Inspector에 연결 가능)
uv run pubdata-server

# 4. 에이전트로 자연어 질의 (CLI)
uv run pubdata-agent "강남구와 서초구 아파트 평균 거래가를 비교해줘"

# 5. 웹 UI (질문 + 답변 + 툴 호출 트레이스 표시) — http://localhost:8000
uv run pubdata-web

웹 UI / API

pubdata-web은 FastAPI 앱을 띄웁니다.

  • GET / — 질문 입력 → 답변과 함께 에이전트가 호출한 MCP 툴 트레이스(인자·결과·소요시간)를 표시

  • POST /api/ask {"question": "..."} → {answer, model, steps, elapsed_ms, trace[]}

  • GET /api/health — 상태 및 설정된 모델

Docker

Ollama는 호스트에서 실행 중이라고 가정합니다. compose는 host networking으로 호스트의 localhost:11434 Ollama에 바로 접근합니다 (Linux 기준).

docker compose up --build
# http://localhost:8000

첫 기동 시 마트가 비어 있으면 mock 데이터로 3개 지역을 자동 적재합니다. 호스트에 tool-calling 모델(ollama pull qwen2.5:7b)이 있어야 합니다. Docker Desktop(mac/Windows)에서는 docker-compose.yml 주석의 대안 설정을 참고하세요.

실제 API 사용

.env.example를 .env로 복사하고 PUBDATA_DATA_GO_KR_KEY에 data.go.kr 서비스 키를 넣으면 mock 대신 실데이터를 수집합니다.

Claude Desktop 연결 예시

{
  "mcpServers": {
    "pubdata": {
      "command": "uv",
      "args": ["run", "pubdata-server"],
      "cwd": "/home/jcyeom/dlwlr/pubdata-mcp"
    }
  }
}

관측성 (Observability)

모든 에이전트 실행은 data/traces.jsonl에 append됩니다 (질문·모델·툴 궤적·소요시간·에러·답변).

  • GET /api/traces?limit=20 — 최근 실행 조회 (최신순)

  • best-effort 로깅이라 실패해도 요청에 영향 없음

평가 (Eval)

골든 케이스로 에이전트 품질을 회귀 검증합니다.

uv run pubdata-eval

각 케이스는 3가지를 채점합니다:

  • answer-correctness: 답변에 기대값(실데이터 숫자)이 포함되는가 (콤마 무시 비교)

  • tool-trajectory: 기대한 MCP 툴이 호출됐는가

  • error-free: 툴 에러가 없었는가

결과는 표로 출력되고 data/eval_report.json에 저장됩니다. 하나라도 실패하면 exit code 1 (CI 연동 가능). 로컬 LLM 재현성을 위해 에이전트는 temperature=0으로 호출합니다.

테스트

uv run pytest   # 23 tests: SQL 가드, 파이프라인, 관측성, eval 스코어링

로드맵

  • end-to-end 파이프라인 (ingest → DuckDB → MCP → Ollama 에이전트)

  • read-only SQL 가드 + idempotent 적재

  • Docker 배포 + 웹 UI (툴 트레이스 표시)

  • 툴콜 정확도 eval 셋 + 요청 트레이싱/관측성

  • 기상/대기질 커넥터 추가 → 크로스 도메인 상관분석 (correlate 툴)

  • 기상/대기질 실 API 연동 (현재 mock, apt_trade는 실 API 지원)

  • 수집 스케줄러(APScheduler) + 증분 적재

Available Tools

5 tools
correlateA

Pearson correlation between two numeric columns across two datasets.

Joins the tables on their shared keys (obs_date and/or region_code) and returns corr(col_a, col_b) plus the joined sample size. Example: correlate weather.avg_wind_ms with air_quality.pm10 to see whether windy days have cleaner air. Table and column names are validated against the live schema before use.

ParametersJSON Schema
NameRequiredDescriptionDefault
col_aYes
col_bYes
table_aYes
table_bYes

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description discloses the join on shared keys, return of correlation and sample size, and schema validation. However, it does not detail side effects, permission requirements, or error handling, which is slightly incomplete.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise (4 sentences), front-loads the core purpose, and includes an illustrative example. Every sentence adds value without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (cross-table join correlation), the description covers essential aspects: statistical method, join keys, return values, and validation. It lacks details on handling missing data or edge cases, but overall is sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, but the description explains the overall function and mentions shared keys and numeric columns, yet does not individually describe each parameter or their constraints beyond the implicit requirement that col_a/col_b are numeric and exist in their respective tables.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool computes Pearson correlation between two numeric columns across two datasets, with specific verbs and resources. It distinguishes from sibling tools like query_sql or price_stats by its focused analytical purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides an illustrative example (correlating wind speed with PM10) that helps understand when to use the tool, but does not explicitly state when not to use it or compare to alternatives like query_sql.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

describe_schemaB

Return column names and types for a table so the agent can write SQL.

ParametersJSON Schema
NameRequiredDescriptionDefault
tableYes

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description alone must disclose behavioral traits. It states the return value but does not mention that the operation is read-only, safe, or free of side effects. The minimal description lacks transparency about non-obvious behaviors like required privileges or potential errors.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no filler. Every word contributes to the purpose: 'Return column names and types for a table so the agent can write SQL.' This is the ideal length for a simple tool with one parameter.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (one parameter, no output schema), the description is mostly sufficient. However, it omits details about the return format (e.g., list of objects with 'name' and 'type') and does not mention that the table must exist. The description is adequate but not thorough.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 0% description coverage, and the tool description does not explain the 'table' parameter (e.g., expected format, case sensitivity, whether full path is needed). The description adds no meaning beyond the schema, failing to compensate for the lack of parameter documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool returns column names and types for a table, with the explicit purpose of enabling SQL writing. It distinguishes itself from siblings like list_datasets (listing tables) and query_sql (executing queries) by focusing on schema introspection.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for writing SQL but does not explicitly state when to use versus siblings (e.g., 'Use before writing queries' or 'Do not use for listing tables'). No exclusions or alternatives are given, leaving the agent to infer context from sibling tool names.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_datasetsA

List datasets available in the mart with their title and description.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description correctly indicates it is a read-only listing operation. The output schema provides further clarity, though it does not explicitly state side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that conveys all necessary information without extraneous content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no parameters, an output schema, and clear sibling context, the description provides complete information for an agent to use the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are no parameters, so the schema coverage is 100%, meeting the baseline. The description does not need to add parameter details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it lists datasets with their title and description, distinguishing it from sibling tools that focus on schema description, querying, or statistics.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The tool's purpose is obvious for listing available datasets, and given the sibling tools are for different operations, usage guidance is clear without explicit exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

price_statsA

Average / min / max / count of price_manwon grouped by a column.

A convenience analytics tool so the agent needn't hand-write aggregation SQL.

ParametersJSON Schema
NameRequiredDescriptionDefault
tableNoapt_trade
group_byNosigungu

TDQS

A3.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description bears the full burden of behavioral disclosure. It only states the tool computes statistics, without mentioning whether it is read-only, requires permissions, or any side effects. For a simple aggregation tool, the lack of explicit safety cues is a minor gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no redundancy. The key function (aggregates of price_manwon) is front-loaded, and the convenience aspect is stated efficiently.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (2 params, no output schema, no annotations), the description covers the core function and rationale. However, it omits the return format, which could be helpful for an agent deciding how to use results. Still, it is mostly complete for the intended use case.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 0% description coverage, and the description only indirectly explains parameters by mentioning 'grouped by a column' and the default column 'sigungu'. It does not describe the 'table' parameter or valid values, leaving the agent to infer details. The description adds some meaning but is insufficient for full understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool computes average, min, max, and count of price_manwon grouped by a column. It distinguishes itself from siblings like query_sql by being a specialized convenience tool, avoiding hand-written SQL.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies use for simple aggregation tasks on price_manwon, suggesting it saves effort compared to writing SQL. However, it does not explicitly state when not to use it or mention alternatives like query_sql for more complex queries.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

query_sqlA

Run a read-only SELECT against the mart.

Only single SELECT / WITH statements are allowed; results are row-capped. Use describe_schema to learn column names before querying.

ParametersJSON Schema
NameRequiredDescriptionDefault
sqlYes

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses read-only behavior, statement type restrictions, and row capping. While it could mention error handling or auth needs, the current disclosure is above average for a simple SQL tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the main purpose. Every sentence adds value without redundancy. Highly efficient and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with one parameter and no output schema, the description covers input constraints, usage context, and a prerequisite sibling. It hints at the output (tabular result) but could specify the row cap value or clarify pagination. Still, it's largely complete for typical use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter 'sql' has 0% schematic description coverage, so the description must compensate. It clarifies allowed SQL constructs (SELECT/WITH) and implies a row limit, but lacks syntax examples or format details. This is adequate but not exceptional.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool runs a 'read-only SELECT' against the 'mart,' specifying the verb and resource. It distinguishes itself from sibling tools like 'describe_schema' and 'list_datasets' by focusing on querying rather than metadata or analysis.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit when-to-use guidance ('Run a read-only SELECT'), what is not allowed ('Only single SELECT / WITH statements'), and a key alternative ('Use describe_schema to learn column names before querying'). This helps the agent choose correctly.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 5 tool updatesv0.1.0
    • First observedcorrelate
    • First observeddescribe_schema
    • First observedlist_datasets
    • First observedprice_stats
    • First observedquery_sql
Deploy Server
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A4/5.0

Scored across 5 tools

Disambiguation5/5

Each tool has a clear, distinct purpose: listing datasets, describing schemas, running SQL queries, computing price statistics, and calculating correlations. No overlap or ambiguity.

Naming Consistency5/5

All tools use consistent snake_case naming with verb-first patterns (list_datasets, describe_schema, query_sql, price_stats, correlate). The single verb 'correlate' fits the pattern.

Tool Count5/5

Five tools cover the core functionality of data discovery, schema exploration, custom queries, and common analytics. This is a well-scoped set for a data mart server.

Completeness4/5

The tool set covers essential workflows: discover (list_datasets), understand (describe_schema), query (query_sql), and two analytics helpers. Minor gaps like missing aggregation functions, but overall complete for the intended domain.

Maintenance

ActivityStale
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • power-plan-db

    dillettante
    A
    license
    Not graded
    quality
    C
    maintenance
    MCP server for Korean government power/energy statutory plans. Enables AI agents to search and retrieve public power plan documents.
    MIT

  • korean-public-data-mcp

    SongT-50
    A
    license
    Not graded
    quality
    D
    maintenance
    Enables AI to query real-time Korean public data including weather, real estate prices, air quality, economic indicators, and business registration via natural language.
    1
    MIT

  • vivory-mcp

    jayjodev
    F
    license
    Not graded
    quality
    D
    maintenance
    Korean public-data MCP servers for AI agents, enabling natural language queries to KOSIS statistics and other Korean official data sources without requiring local accounts or API keys.
    -

  • seoul-air-quality-mcp

    hlucent
    F
    license
    Not graded
    quality
    B
    maintenance
    MCP server for querying Seoul's official air quality data, including real-time and hourly measurements by district. Enables AI assistants to answer questions about fine dust and air pollution using public data from Seoul Open Data Plaza.
    -