Skip to main content
Glama
philogeneperessini-ui

datamgr

by philogeneperessini-ui
OverviewSchemaRelated ServersScoreDiscussions
Python
Local

datamgr — dataset registry for empirical research

A lightweight registry that catalogs your research datasets so an AI (or you) can discover them without reading every file. Each dataset is one folder with a manifest.yaml; a flat index (_registry.json) makes search cheap no matter how many datasets you have.

Architecture: one copy of logic in datamgr.core; the dm CLI and the MCP server are both thin front-ends over it.

Install (editable)

cd data-manage
pip install -e .

Related MCP server: secoda-analysis-mcp

Quick start

# 1. point datamgr at a central warehouse (created if missing)
dm init-root D:/datasets

# 2a. import a raw data folder -- harvests variable names + labels from a .dta,
#     copies the file into a new clean-id folder under the warehouse
dm import "原始数据包" --id csmar_annual --name "CSMAR 上市公司年度财务"

# 2b. (or) scaffold an empty dataset and fill the manifest by hand
dm init csmar_annual --name "CSMAR 上市公司年度财务"

# 3. rebuild the index
dm refresh

# 4. discover
dm list                          # all datasets, one line each
dm show csmar_annual             # full manifest, rendered
dm search 资产负债率               # fuzzy-search across all variables/labels
dm variables --role control      # list all control variables available
dm variables --label 占比          # filter variables by label
dm stats csmar_annual --refresh  # compute rows/cols/n_firms, cache to manifest
dm verify                        # validate manifests + check file paths exist

# 5. pull data into an analysis working directory (backup + provenance)
dm export csmar_annual control_vars --to D:/my_paper
#   -> D:/my_paper/data/*.dta        (copied files, ready for `use "data/xxx.dta"`)
#   -> D:/my_paper/data_sources.txt  (where each file came from, for traceability)

Commands

command

purpose

dm init-root [PATH]

set / show the central warehouse location

dm init <id>

scaffold an empty dataset folder + manifest template

dm import [<folder>] [--id X]

build a manifest by harvesting variables from a .dta/.csv; with --id it copies data into a clean-id folder

dm refresh

rebuild the _registry.json index (reads manifests only, never data files)

dm list [--tag T]

list all datasets, one line each

dm show <id>

print a dataset's full manifest

dm search <query> [--tag T]

fuzzy-search datasets by variable name/label, name, description, tag

dm variables [--role R] [--label L]

list all variables across the index (for picking controls/instruments)

dm stats <id> [--refresh]

show cached stats, or recompute from the data file

dm export <id...> --to <dir>

copy dataset files into a working dir + record provenance

dm verify

validate manifests + check referenced files exist

The manifest (manifest.yaml)

This is the heart of the system. Fields:

field

purpose

id

dataset id, must equal the folder name

name

human-readable name

description

free text — searchable

version, source

provenance

unit_of_observation

granularity, e.g. 公司-年

time_span

[start, end] years

identifiers

{id, time} — fed straight to Stata xtset

variables

list; each has name/label/type/role

tags, dependencies

search facets & lineage

files

relpaths for raw/ pipeline/ processed

stats

cached summary (rows, cols, ...), computed at refresh

notes

any gotchas /口径变更

variables[].role ∈ {id, time, x, y, control, weight, instrument, other} is the key signal that lets an AI judge "can this dataset run the regression I want", which plain variable names can't convey.

Why it stays fast as datasets grow

  • Search reads the in-memory index, never the folder tree at query time.

  • The index stores only search-relevant fields (names, roles, tags, spans) — KB per dataset. A full manifest is loaded only by dm show / get_dataset.

  • Dataset files (.dta, etc.) are read only when you explicitly refresh stats. list/search/show never touch them.

Layout

src/datamgr/
  config.py        warehouse root resolution
  core/
    manifest.py    schema + load/validate/save
    registry.py    scan -> _registry.json index
    search.py      fuzzy search over the index (rapidfuzz)
    stats.py       compute rows/cols/n_firms from a .dta/.csv (explicit --refresh only)
    importing.py   harvest variables from a .dta/.csv -> manifest scaffold
    exporting.py   copy data files to a working dir + provenance txt
  cli/main.py      `dm` command
  mcp/server.py    read-only MCP server for AI discovery (stdio)

MCP server (AI discovery)

A read-only MCP server lets an AI client (e.g. zcode) discover and inspect datasets without touching data files. Exposes 5 tools over stdio:

tool

purpose

list_datasets(tag)

all datasets (id/name/vars/rows/span/tags)

search_datasets(query)

fuzzy search across variables/labels/names

list_variables(role, label)

flat variable list for picking controls

get_dataset(id)

full manifest incl. absolute file paths (locate the .dta on disk)

get_dataset_stats(id)

cached stats only (never reads data files)

Run it directly, or register with an MCP-aware client:

# direct
python -m datamgr.mcp.server
# or via entry point
datamgr-mcp

ZCode workspace registration (.zcode/config.json):

{
  "mcp": {
    "servers": {
      "datamgr": {
        "command": "python",
        "args": ["-m", "datamgr.mcp.server"],
        "env": { "DATAMGR_ROOT": "D:/datasets" }
      }
    }
  }
}

The server is deliberately read-only: importing data and recomputing stats are done from the dm CLI by a human, never by the AI.

Deployment & daily use

See DEPLOYMENT.md for:

  • deploying to another machine or MCP client (zcode / Claude / Cursor)

  • daily workflow when new data arrives (double-click .bat scripts, no CLI needed)

  • troubleshooting and a full migration checklist

This server cannot be installed

F
license - not found
-
quality - not tested
C
maintenance

How are these scores calculated?

Maintenance

–Maintainers
–Response time
–Release cycle
–Releases (12mo)
Commit activity

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Servers

  • eda-mcp

    MLMecham
    A
    license
    A
    quality
    D
    maintenance
    Enables exploratory data analysis through an MCP server, allowing AI assistants to load datasets, compute summary statistics, generate diagnostic plots, perform correlation analysis, and produce full markdown reports.
    Last updated
    6
    MIT

  • TimeVerse Code Reader

    elimyliu
    A
    license
    -
    quality
    B
    maintenance
    A read-only MCP server for code reading with intelligent caching, line-range selection, and language detection, enabling AI assistants to efficiently and safely explore file systems.
    Last updated
    MIT

View all related MCP servers

Related MCP Connectors

  • LLMemory

    Search your AI chat history (ChatGPT, Claude, Codex) from any MCP client. Remote, private, read-only

  • pAIchart MCP Hub

    MCP Hub: AI service discovery, per-user OAuth, and multi-service workflow orchestration

  • driflyte-mcp-server

    Driflyte MCP server which lets AI assistants query topic-specific knowledge from web and GitHub.

View all MCP Connectors

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/philogeneperessini-ui/data-manage'

If you have feedback or need assistance with the MCP directory API, please join our Discord server