datasniff

Is this dataset what I need, and am I allowed to use it?

datasniff searches Hugging Face, Kaggle and data.gov at once and tells you, for each dataset, what columns it has and their types, what licence it is under and what that licence lets you do, and how big it is in bytes and rows. Every value says exactly where it came from: which API, which field, which URL. When a value can't be found, datasniff says so and why, instead of guessing.

It only reads public metadata. It never downloads a whole dataset: when the only way to learn a file's columns is to look at it, it reads at most the first 64 KB.

There are two ways to use it, built on the same engine:

• On the command line: datasniff search "air quality".

• As an MCP server, so an assistant such as Claude Code, Claude Desktop or Cursor can check a dataset's schema and licence before suggesting it.

Install

datasniff needs Python 3.10 or newer. Pick one:

uv tool install datasniff     # with uv (https://docs.astral.sh/uv/)
pipx install datasniff        # with pipx
uvx datasniff search titanic  # run once without installing

This works the same on macOS, Windows and Linux. No account or API key is needed.

Related MCP server: dataset-search-mcp

Use it from the terminal

datasniff search "sentiment reviews"              # all three catalogs, 5 results each
datasniff search titanic --source kaggle -n 10    # one catalog (repeat --source for several)
datasniff details huggingface stanfordnlp/imdb    # schema, licence, size, files
datasniff details https://www.kaggle.com/datasets/heptapod/titanic
datasniff compare kaggle:heptapod/titanic huggingface:stanfordnlp/imdb

Sources are huggingface (or hf), kaggle and datagov (or data.gov). Ids are the ones search prints: owner/name on Hugging Face, owner/slug on Kaggle, the slug on data.gov. A dataset page URL works too.

Options: --json for machine-readable output, --no-cache to skip the cache for one run, --cursor SOURCE=CURSOR for the next page (search prints the exact flag).

Exit codes: 0 success, 1 the dataset was not found or every source failed, 2 bad usage.

Example

$ datasniff details hf stanfordnlp/imdb
IMDB  (huggingface: stanfordnlp/imdb)
https://huggingface.co/datasets/stanfordnlp/imdb

Licence
     other: Custom or unspecified terms; read the dataset's page.
     commercial use: unknown, attribution: unknown, share-alike: unknown
       from: cardData.license in https://huggingface.co/api/datasets/stanfordnlp/imdb?blobs=true
Size
     bytes: 79.6 MB (83,446,840 bytes)
       from: size.dataset.num_bytes_original_files in
       https://datasets-server.huggingface.co/size?dataset=stanfordnlp%2Fimdb
     rows: 100,000
       from: size.dataset.num_rows in
       https://datasets-server.huggingface.co/size?dataset=stanfordnlp%2Fimdb
Updated
     last modified: 2024-01-04T12:09:45.000Z
       from: lastModified in https://huggingface.co/api/datasets/stanfordnlp/imdb?blobs=true
Schema (1 table)
  plain_text: 2 columns; train 25,000, test 25,000, unsupervised 50,000
       from: dataset_info.plain_text.features in
       https://datasets-server.huggingface.co/info?dataset=stanfordnlp%2Fimdb
     text   string
     label  class_label[neg, pos]
Files (5)
     ...

In JSON, every value is an object with a source, and a missing one has a reason:

"size": {
  "bytes": {
    "value": 29981,
    "source": "totalBytes (sum of the dataset's files, uncompressed) in https://www.kaggle.com/api/v1/datasets/view/yasserh/housing-prices-dataset"
  },
  "rows": {
    "value": null,
    "reason": "Kaggle does not publish row counts in its API or Croissant metadata",
    "source": "https://www.kaggle.com/api/v1/datasets/view/yasserh/housing-prices-dataset; ..."
  }
}

Use it with an AI assistant (MCP)

datasniff mcp runs a Model Context Protocol server over stdio with three read-only tools:

Claude Code

claude mcp add --scope user datasniff -- uvx datasniff mcp

--scope user makes it available in every project; leave it out to add it to the current project only. Check it with claude mcp list.

Claude Desktop

Open Settings > Developer > Edit Config, and add datasniff to claude_desktop_config.json (on macOS ~/Library/Application Support/Claude/, on Windows %APPDATA%\Claude\):

{
  "mcpServers": {
    "datasniff": {
      "command": "uvx",
      "args": ["datasniff", "mcp"]
    }
  }
}

Restart Claude Desktop. If it can't find uvx, use the full path that which uvx (macOS) or where uvx (Windows) prints as the command.

Cursor

Add the same block to ~/.cursor/mcp.json (all projects) or .cursor/mcp.json in a project.

If you installed datasniff with uv tool install or pipx, you can use "command": "datasniff" with "args": ["mcp"] instead. To pass credentials (below), add an "env" block.

Where each value comes from

Types are reported in the vocabulary of their source (int64 and class_label[...] from Hugging Face, sc:Integer from Croissant, number or calendar_date from a Socrata data dictionary, integer / number / date / string when sniffed), so nothing is lost in translation.

data.gov has no documented public JSON API any more (the old CKAN API was retired, and the api.gsa.gov one needs a key). datasniff reads two undocumented endpoints of the current catalog site: catalog.data.gov/search?q=... with Accept: application/json, and catalog.data.gov/api/dataset/<slug>. They are isolated in one adapter, and if they change, data.gov reports an error while the other catalogs keep working.

data.gov ranks results by relevance score, and those scores can shift slightly between requests, so paging through results can occasionally repeat one from the previous page.

Licences

datasniff turns a licence string or URL into an SPDX id, but only when that is unambiguous (cc-by-4.0, CC BY-NC-SA 4.0, Attribution 4.0 International (CC BY 4.0) and https://creativecommons.org/licenses/by/4.0 all become CC-BY-4.0). Then it says, in one line, whether commercial use is allowed, whether you must credit the source, and whether adaptations must be shared under the same terms. Anything it doesn't recognise, including Hugging Face's other and most custom terms, is reported as unknown with the original text, never guessed. compare_licenses flags combinations such as two different share-alike licences.

These meanings come from a small built-in table and are a reading aid, not legal advice. Read the actual terms before you rely on them.

Credentials (optional)

Everything works anonymously. If you have them:

• HF_TOKEN: a Hugging Face token, for gated or private datasets you have access to.

• KAGGLE_USERNAME and KAGGLE_KEY, or ~/.kaggle/kaggle.json (Kaggle > Settings > API > Create New Token): used if Kaggle stops answering anonymous requests.

Tokens are sent only to their own service (never to another host, including after a redirect), and are never printed: error messages are scrubbed of anything that looks like a token, key or signed URL.

Being a polite client

• Only GET and HEAD requests, enforced in code; one module does all network access.

• A clear User-Agent: datasniff/<version> (+https://github.com/Srikarmk/datasniff).

• At most four requests at a time, timeouts on every request, and retries with exponential backoff (honouring Retry-After) on 429 and 5xx responses.

• Responses are cached on disk for 24 hours, in your platform's cache folder (~/Library/Caches/datasniff on macOS, %LOCALAPPDATA%\datasniff\Cache on Windows, ~/.cache/datasniff on Linux). Set DATASNIFF_CACHE_TTL (seconds), DATASNIFF_CACHE_DIR, or DATASNIFF_NO_CACHE=1 to change that.

• File sniffing asks for the first 64 KB with an HTTP Range request, and stops reading at 64 KB even when a server ignores the Range header and starts sending the whole file.

Privacy

datasniff collects nothing. It sends your search terms to the catalogs you search, and nothing anywhere else.

Licence

MIT. Copyright (c) 2026 Kameswara Sai Srikar Manda.