GET /v1/datasets (public) — List Datasets — List all datasets with pagination.

Returns a lightweight summary of each dataset. Use the individual dataset endpoints to get full details including schema_info and column_metadata.

GET /v1/search (public) — Search Datasets — Search across all datasets using full-text search.

Searches dataset names, descriptions, provider names, and column names. Results are ranked by relevance using PostgreSQL's FTS capabilities.

Search Modes:

• keyword: Traditional FTS with tsvector matching

• semantic: Embedding-based similarity search (conceptual matching)

• hybrid: Combines both using Reciprocal Rank Fusion (RRF)

Query Syntax:

• inflation - simple term search

• "consumer price index" - exact phrase search

• census -historical - exclude term

• inflation OR unemployment - alternative terms

Sort Options:

• relevance: FTS ranking (default when query provided)

• recency: Most recently updated first

• name: Alphabetical by dataset name

• `populari…

GET /v1/search/suggest (public) — Suggest Datasets — Get autocomplete suggestions for search typeahead.

Returns dataset names that start with the given prefix, for use in search input autocomplete. Only returns datasets with status='ready'.

Response:

• suggestions: List of matching datasets with name, slug, provider, and path

• query: The prefix echoed back

Example: GET /v1/search/suggest?q=con might return suggestions like "Consumer Price Index", "Congressional Budget Data", etc.

GET /v1/datasets/{provider}/{dataset}/related (public) — Get Related Datasets — Get related datasets using unified multi-signal scoring.

Combines pgvector semantic similarity, join suggestions, and Neo4j graph signals (shared topics, community membership, structural similarity) into a single ranked list. Results are cached in Redis for fast repeat access.

Categories:

• joinable: Datasets that can be joined on shared columns

• similar: Semantically related datasets

Gracefully degrades when Neo4j is unavailable (pgvector + join signals only).

GET /v1/datasets/{provider}/{dataset}/activity (public) — Get Dataset Activity — Get recent activity events for a dataset.

Returns system events (enrichment, ingestion, schema changes) in reverse chronological order. Used for the activity feed on dataset pages.

GET /v1/categories (public) — List all categories — List all data categories with dataset counts.

Returns categories that have at least one dataset, ordered by dataset count (most popular first).

Categories are derived from dataset metadata labels set during LLM enrichment.

No authentication required - this is a public endpoint.

GET /v1/categories/{slug} (public) — Get category with datasets — Get category details with paginated datasets.

Returns the category metadata along with a paginated list of datasets in that category.

Sort options:

• stars: Most starred datasets first (default)

• queries: Most queried datasets first (requires backend, falls back to stars)

• updated: Most recently updated first

• name: Alphabetical by name

No authentication required - this is a public endpoint.

GET /v1/discover (public) — Discover Datasets — Discover datasets optimized for LLM agents.

Returns enriched dataset metadata including column schemas, canonical questions, methodology summaries, and quality scores. Requires authentication.

Key differences from /search:

• Always uses hybrid search with relevance sorting

• Includes enriched metadata (columns, canonical questions, methodology)

• Smaller default limit (5) for focused results

• Requires authentication (designed for API/agent access)

• No pagination (use /search for browsing)

GET /v1/datasets/{provider}/{dataset}/joinable (public) — List previewable joinable datasets — Return join targets for the compose/preview UI.

Filters out low-signal joins (date-only at low confidence, cross-community). Falls back to a small number of low-signal matches when the filter would produce an empty list so the UI still has something to render.

Public endpoint — matches the auth stance of other dataset-detail routes. Returns an empty target list (not 503) when Neo4j is unavailable.

POST /v1/datasets/{provider}/{dataset}/compose/preview (auth: Bearer OPENDATA_API_KEY) — Preview a compose/join without saving it — Execute a single-join preview against provider/dataset.

Kept narrow: exactly one join spec per request. Multi-join composition would compound cardinality risk during preview — we want users to vet each join in isolation before they chain them.

Preflight: when the base dataset has more than CARDINALITY_PREFLIGHT_BASE_ROW_THRESHOLD rows, we run a COUNT(*) on the joined query first. If the count exceeds CARDINALITY_PREFLIGHT_HARD_LIMIT we refuse with 400. If it's above the dynamic warning threshold we still run the preview but emit a cardinality_warning and cap rows at CARDINALITY_PREFLIGHT_ROW_CAP.

GET /v1/datasets/{provider}/{dataset}/compose/download.csv (public) — Download a composed join as CSV — Stream a composed CSV of provider/dataset joined with target.

Kept narrow for v1: single join, single join column, no computed columns, no filters. Mirrors the shape of POST /compose/preview so users can preview then download without tweaking parameters.

GET /v1/datasets/{provider} (public) — List Provider Datasets — List all datasets for a provider.

GET /v1/datasets/{provider}/{dataset}/meta (public) — Get Dataset Meta — Get dataset metadata without data.

Returns metadata about the dataset including schema, available views, and query capabilities. Use this endpoint to understand dataset structure before fetching data.

GET /v1/datasets/{provider}/{dataset}/sources (public) — Get Dataset Sources — List all source URLs for a dataset.

For multi-source datasets, returns each source with its description. For aggregator datasets, returns the URL template and variable info. For single-source datasets, returns the single source URL.

GET /v1/datasets/{provider}/{dataset}/{subdataset}/meta (public) — Get Subdataset Meta — Get subdataset metadata without data.

Returns metadata about the subdataset including schema and available views.

GET /v1/datasets/{provider}/{dataset}/columns (public) — Get Dataset Columns — Get column metadata and statistics for a dataset.

Returns column names, types, distinct counts, value distributions, and row count. Uses sampling for efficient stats on large datasets.

When a view is active (explicitly or via default_view), returns columns as they appear in the view output, including computed and joined columns with proper types.

GET /v1/datasets/{provider}/{dataset}/columns/{column_name} (public) — Get Dataset Column Detail — Get detailed statistics for a single column.

Returns full value list (up to 1000) for low-cardinality columns.

GET /v1/datasets/{provider}/{dataset}/views (public) — List Dataset Views — List available views for a specific dataset.

Returns all views applicable to this dataset, including:

• Inline views defined in dataset.yaml (spec.views field)

• Global predefined views that match the dataset schema

Dataset views take priority over predefined views with the same name.

GET /v1/datasets/{provider}/{dataset}/views/{name} (public) — Get Dataset View — Get detailed configuration for a specific view on a dataset.

Includes all alias definitions, column projections, and lookup configurations for data enrichment.

GET /v1/datasets/{provider}/{dataset} (public) — Get Dataset By Path — Get dataset data (flat) or subdataset list (hierarchical).

For flat datasets: Returns paginated data with full query support. For hierarchical datasets: Returns list of subdatasets.

Metadata is available at GET /v1/datasets/{provider}/{dataset}/meta

Format Selection (in priority order):

1. Accept header: application/json, text/csv, text/tab-separated-values, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet, application/vnd.apache.parquet

2. ?format query param: json, csv, tsv, xlsx, parquet

csv/tsv/xlsx/parquet always return file attachments with the full dataset (size-guarded at MAX_DOWNLOAD_SIZE_BYTES). ?format=json also returns the full dataset as a JSON file; without ?format=json, JSON …

GET /v1/datasets/{provider}/{dataset}/export (public) — Export Dataset As Yaml — Export dataset configuration as YAML.

Returns the dataset configuration in dataset.yaml format, suitable for re-importing via the sync command or POST /datasets endpoint.

The exported YAML can be used to:

• Create a YAML-based dataset configuration

• Back up dataset configuration

• Share dataset definitions across environments

• Migrate from API-created to YAML-sourced datasets

Note: Only exports configuration, not the actual data.

GET /v1/datasets/{provider}/{dataset}/{subdataset} (public) — Get Subdataset By Path — Get subdataset with inline data.

Returns subdataset metadata with paginated data rows.

Pagination:

• Use cursor for keyset pagination (efficient for large datasets with views)

• The response includes next_cursor if more data is available

• Pass the next_cursor value as cursor in the next request

GET /v1/directories (public) — List Directory — List and analyze a web directory.

Fetches the directory listing from the provided URL, parses the content to extract file and subdirectory entries, and analyzes the contents to suggest appropriate next actions.

The response includes a suggested_action field:

• "discover": Directory contains data files ready to be added as datasets

• "browse": Directory contains subdirectories to explore further

• "mixed": Directory contains both data files and subdirectories

Results are cached for 5 minutes to reduce load on remote servers. Use refresh=true to bypass the cache and fetch fresh data.

Args: request: FastAPI request for getting client IP url: URL of the directory to list (must be a valid HTTP/HTTPS URL) refresh: If true, by…

GET /v1/providers (public) — List Providers — List all data providers with dataset stats.

GET /v1/providers/{provider} (public) — Get Provider — Get provider details by slug.

GET /v1/views (public) — List Views — List all available predefined views.

Returns a summary of each view including name, description, available aliases, and default sorting.

GET /v1/views/{name} (public) — Get View — Get detailed configuration for a specific view.

Includes all alias definitions, column projections, and lookup configurations for data enrichment.

GET /v1/users/{user_id}/activity (public) — Get User Activity — Get paginated activity feed for a user.

Returns a list of activities (dataset creation, starring, syncing, etc.) ordered by most recent first.

GET /v1/users/@{username}/views (public) — List a user's published views (portfolio) — List published views for a profile portfolio.

include_drafts=true is only honored when the caller is the owner; other callers see published views only. Username casing is canonicalized via a 301 if necessary.

PUT /v1/users/{clerk_id}/follow (auth: Bearer OPENDATA_API_KEY) — Follow a user — Follow clerk_id. Idempotent — repeat calls return current count.

DELETE /v1/users/{clerk_id}/follow (auth: Bearer OPENDATA_API_KEY) — Unfollow a user — Unfollow clerk_id. Idempotent — DELETE on a missing edge returns OK.

GET /v1/users/{clerk_id}/follow-status (public) — Check follow status — Return whether the caller is following clerk_id.

GET /v1/users/{clerk_id}/followers (public) — List a user's followers — List users following clerk_id (newest first).

GET /v1/users/{clerk_id}/following (public) — List who a user is following — List users that clerk_id is following (newest first).

GET /v1/users/me/feed (public) — Activity feed of followed users — Activity from users the caller follows.

GET /v1/users/{clerk_id}/social-activity (public) — Public social activity timeline for a user — Public timeline for one user.

GET /v1/users (public) — Look up user by username — Look up a user by username.

When username is provided, returns the matching user's public profile (same shape as GET /v1/users/{user_id}).

This endpoint is public and does not require authentication.

GET /v1/users/{user_id} (public) — Get user public profile — Get a user's public profile.

Returns the user's public information including dataset count, starred count, and public collection count.

This endpoint is public and does not require authentication.

GET /v1/users/{user_id}/datasets (public) — List user's datasets — List datasets owned by a user.

Returns a paginated list of datasets that the user has created, ordered by creation date (most recent first).

Returns empty list for users who haven't created any datasets yet. This endpoint is public and does not require authentication.

GET /v1/users/{user_id}/stars/datasets (public) — List user's starred datasets — List datasets starred by a user.

Returns a paginated list of datasets the user has starred, ordered by when they were starred (most recent first).

Returns empty list for users who haven't starred any datasets yet. This endpoint is public and does not require authentication.

GET /v1/users/{user_id}/collections (public) — List user's public collections — List public collections created by a user.

Returns a paginated list of the user's public collections, ordered by last update time (most recent first).

Returns empty list for users who haven't created any collections yet. Private collections are not included. This endpoint is public and does not require authentication.

GET /v1/users/{user_id}/views (public) — List user's public views — List public views created by a user.

Returns a paginated list of the user's public views, ordered by last update time (most recent first).

Returns empty list for users who haven't created any views yet. Private views are not included. This endpoint is public and does not require authentication.

GET /v1/users/{user_id}/figures (public) — List user's public figures — List public figures created by a user.

Figures are views that have a viz_spec set (chart/visualization configuration). Results are ordered by like_count descending (most liked first).

Returns empty list for users who haven't created any figures yet. This endpoint is public and does not require authentication.

GET /v1/users/{user_id}/likes/figures (public) — List figures liked by a user — List figures liked by a user.

Returns public figures the user has liked, ordered by when they liked them (most recent first). liked_at is included in each item.

Returns empty list for users who haven't liked any figures yet. This endpoint is public and does not require authentication.

GET /v1/users/{user_id}/views/check-slug (public) — Check whether a slug is available for the current user — Slug-availability probe used by the sandbox save dialog.

Auth + ownership: the URL clerk_id must match the caller. Returns {available, normalized_slug, reason} where reason is one of "taken" | "reserved" | "invalid" | null.

GET /v1/users/{user_id}/stars/views (public) — List a user's starred views (figures they have liked) — Published views the user has liked, ordered by most-recent like first.

Used by the profile "Starred" tab in phase 4. Mirrors GET /v1/users/{user_id}/stars/datasets.

GET /v1/changelog (public) — List changelog entries — List recent platform announcements.

Returns a paginated list of changelog entries, ordered by published_at descending (most recent first).

This is a public endpoint and does not require authentication.

GET /v1/providers/{provider}/stats (public) — Get provider statistics — Get aggregate statistics for a provider including dataset count, total rows, and update info.

GET /v1/providers/{provider}/enriched (public) — Get enhanced provider details — Get provider details including AI-enriched metadata when available.

GET /v1/providers/{provider}/datasets/metadata (public) — Get enriched metadata for all datasets — Get AI-enriched metadata (layman descriptions) for all datasets in a provider.

GET /v1/providers/{provider}/related (public) — Get related providers — Find providers related to this one based on shared data domains.

GET /v1/stats (public) — Get platform statistics — Get platform-wide aggregate statistics including total providers, datasets, and rows.

GET /v1/me/activation (public) — Get user activation signals — Get user activation signals for SSR branching.

Returns flags indicating whether the user has engaged with various parts of the platform. Used by SSR to decide which components to render.

Authentication: Requires Clerk authentication (browser session or JWT).

GET /v1/me/recommendations/{kind} (public) — Get personalized recommendations — Get recommendations for datasets, providers, or users.

Returns personalized recommendations based on user's activity (starred datasets, followed providers, created views). Cold-start users get recommendations based on popularity rankings.

Recommendation Kinds:

• datasets: Popular or topically-related datasets

• providers: Data providers similar to user's interests

• users: Users with similar taste in datasets

Authentication: Requires Clerk authentication (browser session or JWT).

GET /v1/me/feed (public) — Get user activity feed — Get a feed of updates relevant to the user.

Returns dataset activity from providers the user follows, interleaved by time. Currently supports followed provider updates; user follow system is not yet available.

Feed Types:

• followed: Dataset updates from providers you follow

• people: Updates from users you follow (not yet available, returns empty)

• merged: Currently coerced to followed (the people feed is empty so the merge step would produce broken cursors).

Pagination: Use the next_cursor from the response as the cursor parameter for the next page.

Authentication: Requires Clerk authentication (browser session or JWT).

POST /v1/enrichment/request/{provider}/{dataset} (auth: Bearer OPENDATA_API_KEY) — Request Enrichment — Public endpoint: any authenticated user can request enrichment.

Rate-limited to one request per user per dataset per hour. Records the request in enrichment_requests for audit, and (if not rate-limited) queues the dataset by setting enrichment_status='pending' and waking the worker -- same effect as the admin retry endpoint.

POST /v1/user-views (auth: Bearer OPENDATA_API_KEY) — Create a user view — Create a new user view.

Slugs are unique per creator. Defaults to draft (is_draft=True). Reserved slugs (latest, versions, etc., or any starting with a digit) are rejected with 400.

POST /v1/user-views/create-and-publish (auth: Bearer OPENDATA_API_KEY) — Create a view and publish 1.0.0 atomically — Create a view and immediately publish 1.0.0.

Atomic: the create + publish run inside a single transaction with one final commit. If publish fails for any reason, the create rolls back — no orphan draft is left behind. release_note comes from the request body (see :class:UserViewCreate).

GET /v1/user-views/{clerk_id}/{slug} (public) — Get a specific user view — Get a view by (clerk_id, slug).

Drafts are owner-only. Published views are public.

PATCH /v1/user-views/{clerk_id}/{slug} (auth: Bearer OPENDATA_API_KEY) — Update a user view — Update a view.

For published views, config edits go to draft_config so the published snapshot stays frozen. Slug changes are allowed only on drafts.

DELETE /v1/user-views/{clerk_id}/{slug} (auth: Bearer OPENDATA_API_KEY) — Delete a user view

POST /v1/user-views/{clerk_id}/{slug}/publish (auth: Bearer OPENDATA_API_KEY) — Publish a new version — Publish the current draft (or working-copy edits) as a new version.

bump_type is ignored for the first publish of a draft (always 1.0.0).

GET /v1/user-views/{clerk_id}/{slug}/diff (public) — Diff working copy vs the last published version

GET /v1/user-views/{clerk_id}/{slug}/versions (public) — List a view's published versions (owner-aware) — List versions for a view addressed by (clerk_id, slug).

Mirrors the public /v1/views/@user/slug/versions endpoint but uses the clerk_id-based URL so the sandbox can fetch its own drafts.

GET /v1/views/@{username}/{slug}/versions (public) — List a view's published versions — Return every published version for @user/slug.

GET /v1/views/@{username}/{slug_with_version}/meta (public) — Get version metadata — Return metadata for the resolved version (display name, schema, etc.).

GET /v1/views/@{username}/{slug_with_version} (public) — Get versioned view data — Resolve a versioned view and execute its query against parquet.

Parses the frozen user_view_versions.config snapshot into a UserViewConfig, looks up the dataset's parquet path, and routes the query through the same execution path used by sandbox preview (execute_user_view_config). Rows are recorded as a request_event so the view_count trigger fires.

limit and offset are accepted on the URL; offset is currently a no-op for keyset-paginated query_with_view (the executor uses an opaque cursor instead). Honoring limit is the priority — a UI that wants paging should land here once the executor exposes offset semantics.

GET /v1/views/@{username}/{slug}/top-callers (public) — Top callers for a view in a time window — Top-5 named callers + an "unattributed" bucket for the last N days.

GET /v1/figures (public) — List figures (discovery feed) — List public figures for the discovery feed.

A figure is a user view with a viz_spec (visualization configuration) set. Results include viz_spec, like_count, and standard view metadata.

Sort options:

• popular: ordered by like_count DESC, then created_at DESC

• recent: ordered by created_at DESC

This endpoint is public and does not require authentication.

GET /v1/figures/{figure_id} (public) — Get a figure by UUID — Get a figure by its UUID.

Returns full view details including viz_spec, transform_config, inline_data, origin, like_count, and is_liked_by_me.

Visibility: public figures are returned to anyone; private drafts are returned only to their creator. Non-owners get a 404 for private drafts (we don't leak existence).

Authentication: Optional. When authenticated, the creator can read their own drafts and is_liked_by_me reflects the user's like status.

POST /v1/figures/drafts (auth: Bearer OPENDATA_API_KEY) — Create a draft figure (snapshot mode) — Create a snapshot draft figure on the authenticated user's profile.

Designed for external clients (MCP today, web sandbox tomorrow) that already have rendered data in hand and want to hand the user off into the figure editor.

Behavior:

• Caps inline_data at 1000 rows / 512KB serialized. Anything larger is truncated and truncated=true is returned.

• Per-user soft cap of 100 drafts per origin. Over-limit clients get a 429 asking them to delete or publish before creating more.

• Idempotent within 60 seconds on the SHA256 of (viz_spec, inline_data, creator_id). Re-posts inside the window return the existing draft id.

• Creates the row as a private, standalone figure. Pre-p…

GET /v1/requests (public) — List Requests — List dataset requests in the public queue.

Sorted by demand (vote count) descending by default. Pass sort=newest to sort by creation time instead. If the caller is authenticated, each entry includes whether they've upvoted it.

POST /v1/requests (auth: Bearer OPENDATA_API_KEY) — Submit Request — Submit a new dataset request to the review queue.

Description-first intake: user_description is required (>=20 chars, enforced by the schema). URL fields are optional. When a landing_page_url is provided, we run URL validation + a lightweight metadata scrape so the admin queue can show a page title and scraped meta description. We do NOT auto-discover or ingest — fulfillment is fully manual.

For description-only submissions, we run a quick search to surface a potential dataset match (hint only — the request is still created).

PUT /v1/requests/{request_id}/upvote (auth: Bearer OPENDATA_API_KEY) — Upvote Request — Upvote a dataset request to increase its priority.

DELETE /v1/requests/{request_id}/upvote (auth: Bearer OPENDATA_API_KEY) — Remove Upvote — Remove an upvote from a dataset request.

GET /v1/requests/{request_id} (public) — Get Request Detail — Public detail view of a dataset request.

Includes the linked-dataset slug when fulfilled, the submitter's display info, and the comment count. Caller's upvote state is included when authenticated.

GET /v1/requests/{request_id}/events (public) — List Request Events — Paginated thread events for a single request (oldest-first).

Soft-deleted comments come back as tombstones (body=null, deleted_at set) so the frontend can render "[deleted on ]" without losing position in the timeline.

POST /v1/requests/{request_id}/comments (auth: Bearer OPENDATA_API_KEY) — Add Comment — Add a comment to the request thread. Authenticated users only.

PATCH /v1/requests/{request_id}/comments/{event_id} (auth: Bearer OPENDATA_API_KEY) — Edit Comment — Edit a comment. Author or admin only. Sets edited_at.

DELETE /v1/requests/{request_id}/comments/{event_id} (auth: Bearer OPENDATA_API_KEY) — Delete Comment — Soft-delete a comment (sets deleted_at). Author or admin only.

POST /v1/requests/{request_id}/approve (auth: Bearer OPENDATA_API_KEY) — Approve Request — Approve a dataset request (pure status change, no pipeline trigger).

Admin-only. Transitions status to approved. The actual dataset creation and ingestion happens separately via the fulfill endpoint.

POST /v1/requests/{request_id}/in-progress (auth: Bearer OPENDATA_API_KEY) — Mark In Progress — Mark a dataset request as in-progress. Admin-only.

POST /v1/requests/{request_id}/fulfill (auth: Bearer OPENDATA_API_KEY) — Fulfill Request — Fulfill a dataset request by linking it to a created dataset. Admin-only.

GET /v1/requests/{request_id}/admin (public) — Get Request Admin Detail — Get full admin detail view for a dataset request. Admin-only.

PATCH /v1/requests/{request_id}/notes (auth: Bearer OPENDATA_API_KEY) — Update Operator Notes — Update admin operator notes on a request. Admin-only.

POST /v1/requests/{request_id}/reject (auth: Bearer OPENDATA_API_KEY) — Reject Request — Reject a dataset request with a reason. Admin-only.

POST /v1/datasets/{provider}/{dataset}/query (auth: Bearer OPENDATA_API_KEY) — Execute SQL query against a dataset — Execute a SQL query against a dataset's parquet file.

The query runs in a sandboxed DuckDB environment. Only SELECT statements are allowed. Table references like data or provider/dataset are bound to the dataset's parquet file automatically.

POST /v1/query (auth: Bearer OPENDATA_API_KEY) — Execute SQL query across multiple datasets — Execute a SQL query across multiple datasets.

Table references use provider.dataset or provider/dataset notation. Each referenced dataset's parquet file is loaded into DuckDB as a named table. Only SELECT statements are allowed.

GET /v1/graph/datasets/{provider}/{dataset}/stats (public) — Get Dataset Stats — Get graph-computed statistics for a dataset.

GET /v1/graph/datasets/{provider}/{dataset}/join-paths (public) — Get Join Paths — Discover multi-hop join paths from a dataset.

GET /v1/graph/datasets/{provider}/{dataset}/related (public) — Get Related — Get graph-powered related datasets (structural + semantic blended).

GET /v1/graph/datasets/{provider}/{dataset}/neighbors (public) — Get Neighbors — Get direct 1-hop connections from a dataset.

GET /v1/graph/datasets/{provider}/{dataset}/schema-graph (public) — Get Schema Graph — Get the schema-level subgraph around a dataset for D3 visualization.

GET /v1/graph/bridges (public) — Get Bridges — Get top bridge datasets by betweenness centrality.

GET /v1/graph/communities (public) — Get Communities — Get community listing with top datasets and dominant topics.

GET /v1/graph/communities/{community_id}/datasets (public) — Get Community Datasets — List datasets in a graph community, ordered by importance.

GET /v1/graph/subgraph (public) — Get Subgraph — Get a seeded subgraph for the graph explorer.

When seed_type/seed_id are omitted, returns a sample of the full graph. For dataset seeds, seed_id should be 'provider/slug' (e.g. 'bls/cpi-u').

GET /v1/graph/entities/{entity_type}/{entity_id}/datasets (public) — Get Entity Datasets — Get datasets referencing a specific entity.

GET /v1/graph/health (public) — Get Health — Get graph health and sync status.