🔌 API Registry MCP Server

A Databricks app that helps you discover, register, and manage external API endpoints with an AI-powered chat interface.

What is this?

An API discovery and management platform that runs on Databricks Apps:

• 🤖 AI Chat Interface: Register APIs using natural language powered by Claude

• 📊 API Registry: Database-backed catalog of external API endpoints

• 🔐 Secure Auth: Support for public APIs, API keys, and bearer tokens

• 🛠️ MCP Server: Programmatic API management tools

• 📚 Smart Discovery: Automatic endpoint testing and documentation parsing

Quick Start

Prerequisites

Required Tools (Install on your local machine)

1. Python Package Manager - uv:

uv documentation

2. Databricks CLI:

Databricks CLI documentation

3. Bun (Optional - only for frontend development):

Bun documentation

Databricks Workspace Requirements

Your workspace needs:

• Databricks Apps enabled (Public Preview)

• Foundation Model API with a tool-enabled model (Claude, Llama, etc.)

• SQL Warehouse - At least one warehouse (create one)

• Unity Catalog - With a catalog and schema you can write to

📖 Detailed requirements: WORKSPACE_REQUIREMENTS.md

Step 1: Clone and Setup

Run this on your local machine (not in Databricks):

The setup script will prompt you for:

⚠️ Important: Use Personal Access Token (PAT) authentication

• PAT is the recommended method for local development

• OAuth is experimental and may have issues

• Get your PAT: Workspace → Settings → Developer → Access Tokens → Generate New Token

• Full PAT documentation

What this does:

• Installs Python and JavaScript dependencies

• Configures Databricks CLI authentication

• Creates .env.local with your configuration

• Validates your workspace connection

Step 2: Create the API Registry Table

Create the Delta table that stores API metadata:

Example:

What this does:

• Creates api_http_registry table in your specified catalog.schema

• Table stores: API name, endpoints, auth type, HTTP connection details, parameters

• Required for the app to track registered APIs

Alternative - Manual SQL: Run the SQL from setup_api_http_registry_table.sql in Databricks SQL Editor

Note: Ensure your catalog and schema exist first. Create them in Databricks SQL Editor if needed.

Step 3: Deploy to Databricks Apps

Deploy your application code to Databricks:

During deployment, you'll be prompted for:

• App name: Must start with mcp- (e.g., mcp-api-registry, mcp-prod-api)

What happens during deployment:

1. ✅ Builds the frontend - Compiles React TypeScript to static assets

2. ✅ Packages the backend - Prepares FastAPI server and MCP tools

3. ✅ Creates Databricks App - Registers your app in the workspace

4. ✅ Generates Service Principal - Automatically creates a service principal for your app

5. ✅ Deploys code to the app - Uploads your code and automatically attaches it to the app compute

6. ✅ Starts the application - Your app is now running and accessible

7. ✅ Enables OAuth (OBO) - Configures On-Behalf-Of authentication automatically

⚠️ Important: No manual attachment needed! The deploy.sh script handles the entire deployment pipeline. Your code is automatically:

• Packaged into a deployable artifact

• Uploaded to Databricks

• Attached to the app's compute environment

• Started and made accessible at the app URL

You don't need to manually connect code to compute - it's all handled by the deployment process!

Finding your deployed app:

Or in Databricks UI:

• Workspace → Compute → Apps → Click your app name

🔐 On-Behalf-Of (OBO) Authentication:

Databricks Apps automatically handles OAuth authentication:

• ✅ Users log in through Databricks UI - no separate auth setup

• ✅ All operations run with the user's permissions - proper access control

• ✅ Full audit logging - track who did what

• ✅ No manual OAuth configuration needed!

The app configuration (app.yaml) specifies required scopes. When users access the app, they automatically get an OAuth token with their Databricks permissions.

📖 More details: See app.yaml in the project root

Step 4: Setup Secret Scopes (For Authenticated APIs)

⚠️ Important: Do this AFTER Step 3 - You need the Service Principal ID from deployment first!

Skip if you only use public APIs with no authentication.

For APIs requiring API keys or bearer tokens:

When prompted, enter your app's Service Principal ID from Step 3.

Where to find your Service Principal ID:

1. From terminal: Run ./app_status.sh (shown in output)

2. From UI: Databricks workspace → Compute → Apps → Click your app → "Service Principal ID"

3. Format: Looks like 00000000-0000-0000-0000-000000000000

What this script does:

1. Creates mcp_api_keys scope - for API key authentication

2. Creates mcp_bearer_tokens scope - for bearer token authentication

3. Grants your app's service principal WRITE access to both scopes

4. Verifies the permissions were set correctly

Why this is needed:

• API keys and bearer tokens must be stored securely

• Databricks Secrets provide encryption at rest

• The app's service principal manages secrets on behalf of all users

• Users never see or handle raw credentials - they're encrypted automatically

Verification:

Troubleshooting:

• If scope creation fails: You may need admin permissions

• If permission grant fails: Your SPN ID may be incorrect (check ./app_status.sh)

📖 Detailed guide: SECRETS_WORKAROUND.md

API Authentication Types

The app supports three authentication types:

Quick Examples

Public API (no auth):

API Key Authentication:

Bearer Token Authentication:

How It Works

API Key Auth:

• Key stored in mcp_api_keys scope

• HTTP connection has empty bearer_token

• Key retrieved from secrets and added to params at runtime

Bearer Token Auth:

• Token stored in mcp_bearer_tokens scope

• HTTP connection references the secret

• Databricks automatically adds Authorization: Bearer <token> header

📖 Detailed auth mechanics: See "API Authentication Types" section in SECRETS_WORKAROUND.md

Using the App

Web Interface

Open your app URL to access:

1. Chat Playground - Natural language API registration and queries

2. API Registry - View, edit, delete registered APIs

3. Traces - Debug AI agent execution

4. MCP Info - View available MCP tools

Example Workflow

Configuration

Environment Variables (.env.local)

Created automatically by ./setup.sh:

Authentication

The app uses On-Behalf-Of (OBO) authentication by default:

• Users authenticate with Databricks OAuth

• All operations run with the user's permissions

• Proper access control and audit logging

📖 OBO details: See app.yaml configuration in the project root

Development

Local Development

Debugging

Multiple Environments

Deploy separate instances for dev/staging/prod:

Project Structure

Troubleshooting

Deployment Issues - App Created But Code Not Working

If ./deploy.sh completes successfully but your app doesn't work properly, follow these steps:

1. Check App Logs (MOST IMPORTANT):

2. Verify App Status:

3. Common Causes & Fixes:

4. Redeploy with Verbose Output:

5. Manual Verification:

Authentication failures:

• Run: databricks current-user me to verify CLI auth

• Check .env.local has correct DATABRICKS_HOST

Table not found:

• Run setup_table.py or manually create via SQL Editor

Secret scope errors:

App not accessible:

• Check deployment: ./app_status.sh

• View logs: https://your-app.databricksapps.com/logz

API calls failing after registration:

• Verify secret exists: databricks secrets list-secrets --scope mcp_api_keys

• Check app logs for connection creation errors

• For API key auth: Ensure key is in mcp_api_keys scope

• For bearer token auth: Ensure token is in mcp_bearer_tokens scope

📖 Detailed troubleshooting:

• WORKSPACE_REQUIREMENTS.md - Workspace setup issues

• SECRETS_WORKAROUND.md - Secret scope issues

Key Features

MCP Tools Available

The app exposes these tools via its MCP server:

• smart_register_api - One-step API registration with auto-discovery

• register_api_in_registry - Manual API registration with full control

• check_api_http_registry - List and search registered APIs

• discover_endpoints_from_docs - Extract endpoints from documentation URLs

• test_api_endpoint - Validate endpoints before registration

• execute_dbsql - Run SQL queries against warehouses

AI Agent Capabilities

The chat interface can:

• Parse API documentation to discover endpoints

• Test endpoints automatically

• Register APIs with proper authentication

• Call registered APIs to answer queries

• Combine multiple API calls for complex requests

Documentation

• WORKSPACE_REQUIREMENTS.md - Prerequisites, setup, workspace configuration

• SECRETS_WORKAROUND.md - Secret management, auth types, troubleshooting

• SECURITY.md - Security policies

• LICENSE.md - License information

License

See LICENSE.md

Security

Report vulnerabilities: See SECURITY.md