# AWS S3 MCP Server (Model Context Protocol)

[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![MCP SDK](https://img.shields.io/badge/MCP%20SDK-1.0+-purple.svg)](https://modelcontextprotocol.io/)
[![AWS Boto3](https://img.shields.io/badge/AWS-boto3-orange.svg)](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html)
[![Tests Passing](https://img.shields.io/badge/tests-23%20passed-brightgreen.svg)]()
[![Security](https://img.shields.io/badge/security-zero--credential--leakage-green.svg)]()
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)

A production-grade, **plug-and-play** [Model Context Protocol (MCP)](https://modelcontextprotocol.io/) server built with Python, the official MCP Python SDK, and AWS SDK (`boto3`).

It empowers AI assistants (such as **Claude Code**, **Claude Desktop**, **Cursor**, and **Antigravity**) to explore, scan, inspect, upload, replicate, move, snapshot, and back up data across any Amazon S3 bucket with enterprise-grade reliability.

---

## Key Highlights

- **First-Run Guided Onboarding**: AI assistants proactively guide users on first launch (`check_onboarding_status`, `configure_s3_connection`) to connect their AWS credentials and buckets without manual file editing. Also includes interactive `python setup.py` CLI wizard.
- **Plug-and-Play Bucket Support**: Works out of the box with standard S3 buckets (`my-bucket`), bucket ARNs (`arn:aws:s3:::my-bucket`), S3 Access Point ARNs, or Access Point Aliases.
- **Dynamic Multi-Bucket Support**: Seamlessly switch buckets on the fly by passing `bucket="other-bucket"` in any tool call, or configure a default target in `.env`.
- **Multi-Bucket Simultaneous Ingestion**: Write or broadcast data to multiple S3 buckets simultaneously (`write_to_multiple_buckets`) for multi-region replication and backup distribution.
- **Cross-Bucket Data Transfer (Move & Copy)**: Transfer data between buckets server-side on AWS without consuming local client bandwidth (`copy_s3_object`, `move_s3_object`).
- **Snapshots & Point-in-Time Backups**:
  - `create_s3_backup`: Creates point-in-time timestamped snapshots of entire buckets or folder prefixes into a backup bucket.
  - `create_object_snapshot`: Creates timestamped object-level snapshots instantly.
  - `configure_bucket_versioning`: Configures native AWS S3 versioning protection.
- **Account-Wide Bucket Scanner & Feasibility Audit**: `scan_all_buckets` discovers all account buckets, checking accessibility, AWS regions, versioning status, and multi-bucket operation feasibility.
- **Universal Object Scanner**: Scan and filter any object type (JSON, CSV, Parquet, PDF, images, logs, text, binaries) by extension, size range, date range, or regex pattern via `scan_s3_objects`.
- **Content Inspection & Preview**: Directly read and preview text, code, logs, CSV tables, and JSON data with built-in line/byte pagination (`read_s3_file`), preventing prompt bloat and token exhaustion.
- **Hierarchical Directory Navigation**: Delimiter-based folder exploration (`list_s3_folders`) for deep directory tree traversal.
- **Safe File Downloads**: Stream any object to local disk with atomic temp files and overwrite protection (`download_s3_file`).
- **Zero Credential Leakage**: Secrets and tokens are never printed or logged. Stderr-only logging keeps the stdio JSON-RPC stream clean.

---

## Architecture Overview

```text
+-----------------------------------------------------------------------------------+
|  AI Assistant / MCP Client (Claude Code, Claude Desktop, Cursor, Antigravity)     |
+-----------------------------------------------------------------------------------+
                                          |
                             (JSON-RPC via stdio)
                                          v
+-----------------------------------------------------------------------------------+
|  AWS S3 MCP Server (server.py)                                                    |
|  - Multi-Bucket & Access Point Resolver (dynamic override or .env default)        |
|  - Multi-Bucket Broadcast Ingestion (parallel/multi-target put)                   |
|  - Server-Side Cross-Bucket Copy & Move (zero local bandwidth)                    |
|  - Automated Point-in-Time Snapshots & Backup Engine                              |
|  - Universal Object Scanner (extensions, size, regex, timestamps)                |
|  - Safe Content Reader (line/byte pagination, CSV & JSON parsing)                |
|  - Memory-safe streaming downloader with atomic replace                           |
|  - Stderr-only logging (preserves pristine JSON-RPC transport)                    |
+-----------------------------------------------------------------------------------+
                                          |
                               (AWS SigV4 HTTPS / TLS)
                                          v
+-----------------------------------------------------------------------------------+
|  Amazon S3                                                                        |
|  - Primary Buckets (e.g. s3://prod-data-lake)                                     |
|  - Backup / Disaster Recovery Buckets (e.g. s3://dr-backup-vault)                 |
|  - S3 Access Points (arn:aws:s3:<region>:<account>:accesspoint/<name> / alias)    |
|  - Objects: Data (CSV/JSON/Parquet), Docs (PDF), Code, Media, Logs, Binaries      |
+-----------------------------------------------------------------------------------+
```

---

## Complete MCP Tools Reference

### 1. First-Run Onboarding & Account Connection
| Tool Name | Parameters | Description |
| :--- | :--- | :--- |
| `check_onboarding_status` | None | **First-run check**: Inspects `.env` presence, AWS credentials, STS identity, and guides Claude to help the user connect their account. |
| `configure_s3_connection` | `aws_region="us-east-1"`, `s3_bucket=""`, `aws_profile=""`, `aws_access_key_id=""`, `aws_secret_access_key=""`, `aws_session_token=""`, `s3_access_point_arn=""` | **Automated setup tool**: Allows Claude to configure or update `.env` with AWS credentials and target bucket without manual file editing. |

### 2. Ingestion & Multi-Bucket Operations
| Tool Name | Parameters | Description |
| :--- | :--- | :--- |
| `put_s3_object` | `key` *(req)*, `content` *(req)*, `bucket=""`, `content_type=""` | Writes string, JSON, CSV, code, or logs directly to an S3 object. |
| `upload_s3_file` | `local_path` *(req)*, `key=""`, `bucket=""`, `content_type=""` | Uploads a local file from disk to S3. Auto-detects filename and MIME type. |
| `write_to_multiple_buckets` | `key` *(req)*, `content` *(req)*, `buckets` *(req)*, `content_type=""` | **Broadcasts data to multiple S3 buckets simultaneously**. Accepts a list or comma-separated string of bucket names. |

### 2. Cross-Bucket Transfer, Snapshots & Backups
| Tool Name | Parameters | Description |
| :--- | :--- | :--- |
| `copy_s3_object` | `source_bucket` *(req)*, `source_key` *(req)*, `destination_bucket` *(req)*, `destination_key=""` | Copies an object between buckets using AWS server-side copy (zero local bandwidth). |
| `move_s3_object` | `source_bucket` *(req)*, `source_key` *(req)*, `destination_bucket` *(req)*, `destination_key=""` | Moves an object between buckets (server-side copy + atomic source deletion). |
| `create_s3_backup` | `source_bucket` *(req)*, `backup_bucket` *(req)*, `prefix=""`, `backup_prefix=""`, `timestamp_snapshot=True`, `max_objects=500` | **Point-in-time snapshot backup**: Copies bucket/prefix objects into a timestamped snapshot structure in the backup bucket. |
| `create_object_snapshot` | `key` *(req)*, `bucket=""`, `snapshot_prefix="snapshots/"` | Creates an instant timestamped copy snapshot of a specific S3 object. |
| `configure_bucket_versioning` | `bucket=""`, `status="Enabled"` | Enables or suspends AWS native bucket object versioning. |

### 3. Account-Wide & Bucket Discovery
| Tool Name | Parameters | Description |
| :--- | :--- | :--- |
| `list_s3_buckets` | None | Lists all S3 buckets in the AWS account with creation dates and default target. |
| `scan_all_buckets` | `include_details=True`, `max_buckets=50` | **Account-level feasibility audit**: Scans all buckets, checking read accessibility, AWS regions, versioning status, and object count estimates. |
| `test_s3_connection` | `bucket=""` | Validates AWS credentials, inspects STS caller identity, and runs a read canary. |

### 4. Inspection, Scanning & Downloads
| Tool Name | Parameters | Description |
| :--- | :--- | :--- |
| `list_s3_files` | `bucket=""`, `prefix=""`, `max_keys=100`, `continuation_token=""` | Lists object keys with size, human-readable size, last modified, category, and storage class. |
| `list_s3_folders` | `bucket=""`, `prefix=""`, `max_keys=100`, `continuation_token=""` | Hierarchical folder navigation using delimiter (`/`). |
| `search_s3_files` | `query` *(req)*, `bucket=""`, `prefix=""`, `max_results=50` | Substring search across object keys and filenames. |
| `scan_s3_objects` | `bucket=""`, `prefix=""`, `extensions=None`, `min_size_bytes=None`, `max_size_bytes=None`, `modified_after=None`, `modified_before=None`, `key_pattern=""`, `max_keys=100`, `continuation_token=""` | **Multi-criteria scanner**: Filter any bucket by extensions (e.g. `['.parquet', '.csv']`), size bounds, ISO dates, and regex patterns. Returns category summaries. |
| `read_s3_file` | `key` *(req)*, `bucket=""`, `max_bytes=65536`, `max_lines=500`, `line_offset=0` | **Read & preview file content**: Text/code/logs with line pagination; JSON preview; CSV tabular preview; binary detection. |
| `get_s3_object_metadata` | `key` *(req)*, `bucket=""` | Retrieves HTTP headers and S3 metadata (`HeadObject`) without downloading the body. |
| `get_s3_bucket_summary` | `bucket=""`, `prefix=""` | Computes inventory metrics: total object count, total bytes, storage class breakdown, and extension distribution. |
| `download_s3_file` | `key` *(req)*, `bucket=""`, `local_path=""`, `overwrite=False` | Streams object to local disk with atomic temporary files, progress safety, and overwrite guard. |
| `download_s3_object` | `key` *(req)*, `bucket=""`, `local_path=""`, `overwrite=False` | Convenience alias for `download_s3_file`. |

---

## Quickstart & Setup Guide

### 1. Clone Repository & Setup Environment

#### On Windows (PowerShell):
```powershell
# Clone the repository
git clone https://github.com/jnvsainikhil/s3-mcp.git
cd s3-mcp

# Create Python virtual environment
python -m venv .venv

# Activate virtual environment
.\.venv\Scripts\Activate.ps1

# Upgrade pip and install dependencies
pip install --upgrade pip
pip install -r requirements.txt
```

#### On Linux / macOS:
```bash
# Clone the repository
git clone https://github.com/jnvsainikhil/s3-mcp.git
cd s3-mcp

# Create Python virtual environment
python3 -m venv .venv

# Activate virtual environment
source .venv/bin/activate

# Upgrade pip and install dependencies
pip install --upgrade pip
pip install -r requirements.txt
```

---

### 2. Configure AWS Credentials

Copy the template configuration file:
```bash
cp .env.example .env
```

Open `.env` and configure your credentials:

```env
AWS_REGION=us-east-1

# Optional default bucket (multi-bucket mode works even without this!)
S3_BUCKET=my-primary-bucket

# If using AWS CLI profiles or SSO (Recommended):
# AWS_PROFILE=default

# If using direct keys:
# AWS_ACCESS_KEY_ID=AKIAEXAMPLEKEY
# AWS_SECRET_ACCESS_KEY=exampleSecretKey123
```

---

### 3. Verify Setup with Diagnostics

Run the diagnostic tool to test your environment:

```bash
# Windows
.\.venv\Scripts\python.exe diagnose_s3.py

# Linux / macOS
python diagnose_s3.py
```

---

### 4. Run Unit Tests

Execute the automated test suite (21 offline unit tests with mocks):

```bash
# Windows
.\.venv\Scripts\python.exe -m pytest -v

# Linux / macOS
pytest -v
```

---

## Client Integration Guides

### Claude Code

Claude Code supports project-scoped MCP configurations via `.mcp.json`:

```bash
# Register project-scoped MCP server:
claude mcp add --scope project s3-mcp -- python server.py
```

Or copy `.mcp.json.example` to `.mcp.json`:
```json
{
  "mcpServers": {
    "s3-mcp": {
      "type": "stdio",
      "command": "python",
      "args": [
        "server.py"
      ],
      "env": {}
    }
  }
}
```

Verify with:
```bash
claude mcp list
claude mcp get s3-mcp
```

---

### Claude Desktop

Add the server to your `claude_desktop_config.json`:

- **Windows**: `%APPDATA%\Claude\claude_desktop_config.json`
- **macOS**: `~/Library/Application Support/Claude/claude_desktop_config.json`

```json
{
  "mcpServers": {
    "s3-mcp": {
      "command": "python",
      "args": [
        "/path/to/s3-mcp/server.py"
      ],
      "env": {
        "AWS_REGION": "us-east-1",
        "S3_BUCKET": "my-s3-bucket",
        "AWS_PROFILE": "default"
      }
    }
  }
}
```

---

### Cursor / Antigravity IDE

Add to your MCP server configuration:
```json
{
  "name": "s3-mcp",
  "command": "python",
  "args": ["server.py"]
}
```

---

## Practical Usage Examples

### 1. Ingest Data into Multiple Buckets at Once
```python
# Broadcast config or telemetry to primary and secondary buckets:
write_to_multiple_buckets(
    key="config/global-settings.json",
    content='{"version": "2.4", "replication": true}',
    buckets=["prod-us-east-1", "backup-us-west-2", "dr-eu-central-1"]
)
```

### 2. Move or Migrate Data Between Buckets
```python
# Move processed dataset from incoming to warehouse:
move_s3_object(
    source_bucket="raw-data-landing",
    source_key="2026/05/transactions.parquet",
    destination_bucket="analytics-warehouse",
    destination_key="gold/transactions.parquet"
)
```

### 3. Create a Point-in-Time Snapshot Backup
```python
# Snapshot entire folder prefix into a backup bucket:
create_s3_backup(
    source_bucket="analytics-warehouse",
    backup_bucket="long-term-archives",
    prefix="gold/",
    timestamp_snapshot=True
)
```

### 4. Audit All Account Buckets for Feasibility
```python
# Scan all buckets, check permissions, regions, and versioning:
scan_all_buckets(include_details=True)
```

### 5. Multi-Criteria Scanning for Datasets
```python
# Scan for Parquet and CSV files larger than 1MB:
scan_s3_objects(
    bucket="analytics-warehouse",
    prefix="datasets/",
    extensions=[".parquet", ".csv"],
    min_size_bytes=1048576
)
```

---

## AWS IAM Security Policies

Production-ready least-privilege IAM and bucket policies are included in the [`aws/`](aws/) directory:

1. **[`aws/iam-bucket-policy.json`](aws/iam-bucket-policy.json)**: Least-privilege IAM policy covering reading, uploading, cross-bucket copying/moving, and versioning/backups.
2. **[`aws/iam-policy.json`](aws/iam-policy.json)**: Least-privilege IAM policy for S3 Access Point usage.
3. **[`aws/access-point-policy.json`](aws/access-point-policy.json)**: Access Point policy restricting access to specific principals.
4. **[`aws/bucket-policy.json`](aws/bucket-policy.json)**: Bucket policy delegating access exclusively through the S3 Access Point.

---

## Security & Privacy Guarantee

- **Zero Secret Exposure**: AWS credentials, access keys, and session tokens are strictly excluded from logging and output payloads.
- **No Inbound Open Ports**: Communicates exclusively over standard I/O (`stdio`).
- **Memory Safety**: Streaming downloads and uploads are chunked and temporary files are used for atomic consistency.

---

## Troubleshooting

| Issue | Cause | Solution |
| :--- | :--- | :--- |
| `AccessDenied (403)` on list | IAM lacks `s3:ListBucket` or `s3:ListAllMyBuckets` | Attach [`aws/iam-bucket-policy.json`](aws/iam-bucket-policy.json) to your IAM principal. |
| `AccessDenied (403)` on copy/move | IAM lacks `s3:PutObject` on destination or `s3:DeleteObject` on source | Check IAM permissions on both source and destination buckets. |
| `NoSuchBucket` | Bucket does not exist or region mismatch | Ensure `AWS_REGION` matches the bucket region and check the bucket name spelling. |
| `NoCredentialsError` | Boto3 cannot discover credentials | Run `aws configure` or populate credentials / profile in `.env`. |
| Claude shows `Pending approval` | Project server requires approval | In Claude Code, run `claude` and confirm approval for `s3-mcp`. |

---

## License

This project is licensed under the [MIT License](LICENSE).