<div align="center">

# databricks-capstone-delivery

[![Spark](https://img.shields.io/badge/Apache_Spark-3.5%2B-E25A1C?style=for-the-badge&logo=apachespark&logoColor=white)](https://spark.apache.org/)
[![Databricks](https://img.shields.io/badge/Databricks-Platform-FF3621?style=for-the-badge&logo=databricks&logoColor=white)](https://databricks.com/)
[![Python](https://img.shields.io/badge/Python-3.10%2B-3776AB?style=for-the-badge&logo=python&logoColor=white)](https://www.python.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg?style=for-the-badge)](LICENSE)

<p><b>Assistente de pesquisa de mercado de ações impulsionado por Databricks, RAG com pgvector e Agentes MCP.</b></p>

<img src="docs/images/project-hero.svg" alt="databricks-capstone-delivery Hero Banner" width="760">

</div>

# Weather Intelligence

Databricks AI Bootcamp capstone: a weather intelligence service that ingests National Weather Service data, creates 384-dimensional embeddings, stores them in Lakebase Postgres with pgvector, and exposes grounded retrieval through a dashboard and MCP server.

## Architecture

- **Ingestion:** Python notebook fetches NWS observations, alerts, and forecasts.
- **Storage:** Lakebase Autoscaling Postgres stores weather documents and `VECTOR(384)` embeddings with an HNSW index.
- **Retrieval:** Flask `POST /weather/search` embeds a query and performs parameterized cosine-distance search.
- **MCP:** FastMCP exposes `get_current_weather`, `get_forecast`, `predict_umbrella_needed`, `save_weather_note`, `add_weather_watchlist`, and `remove_weather_watchlist`.
- **Apps:** `weather-dashboard` serves the browser UI; `weather-mcp` is attached to the Agent Bricks supervisor agent.

## Repository layout

```text
dashboard/                         Flask app, Lakebase helpers, weather UI
weather_mcp_server/                FastMCP service and write tools
notebooks/ingest_weather_embeddings.py
sql/05_setup_weather_documents.sql
sql/06_setup_weather_embeddings.sql
sql/07_setup_research_notes.sql
resources/                         Databricks bundle resources
docs/EVIDENCIAS_WEATHER.md        Evidence mapping and validation notes
evidence/                          Reproducible execution evidence
submissions/                       Three capstone submission archives
```

## Run locally

```bash
pip install -r dashboard/requirements.txt
FLASK_APP=dashboard.app flask run --port 8001
```

The deployed Apps receive Lakebase access through Databricks Secrets. Do not place connection URLs, OAuth tokens, or other credentials in Git.

## API examples

```bash
curl -X POST http://localhost:8001/weather/search \
  -H 'Content-Type: application/json' \
  -d '{"query":"heavy rain in Lisbon","top_k":5}'

curl http://localhost:8001/api/watchlist
curl -X POST http://localhost:8001/api/watchlist/LISBON
curl -X DELETE http://localhost:8001/api/watchlist/LISBON
```

## Validation evidence

The implementation and runtime evidence are mapped in [`docs/EVIDENCIAS_WEATHER.md`](docs/EVIDENCIAS_WEATHER.md). The three submission packages are:

- `submissions/vector-weather-retrieval-service.zip`
- `submissions/build-your-own-weather-mcp-server.zip`
- `submissions/capstone-project-submission.zip`

Each archive has a companion SHA-256 checksum file.

## Security

Secrets remain in Databricks Secret Scopes. All user-scoped writes use the authenticated email and parameterized SQL. The dashboard and MCP write paths do not accept a caller-supplied identity in place of the authenticated identity when deployed behind Databricks Apps.