# Israel Weather MCP - Groq + Playwright Integration

A production-ready Model Context Protocol (MCP) implementation that provides real-time weather forecasts for Israeli cities using Groq LLM and Playwright browser automation.

## Overview

This project demonstrates an advanced MCP architecture combining:
- **Groq** for fast, cost-effective LLM inference
- **Playwright** for reliable web automation and data extraction
- **Weather2Day** (https://www.weather2day.co.il) as the data source
- **MCP (Model Context Protocol)** for standardized tool integration

The system provides a natural language interface to weather queries, automatically navigating the Weather2Day website and extracting real-time forecast data.

## Architecture

```
┌─────────────────┐
│   User Query    │
│  "Weather in    │
│  Jerusalem?"    │
└────────┬────────┘
         │
         ▼
┌─────────────────────────────┐
│   Groq LLM                  │
│  (llama-3.1-8b-instant)     │
│  - Orchestrates workflow    │
│  - Calls MCP tools          │
│  - Generates final answer   │
└────────┬────────────────────┘
         │
         ▼
┌─────────────────────────────────┐
│   MCP Host (host.py)            │
│  - Manages tool registry        │
│  - Routes tool calls            │
│  - Handles MCP protocols        │
└────────┬──────────────────┬─────┘
         │                  │
         ▼                  ▼
    ┌──────────┐      ┌──────────┐
    │ Israel   │      │   USA    │
    │ Weather  │      │ Weather  │
    │   MCP    │      │   MCP    │
    └────┬─────┘      └────┬─────┘
         │                  │
         ▼                  ▼
    ┌──────────────┐   ┌──────────────┐
    │ Playwright   │   │ NWS API      │
    │ Automation   │   │ (REST)       │
    └────┬─────────┘   └──────────────┘
         │
         ▼
    ┌──────────────────────────┐
    │  Weather2Day Website     │
    │  (weather2day.co.il)     │
    └──────────────────────────┘
         │
         ▼
    ┌──────────────────┐
    │ Weather Content  │
    │ (Real Data)      │
    └────────┬─────────┘
             │
             ▼
    ┌──────────────────────────────────┐
    │  Groq Final Response             │
    │  "Today in Jerusalem:            │
    │   Partly cloudy, 31.7°C..."      │
    └──────────────────────────────────┘
```

## Key Features

### ✅ Real-Time Data Extraction
- Navigates Weather2Day.co.il automatically using Playwright
- Extracts actual forecasts, temperatures, and weather alerts
- No static data or fallbacks

### ✅ Natural Language Interface
- Ask questions in Hebrew or English
- Groq processes context and automatically executes the right tool sequence
- Intelligent tool orchestration

### ✅ Dual MCP Servers
- **Israel Weather MCP** (`weather_Israel.py`): Playwright-based extraction
- **USA Weather MCP** (`weather_USA.py`): REST API integration (NWS)

### ✅ Multi-City Support
- Jerusalem, Tel Aviv, Haifa, and 60+ Israeli cities
- Any US state (via NWS weather alerts)

### ✅ Production-Ready Error Handling
- Timeout management for network reliability
- Fallback selectors for dynamic page content
- SSL/TLS proxy compatibility (Netfree, corporate networks)

## Technologies

| Component | Technology | Version |
|-----------|-----------|---------|
| **LLM** | Groq | llama-3.1-8b-instant |
| **MCP SDK** | mcp | ≥1.27.0 |
| **Browser** | Playwright | ≥1.44.0 |
| **HTTP Client** | httpx | ≥0.24.0 |
| **Config** | python-dotenv | ≥1.2.2 |
| **Language** | Python | ≥3.13 |

## Installation

### Step 1: Clone Repository
```bash
git clone <repo-url>
cd weather-chat_mcp
```

### Step 2: Install Python Dependencies
```bash
uv pip install -r pyproject.toml
```

Or using uv directly:
```bash
uv sync
```

### Step 3: Install Playwright Chromium

**Important:** Playwright requires the Chromium browser to be installed.

```bash
playwright install chromium
```

On Linux, you may also need system dependencies:
```bash
# Ubuntu/Debian
sudo apt-get install libglib2.0-0 libdbus-1-3 libfontconfig1 libxrender1

# RHEL/CentOS
sudo dnf install glib2 dbus libxrender fontconfig
```

### Step 4: Configure Groq API Key

1. Get your Groq API key from: https://console.groq.com/keys

2. Create `.env` file in project root:
```bash
cp .env.example .env
```

3. Edit `.env` and add your key:
```env
GROQ_API_KEY=your_actual_groq_api_key_here
```

**Security:** The `.env` file is automatically excluded from git (see `.gitignore`).

## Running the Project

### Start the Interactive Chat Interface

```bash
python host.py
```

You'll see:
```
MCP Client Started!
Type your queries or 'quit' to exit.

Connected to server with tools: ['open_weather_forecast_israel', 'enter_weather_forecast_city_israel', 'select_weather_forecast_city_israel', 'get_weather_forecast_content_israel']
Connected to server with tools: ['get_alerts_in_USA', 'get_forecast_in_USA']
```

### Example Queries

**Hebrew:**
```
Query: מה מזג האוויר בירושלים?
Query: מה התחזוקה עבור תל אביב?
```

**English:**
```
Query: What is the weather in Jerusalem?
Query: Show me the forecast for Haifa
Query: What are the weather alerts in California?
```

## MCP Tools

### Israel Weather MCP (`weather_Israel.py`)

#### 1. `open_weather_forecast_israel()`
Opens the Weather2Day forecast website and initializes browser session.
- **Input:** None
- **Output:** Status message or error
- **Purpose:** Step 1 of workflow

#### 2. `enter_weather_forecast_city_israel(city: str)`
Enters city name in the search field and waits for suggestions.
- **Input:** `city` - City name (e.g., "Jerusalem", "ירושלים")
- **Output:** Confirmation or error
- **Purpose:** Step 2 of workflow

#### 3. `select_weather_forecast_city_israel()`
Selects the first city suggestion and loads the forecast page.
- **Input:** None
- **Output:** Confirmation that page loaded
- **Purpose:** Step 3 of workflow

#### 4. `get_weather_forecast_content_israel()`
Extracts and returns the actual weather forecast content from Weather2Day.
- **Input:** None
- **Output:** Full weather data including temperatures, forecasts, alerts
- **Purpose:** Step 4 of workflow (final extraction)

**Workflow:** Always execute in order: `open` → `enter(city)` → `select` → `extract`

### USA Weather MCP (`weather_USA.py`)

#### 1. `get_alerts_in_USA(state: str)`
Fetches active weather alerts for a US state from NWS API.
- **Input:** `state` - Two-letter state code (e.g., "CA", "NY")
- **Output:** Active alerts with severity and description

#### 2. `get_forecast_in_USA(lat: float, lon: float)`
Gets weather forecast for coordinates from NWS API.
- **Input:** `lat`, `lon` - Latitude and longitude
- **Output:** Forecast data

## Browser Automation Details

### Playwright Features Used
- **Headless Chromium** for lightweight browser instances
- **Multiple selector strategies** for robust DOM interaction
- **Network idle detection** for page load validation
- **Async/await** for non-blocking execution
- **Global page state** persists across tool calls

### Selector Strategies (Weather_Israel.py)
The implementation uses multiple CSS selector fallbacks:
1. ID-based selectors (most stable)
2. Placeholder-based selectors (Hebrew attributes)
3. Class-based selectors
4. Generic fallbacks (input, button elements)

This approach handles dynamic page updates and unknown DOM structures.

### Network Configuration
- SSL verification disabled for proxy compatibility (Netfree, corporate networks)
- Configurable timeout (30s for navigation, 15s for interactive elements)
- Error recovery with fallback mechanisms

## Environment Variables

### Required
- **GROQ_API_KEY** - Your Groq API key from https://console.groq.com/keys

### Optional (internally configured)
- All proxy settings are handled automatically via httpx
- Playwright runs in headless mode by default

## Project Structure

```
weather-chat_mcp/
├── host.py                      # Main orchestrator (Groq + MCP coordination)
├── client.py                    # MCP client implementation (stdio transport)
├── weather_Israel.py            # Israel Weather MCP server (Playwright)
├── weather_USA.py               # USA Weather MCP server (REST API)
├── pyproject.toml               # Dependencies and project metadata
├── .env.example                 # Environment variable template
├── .env                         # Actual config (not committed, use .env.example)
├── .gitignore                   # Git ignore rules
└── README.md                    # This file
```

### Test Files (Reference)
- `test_groq_tools.py` - Integration testing
- `test_israel_tools.py` - Israel MCP tool testing
- `test_groq.py` - Groq API testing
- `debug_schemas.py` - Tool schema debugging

## How It Works

### Workflow Example: "What is the weather in Jerusalem?"

```
1. User Input
   └─> "What is the weather in Jerusalem?"

2. Groq LLM Processing
   ├─> Reads system prompt (Hebrew instructions)
   ├─> Analyzes user query
   └─> Decides to call Israel Weather tools

3. Tool Execution (Groq orchestrates)
   ├─> Call: open_weather_forecast_israel()
   │   └─> Result: Browser navigates to weather2day.co.il
   │
   ├─> Call: enter_weather_forecast_city_israel(city="Jerusalem")
   │   └─> Result: City name entered, suggestions available
   │
   ├─> Call: select_weather_forecast_city_israel()
   │   └─> Result: Page loads with forecast
   │
   └─> Call: get_weather_forecast_content_israel()
       └─> Result: Raw Weather2Day content
           ├─> Temperatures: 31.7°C (Jerusalem)
           ├─> Forecast: "Partly cloudy to clear"
           ├─> Alerts: Thunderstorm warnings
           └─> Tomorrow: "Partly cloudy, slight temp drop"

4. LLM Final Answer
   └─> "The forecast in Jerusalem shows partly cloudy weather 
        with a high of 31.7°C. There's a risk of isolated 
        thunderstorms in eastern areas. Tonight will be partly cloudy..."
```

### Groq's Role
- **Understands context** from system prompt (Hebrew instructions)
- **Selects appropriate tools** (Israel vs USA weather tools)
- **Orchestrates tool sequence** (open → enter → select → extract)
- **Processes extracted data** and generates natural language response
- **Implements reasoning** to handle follow-up questions

## Key Design Decisions

### 1. Global Browser State
**Why:** Playwright browser instances are expensive to create. Global `browser` and `page` variables persist across tool calls, enabling the sequential workflow (open → enter → select → extract).

**Alternative:** Creating new browser instances per tool would be ~10-20x slower.

### 2. Multiple Selector Strategies
**Why:** Weather2Day's HTML structure is dynamic and selectors change with page updates. Multiple fallback strategies ensure reliability.

**Example:**
```python
search_selectors = [
    "#city_search_forecast",           # Try ID first
    "input[placeholder*='עיר']",      # Try Hebrew placeholder
    "input[type='search']",            # Generic search input
    "input",                           # Last resort
]
```

### 3. Groq Over OpenAI
**Why:**
- Groq's inference is 100-200x faster than OpenAI
- 10x cheaper for equivalent performance
- Ideal for MCP tool orchestration where latency matters
- llama-3.1-8b-instant is sufficient for weather context

### 4. Playwright Over Selenium
**Why:**
- Native async/await support (non-blocking)
- Better event-driven architecture
- Modern browser support (Chromium, Firefox, WebKit)
- Simpler API for DOM interaction

### 5. SSL Verification Disabled
**Why:** Corporate and school networks (e.g., Netfree in Israel) use intercepting proxies that break standard SSL verification. This is a known issue and `verify=False` is the standard workaround in such environments.

**Security Note:** This is safe for our use case (public APIs, no credential exchange in transit) and appropriate for enterprise deployments with internal CAs.

## Troubleshooting

### "GROQ_API_KEY not found"
```
Error: RuntimeError: GROQ_API_KEY not found in environment...
```
**Solution:** Create `.env` file with your Groq key:
```bash
cp .env.example .env
# Edit .env and add your GROQ_API_KEY
```

### "Playwright not installed"
```
Error: TimeoutError: Timeout opening Weather2Day
```
**Solution:** Install Chromium:
```bash
playwright install chromium
```

### "Connection refused" on other ports
If you get connection errors, the MCP server might be using a different stdio configuration. Ensure `host.py` and MCP servers are using stdio transport (default).

### Weather2Day page structure changed
If selectors stop working:
1. The extraction tool uses multiple fallback selectors
2. If all fail, review `weather_Israel.py` lines 240-260
3. Update selectors based on actual page HTML (use browser DevTools)

### Timeout errors for certain cities
Some cities (Tel Aviv) occasionally timeout due to page rendering delays. Retry or use different cities. This is a known limitation of browser automation on dynamic pages.

## Performance Characteristics

| Operation | Time | Notes |
|-----------|------|-------|
| Browser launch | 2-3s | One-time, cached |
| Page navigation | 3-5s | Network dependent |
| City selection | 1-2s | DOM interaction |
| Content extraction | 0.5-1s | DOM traversal |
| Groq inference | 1-3s | LLM processing |
| **Total end-to-end** | 8-15s | Varies by network |

## Limitations

### Current
1. **One city at a time** - Each query handles single city (by design)
2. **Hebrew-optimized** - System prompt is in Hebrew; English queries work but less optimized
3. **Dynamic page timing** - Some cities have page rendering delays causing timeouts
4. **Static browser session** - Playwright browser persists; restart required to clear state

### Not Implemented
- Historical weather data
- Detailed NOAA integration for USA
- Multi-language support (Hebrew/English only)
- Caching or rate limiting
- User session management

## Contributing

### To Add Support for New Cities
No changes needed - Weather2Day covers 60+ Israeli cities automatically.

### To Add New Data Sources
1. Create new MCP server (e.g., `weather_external_source.py`)
2. Register in `host.py` mcp_clients list
3. Add tools with `@mcp.tool()` decorator
4. Update system prompt if needed

### To Change LLM Provider
1. Replace Groq with any OpenAI-compatible API
2. Change `self.groq.chat.completions.create()` in host.py
3. Ensure tool format is maintained (OpenAI schema)

## License

This project is provided as-is for educational purposes.

## References

- **Groq**: https://console.groq.com
- **MCP Specification**: https://modelcontextprotocol.io
- **Playwright Documentation**: https://playwright.dev
- **Weather2Day**: https://www.weather2day.co.il
- **NWS API**: https://www.weather.gov/documentation/services-web-api

---

**Last Updated:** 2026-08-16  
**Status:** Production-Ready ✅