OpenDiscourse MCP

# OpenDiscourse MCP Model Context Protocol (MCP) servers for accessing bulk data from govinfo.gov and congress.gov APIs with comprehensive AI-powered code review and data ingestion capabilities. ## Overview This project provides MCP servers that enable AI assistants to access comprehensive legislative and governmental data from: - **GovInfo API**: Congressional documents, Federal Register, Code of Federal Regulations, and more - **Congress.gov API**: Bills, members, committees, nominations, and legislative activities - **Bulk Data Ingestion**: Automated downloading and processing of govinfo.gov bulk data - **AI Code Review**: Multi-agent AI crews for comprehensive code analysis ## Features ### GovInfo Server Tools - `govinfo_list_collections` - List collections updated since a specific date - `govinfo_get_package` - Get detailed package information - `govinfo_list_granules` - List granules within a package - `govinfo_get_granule` - Get detailed granule information - `govinfo_download_content` - Download content (PDF, XML, HTML) ### Congress.gov Server Tools - `congress_get_bill` - Get detailed bill information - `congress_search_bills` - Search for bills by text query - `congress_get_bill_actions` - Get legislative actions for a bill - `congress_get_bill_text` - Get full text of a bill - `congress_search_members` - Search for members of Congress - `congress_get_member` - Get detailed member information - `congress_get_committee_meetings` - Get committee meetings - `congress_get_nominations` - Get presidential nominations ### Bulk Data Ingestion - **Comprehensive Download**: Automated ingestion of all govinfo.gov bulk data - **Congress Coverage**: 113th to 123rd Congress (2013-2024) - **Document Types**: BILLS, BILLSTATUS, PLAW, STATUTE, FR, CREC - **Parallel Processing**: 50 workers for high-performance downloads - **XML Validation**: Schema-based validation for data integrity - **Skip-on-Existing**: Resumable downloads with intelligent file skipping - **Progress Monitoring**: Real-time progress tracking and comprehensive reporting ### AI Code Review Suite - **Security Review**: Vulnerability assessment and security analysis - **Performance Review**: Bottleneck identification and optimization suggestions - **Quality Review**: Code style, testing, architecture, and accessibility analysis - **Documentation Review**: Documentation quality and completeness assessment - **Multi-Agent**: Specialized AI agents for different review aspects - **Observability**: Langfuse integration for monitoring and tracing ## Installation ### Prerequisites 1. **Node.js and npm** (for MCP server) 2. **Python 3.10+** (for setup scripts) 3. **PostgreSQL** (for database storage) 4. **Bitwarden CLI** (for API key management) ### Quick Start ```bash # Clone the repository git clone https://github.com/cbwinslow/opendiscourse_mcp.git cd opendiscourse_mcp # Install Node.js dependencies npm install # Install Bitwarden CLI npm install -g @bitwarden/cli # Set up database ./scripts/setup_database.sh # Set up API keys with Bitwarden python3 scripts/setup_bitwarden.py # Start MCP server npm start ``` ### Detailed Setup #### 1. Python Environment ```bash # Verify Python version (requires 3.10+) python --version # Using uv (recommended) curl -LsSf https://astral.sh/uv/install.sh | sh # Or using pip python3.10 -m venv venv source venv/bin/activate pip install -r requirements.txt ``` #### 2. Database Setup ```bash # Automated database setup ./scripts/setup_database.sh # Manual setup (if script fails) # Start PostgreSQL brew services start postgresql # macOS sudo systemctl start postgresql # Linux # Create database and user sudo -u postgres createdb opendiscourse sudo -u postgres createuser -s opendiscourse sudo -u postgres psql -c "ALTER USER opendiscourse PASSWORD 'opendiscourse123';" sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE opendiscourse TO opendiscourse;" ``` #### 3. Bitwarden Setup ```bash # Install Bitwarden CLI npm install -g @bitwarden/cli # Test Bitwarden integration python3 scripts/test_bitwarden.py # Add API keys to Bitwarden: # - GovInfo API Key # - Congress.gov API Key ``` #### 4. API Key Configuration ```bash # Set environment variables (optional) export BITWARDEN_EMAIL="your-email@example.com" export BITWARDEN_PASSWORD="your-master-password" # Run automated setup python3 scripts/setup_bitwarden.py ``` #### 5. MCP Server Deployment ```bash # Install dependencies uv sync # Start server npm start # Development mode npm run dev ``` **Bitwarden Item Setup:** Create these items in your Bitwarden vault: - **GovInfo API Key** - Name: `GovInfo API Key` - Custom Field: `api_key` with your GovInfo API key - **Congress.gov API Key** - Name: `Congress.gov API Key` - Custom Field: `api_key` with your Congress.gov API key ## Configuration ### Required API Keys 1. **GovInfo API Key** - Sign up at: <https://api.data.gov/signup> - Add to `.env`: `GOVINFO_API_KEY=your_key_here` 2. **Congress.gov API Key** - Sign up at: <https://api.congress.gov/sign-up/> - Add to `.env`: `CONGRESS_API_KEY=your_key_here` ### Optional Configuration - `MCP_LOG_LEVEL` - Logging level (default: info) - `GOVINFO_RATE_LIMIT` - GovInfo rate limit requests/hour (default: 1000) - `CONGRESS_RATE_LIMIT` - Congress.gov rate limit requests/hour (default: 4000) ## Usage ### Bulk Data Ingestion The project includes comprehensive bulk data ingestion capabilities for govinfo.gov: ```bash # Run comprehensive ingestion for all available data python3 scripts/run_comprehensive_ingestion.py # Run specific congress and document type python3 -m scripts.ingest_all_govinfo --congress 118 --doc-types BILLS --workers 50 # Run with validation enabled python3 -m scripts.ingest_all_govinfo --congress 118 --doc-types STATUTE --validate-xml --workers 50 ``` **Ingestion Features:** - **Congress Coverage**: 113th-123rd Congress (2013-2024) - **Document Types**: BILLS, BILLSTATUS, PLAW, STATUTE, FR, CREC - **Parallel Downloads**: 50 concurrent workers - **XML Validation**: Schema-based validation with XSD files - **Resume Capability**: Skip-on-existing functionality - **Progress Tracking**: Real-time monitoring and JSON results **Configuration:** ```python # Use enums for type-safe configuration from scripts.ingestion.enums import DocumentType, CongressSession # Available document types DocumentType.BILLS.value # "BILLS" DocumentType.STATUTE.value # "STATUTE" # Available congress sessions CongressSession.get_available_sessions() # [113, 114, ..., 123] ``` ### AI Code Review The project includes specialized AI crews for comprehensive code review: ```bash # Run all review types (via GitHub Actions) # Or manually trigger specific reviews: # Security review python3 scripts/security_review_crew.py # Performance review python3 scripts/performance_review_crew.py # Quality review (style, testing, architecture, dependencies, accessibility) python3 scripts/quality_review_crew.py # Documentation review python3 scripts/documentation_review_crew.py # Original comprehensive review python3 scripts/code_review_crew.py ``` **AI Review Features:** - **Security Analysis**: Vulnerability assessment, security best practices - **Performance Optimization**: Bottleneck identification, efficiency improvements - **Quality Assessment**: Code style, testing coverage, architecture review - **Documentation Review**: API docs, README, inline comments - **Multi-Agent**: Specialized AI agents for different review aspects - **Observability**: Langfuse integration for monitoring and tracing **GitHub Actions Integration:** - Automatic PR reviews - Configurable review types (security, performance, quality, documentation) - Python 3.10 enforcement - Comprehensive reporting ### As MCP Server ```bash # Start MCP server npm start # Or in development mode with auto-restart npm run dev ``` ### Integration with Claude Desktop Add to your Claude Desktop configuration (`~/Library/Application Support/Claude/claude_desktop_config.json`): ```json { "mcpServers": { "opendiscourse": { "command": "node", "args": ["/path/to/opendiscourse_mcp/src/index.js"], "env": { "GOVINFO_API_KEY": "your_govinfo_key", "CONGRESS_API_KEY": "your_congress_key" } } } } ``` ### API Key Management The project includes automated API key management through Bitwarden: ```bash # Test Bitwarden integration python3 scripts/test_bitwarden.py # Set up API keys from Bitwarden python3 scripts/setup_bitwarden.py ``` **Features:** - 🔍 Automatic API key discovery from Bitwarden vault - 🔓 Secure vault unlocking - 📝 Environment file updates - 🗄️ Database storage with encryption - 🔐 Security masking in console output ### Integration with Claude Desktop Add to your Claude Desktop configuration (`~/Library/Application Support/Claude/claude_desktop_config.json`): ```json { "mcpServers": { "opendiscourse": { "command": "node", "args": ["/path/to/opendiscourse_mcp/src/index.js"], "env": { "GOVINFO_API_KEY": "your_govinfo_key", "CONGRESS_API_KEY": "your_congress_key" } } } } ``` ## API Coverage ### GovInfo Collections - **BILLS** - Congressional bills (all versions) - **CREC** - Congressional Record - **FR** - Federal Register - **CFR** - Code of Federal Regulations - **USCOURTS** - U.S. Courts opinions - **CHRG** - Committee reports and prints ### Congress.gov Data - Bills and amendments (93rd Congress to present) - Member information and voting records - Committee data and meetings - Presidential nominations - Roll call votes - Legislative text and summaries ## Rate Limits - **GovInfo API**: Follows api.data.gov limits (configurable, default 1000/hour) - **Congress.gov API**: 5,000 requests/hour (configurable, default 4000/hour) Both clients implement automatic rate limiting with delays between requests. ## Development ```bash # Run tests npm test # Lint code npm run lint # Full build process npm run build ``` ## Documentation Comprehensive documentation is available in the `/docs` folder: - `docs/README.md` - Complete reference guide - `docs/govinfo-api-summary.md` - GovInfo API quick reference - `docs/congress-api-summary.md` - Congress.gov API quick reference - Downloaded official API repositories and samples ## Contributing 1. Fork the repository 2. Create a feature branch 3. Make your changes 4. Add tests if applicable 5. Run `npm run build` 6. Submit a pull request ## License MIT License - see LICENSE file for details. ## Support - Issues: <https://github.com/cbwinslow/opendiscourse_mcp/issues> - GovInfo API: <https://github.com/usgpo/api/issues> - Congress.gov API: <https://github.com/LibraryOfCongress/api.congress.gov/issues>