README.md•4.26 kB
# Content Fetcher MCP
This MCP server fetches content from various sources including YouTube, RSS feeds, and GitHub releases. It's designed to work with Goose to help track and identify new content.
## Features
- **YouTube**: Fetches videos from the Goose YouTube channel
- **RSS Feeds**: Fetches blog posts from any RSS feed (including the Goose blog)
- **GitHub Releases**: Fetches releases from the Block/Goose repository
- **Content Tracking**: Tracks seen content to identify new items
- **Cross-machine Persistence**: Stores tracking data in `~/.config/goose/content-fetcher-mcp/`
## Setup
1. Ensure you have Node.js installed (version 14 or higher recommended).
2. Install dependencies:
```bash
npm install
```
3. Build the project:
```bash
npm run build
```
## Running the MCP Server
This is an MCP server that uses stdio transport and is designed to be registered with Goose.
To start the server directly:
```bash
npm start
```
For development with auto-reload:
```bash
npm run dev
```
## Registering with Goose
To register this MCP server with Goose, add it to your Goose configuration. The server uses stdio transport, so it should be configured as a local MCP server in your Goose settings.
## Available Tools
### 1. fetchYoutube
Fetches ALL videos from the Goose YouTube channel.
**Parameters:** None
**Returns:** Array of video objects with `id`, `title`, `url`, `published_at`, and `type: "video"`
### 2. fetchRss
Fetches ALL blog posts from any RSS feed.
**Parameters:**
- `url` (string): RSS feed URL
**Returns:** Array of blog post objects with `id`, `title`, `url`, `published_at`, and `type: "blog"`
### 3. fetchGooseBlog
Fetches ALL blog posts from the official Goose blog.
**Parameters:** None
**Returns:** Array of blog post objects with `id`, `title`, `url`, `published_at`, and `type: "blog"`
### 4. fetchGithubReleases
Fetches ALL releases from the Block/Goose GitHub repository.
**Parameters:** None
**Returns:** Array of release objects with `id`, `title`, `url`, `published_at`, and `type: "release"`
### 5. isNewContent
Checks if a content item has been seen before.
**Parameters:**
- `id` (string): Unique identifier for the content
- `type` (enum): One of `"youtube"`, `"blog"`, or `"release"`
**Returns:** `{ "is_new": true/false }`
### 6. markContentSeen
Marks a content item as seen (typically after posting).
**Parameters:**
- `id` (string): Unique identifier for the content
- `type` (enum): One of `"youtube"`, `"blog"`, or `"release"`
**Returns:** `{ "success": true }`
## How It Works
1. **Fetching**: The fetch tools retrieve all available content from their respective sources
2. **Filtering**: Use `isNewContent` to check if an item hasn't been seen before
3. **Tracking**: After processing new content, use `markContentSeen` to mark it as seen
4. **Persistence**: Seen content is stored in `~/.config/goose/content-fetcher-mcp/last_seen.json`
## Example Workflow
```javascript
// 1. Fetch all YouTube videos
const videos = await fetchYoutube();
// 2. Check which ones are new
for (const video of videos) {
const result = await isNewContent({ id: video.id, type: "youtube" });
if (result.is_new) {
// Process the new video...
// 3. Mark as seen after processing
await markContentSeen({ id: video.id, type: "youtube" });
}
}
```
## Configuration
The server is pre-configured with:
- **YouTube Channel**: Goose channel (`UCVLuT_AS687XAJ__-COCRFw`)
- **Goose Blog RSS**: `https://block.github.io/goose/blog/rss.xml`
- **GitHub Repository**: `block/goose`
To customize these, edit the constants in `src/server.ts`.
## Notes
- The server uses stdio transport, making it suitable for local MCP integration with Goose
- Content tracking is persistent across restarts via the `last_seen.json` file
- All fetch operations return the complete list of content; filtering for "new" items is done separately via `isNewContent`
## Future Improvements
1. Add configuration file support for customizing channels, feeds, and repositories
2. Implement rate limiting and caching to optimize API usage
3. Add more detailed logging and error handling
4. Support for additional content sources
5. Batch operations for checking multiple items at once