scrape_channel_full
Extract all posts from a Telegram channel and save them as MD and JSON files using authenticated sessions for comprehensive data collection and storage.
Instructions
Scrape ALL posts from a Telegram channel and save to file. Uses authenticated session if logged in. Returns file location.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| save_to_file | No | Save results to MD and JSON files | |
| url | Yes | The Telegram channel URL (e.g., https://t.me/channelname) |
Implementation Reference
- src/server.ts:478-530 (handler)Main handler function for the 'scrape_channel_full' tool. Scrapes all posts from a Telegram channel using the appropriate scraper (authenticated if available), saves full results to MD and JSON files, and returns a response with file locations and a sample of the first 5 posts.private async handleScrapeChannelFull(args: any): Promise<any> { // Check if authenticated and use authenticated scraper by default const isAuthenticated = await this.auth.isAuthenticated(); const scraperToUse = isAuthenticated ? this.authScraper : this.scraper; if (isAuthenticated) { logger.info('Using authenticated scraper for full channel scrape (logged in)'); } else { logger.info('Using unauthenticated scraper for full channel scrape (not logged in)'); } const options: ScrapeOptions = { url: args.url, maxPosts: 0, // No limit - get ALL posts includeReactions: true }; const result = await scraperToUse.scrape(options); // The scraper already saves to file, so we just need to inform about it const channelName = result.channel.username; const timestamp = new Date().toISOString().replace(/[:.]/g, '-').slice(0, -5); const windowsPath = `C:\\Users\\User\\AppData\\Roaming\\Claude\\telegram_scraped_data\\${channelName}_${timestamp}_full.md`; // Also return a sample of the content for immediate analysis const samplePosts = result.posts.slice(0, 5); // First 5 posts as sample const sampleResult = { ...result, posts: samplePosts }; const sampleMarkdown = this.formatter.format(sampleResult); return { content: [ { type: 'text', text: `Successfully scraped ${result.totalPosts} posts from @${channelName} Files saved to: - Markdown: ${windowsPath} - JSON: ${windowsPath.replace('.md', '.json')} Total posts: ${result.totalPosts} Date range: ${result.posts.length > 0 ? `${result.posts[result.posts.length - 1]?.date.toISOString().split('T')[0]} to ${result.posts[0]?.date.toISOString().split('T')[0]}` : 'N/A'} The full channel history has been saved. Here's a sample of the first 5 posts: ${sampleMarkdown} To analyze all ${result.totalPosts} posts, open the saved markdown file and copy its contents to Claude. ${isAuthenticated ? '✅ Scraped using authenticated session - all content including restricted posts should be accessible.' : '⚠️ Scraped without authentication - some restricted content may not be accessible.'}` } ] }; }
- src/server.ts:147-165 (registration)Tool registration in the getTools() method, defining the name, description, and input schema for 'scrape_channel_full'.{ name: 'scrape_channel_full', description: 'Scrape ALL posts from a Telegram channel and save to file. Uses authenticated session if logged in. Returns file location.', inputSchema: { type: 'object', properties: { url: { type: 'string', description: 'The Telegram channel URL (e.g., https://t.me/channelname)' }, save_to_file: { type: 'boolean', description: 'Save results to MD and JSON files', default: true } }, required: ['url'] } },
- src/server.ts:150-164 (schema)Input schema definition for the 'scrape_channel_full' tool, specifying required 'url' and optional 'save_to_file' parameters.inputSchema: { type: 'object', properties: { url: { type: 'string', description: 'The Telegram channel URL (e.g., https://t.me/channelname)' }, save_to_file: { type: 'boolean', description: 'Save results to MD and JSON files', default: true } }, required: ['url'] }
- src/server.ts:68-69 (handler)Dispatch case in the tool call handler that routes 'scrape_channel_full' calls to the main handler function.case 'scrape_channel_full': return await this.handleScrapeChannelFull(args);