stream_large_file

Process large files efficiently by streaming them in manageable chunks without loading entire files into memory, enabling step-by-step handling of oversized data.

Instructions

Stream a large file in chunks. Returns multiple chunks for processing very large files efficiently.

Input Schema

TableJSON Schema

Name	Required	Description
`filePath`	Yes	Absolute path to the file
`chunkSize`	No	Chunk size in bytes (default: 65536 - 64KB)
`startOffset`	No	Starting byte offset (default: 0)
`maxBytes`	No	Maximum bytes to stream (optional)
`maxChunks`	No	Maximum number of chunks to return (default: 10)

Implementation Reference

src/server.ts:385-421 (handler)

The primary handler function for the 'stream_large_file' tool. It extracts parameters from arguments, uses FileHandler.streamFile to generate chunks, limits to maxChunks, and formats the response as MCP tool content.

private async handleStreamFile(
  args: Record<string, unknown>
): Promise<{ content: Array<{ type: string; text: string }> }> {
  const filePath = args.filePath as string;
  const chunkSize = (args.chunkSize as number) || 64 * 1024;
  const startOffset = args.startOffset as number | undefined;
  const maxBytes = args.maxBytes as number | undefined;
  const maxChunks = (args.maxChunks as number) || 10;

  const chunks: string[] = [];
  let chunkCount = 0;

  for await (const chunk of FileHandler.streamFile(filePath, {
    chunkSize,
    startOffset,
    maxBytes,
  })) {
    chunks.push(chunk);
    chunkCount++;
    if (chunkCount >= maxChunks) break;
  }

  return {
    content: [
      {
        type: 'text',
        text: JSON.stringify({
          totalChunks: chunks.length,
          chunks,
          note: chunks.length >= maxChunks
            ? 'Reached maxChunks limit. Increase maxChunks or use startOffset to continue.'
            : 'All chunks returned.',
        }, null, 2),
      },
    ],
  };
}

src/server.ts:212-241 (schema)

The input schema and metadata for the 'stream_large_file' tool, defining parameters, descriptions, and requirements. This is returned by getTools() for tool discovery.

{
  name: 'stream_large_file',
  description: 'Stream a large file in chunks. Returns multiple chunks for processing very large files efficiently.',
  inputSchema: {
    type: 'object',
    properties: {
      filePath: {
        type: 'string',
        description: 'Absolute path to the file',
      },
      chunkSize: {
        type: 'number',
        description: 'Chunk size in bytes (default: 65536 - 64KB)',
      },
      startOffset: {
        type: 'number',
        description: 'Starting byte offset (default: 0)',
      },
      maxBytes: {
        type: 'number',
        description: 'Maximum bytes to stream (optional)',
      },
      maxChunks: {
        type: 'number',
        description: 'Maximum number of chunks to return (default: 10)',
      },
    },
    required: ['filePath'],
  },
},

src/server.ts:260-261 (registration)
The switch case in handleToolCall that routes calls to the 'stream_large_file' tool to its handler function.
```
case 'stream_large_file':
  return this.handleStreamFile(args);
```

src/fileHandler.ts:515-536 (helper)

The core streaming utility in FileHandler class. Creates a Node.js ReadStream with configurable chunk size, offset, and limits, yielding chunks as an async generator.

static async *streamFile(
  filePath: string,
  options: StreamOptions = {}
): AsyncGenerator<string> {
  await this.verifyFile(filePath);

  const chunkSize = options.chunkSize || 64 * 1024; // 64KB default
  const encoding = options.encoding || 'utf-8';

  const stream = fs.createReadStream(filePath, {
    encoding,
    start: options.startOffset,
    end: options.maxBytes ?
      (options.startOffset || 0) + options.maxBytes :
      undefined,
    highWaterMark: chunkSize,
  });

  for await (const chunk of stream) {
    yield chunk;
  }
}

Large File MCP Server