Skip to main content
Glama

ingestWebsite

Crawls and ingests website content recursively for knowledge base management, with configurable depth, path filtering, and chunking controls.

Instructions

Crawls and ingests content from a website recursively. Supports depth control and path filtering.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
namespaceIdNo
ingestConfigYes
tenantIdNo

Implementation Reference

  • src/index.ts:291-308 (registration)
    MCP tool registration for 'ingestWebsite' using server.tool, including inline handler that delegates to SourceSyncApiClient.ingestWebsite
    server.tool( 'ingestWebsite', 'Crawls and ingests content from a website recursively. Supports depth control and path filtering.', IngestWebsiteSchema.shape, async (params) => { return safeApiCall(async () => { const { namespaceId, ingestConfig, tenantId } = params // Create a client with the provided parameters const client = createClient({ namespaceId, tenantId }) // Direct passthrough to the API return await client.ingestWebsite({ ingestConfig, }) }) }, )
  • Core handler logic in SourceSyncApiClient.ingestWebsite that sends POST request to SourceSync API /v1/ingest/website endpoint
    public async ingestWebsite({ ingestConfig, }: Omit< SourceSyncIngestWebsiteRequest, 'namespaceId' >): Promise<SourceSyncIngestResponse> { return this.client .url('/v1/ingest/website') .json({ namespaceId: this.namespaceId, ingestConfig: { ...ingestConfig, chunkConfig: SourceSyncApiClient.CHUNK_CONFIG, }, } satisfies SourceSyncIngestWebsiteRequest) .post() .json<SourceSyncIngestResponse>() }
  • Zod schema definition for ingestWebsite tool input validation (IngestWebsiteSchema)
    export const IngestWebsiteSchema = z.object({ namespaceId: namespaceIdSchema.optional(), ingestConfig: z.object({ source: z.literal(SourceSyncIngestionSource.WEBSITE), config: z.object({ url: z.string(), maxDepth: z.number().optional(), maxLinks: z.number().optional(), includePaths: z.array(z.string()).optional(), excludePaths: z.array(z.string()).optional(), metadata: z.record(z.union([z.string(), z.array(z.string())])).optional(), }), chunkConfig: chunkConfigSchema.optional(), }), tenantId: tenantIdSchema, })
  • TypeScript type definition for the SourceSync API request (SourceSyncIngestWebsiteRequest) used by ingestWebsite
    export type SourceSyncIngestWebsiteRequest = { namespaceId: string ingestConfig: { source: SourceSyncIngestionSource.WEBSITE config: { url: string maxDepth?: number maxLinks?: number includePaths?: string[] excludePaths?: string[] scrapeOptions?: SourceSyncScrapeOptions metadata?: Record<string, any> } chunkConfig?: SourceSyncChunkConfig } }

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/pbteja1998/sourcesyncai-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server