fetchUrlContent
Extract content from web URLs to retrieve parsed text files for knowledge management and content ingestion.
Instructions
Fetches the content of a URL. Particularly useful for fetching parsed text file URLs.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| apiKey | No | ||
| tenantId | No |
Implementation Reference
- src/index.ts:739-764 (handler)The handler function that fetches the content from the provided URL using the wretch library. Supports optional API key authentication and tenant ID header. Wraps the fetch in safeApiCall for consistent error handling.
async (params: FetchUrlContentParams) => { return safeApiCall(async () => { const { url, apiKey, tenantId } = params try { // Create a wretch client with authentication if provided let client = wretch(url) if (apiKey) { client = client.auth(`Bearer ${apiKey}`) } if (tenantId) { client = client.headers({ 'X-Tenant-ID': tenantId, }) } // Fetch the content from the URL const content = await client.get().text() return { content } } catch (error: any) { throw new Error(`Error fetching URL content: ${error.message}`) } }) }, - src/index.ts:735-765 (registration)The server.tool call that registers the 'fetchUrlContent' tool with its description, input schema, and handler function.
server.tool( 'fetchUrlContent', 'Fetches the content of a URL. Particularly useful for fetching parsed text file URLs.', FetchUrlContentSchema.shape, async (params: FetchUrlContentParams) => { return safeApiCall(async () => { const { url, apiKey, tenantId } = params try { // Create a wretch client with authentication if provided let client = wretch(url) if (apiKey) { client = client.auth(`Bearer ${apiKey}`) } if (tenantId) { client = client.headers({ 'X-Tenant-ID': tenantId, }) } // Fetch the content from the URL const content = await client.get().text() return { content } } catch (error: any) { throw new Error(`Error fetching URL content: ${error.message}`) } }) }, ) - src/schemas.ts:502-509 (schema)Zod schema definition for the input parameters of fetchUrlContent: url (required string URL), optional apiKey and tenantId.
export const FetchUrlContentSchema = z.object({ url: z.string().url(), // Authentication might be needed for some SourceSync URLs apiKey: apiKeySchema, tenantId: tenantIdSchema, }) export type FetchUrlContentParams = z.infer<typeof FetchUrlContentSchema>