Skip to main content
Glama
ypollak2

ypollak2/llm-router

by ypollak2
OverviewSchemaRelated ServersScoreDiscussions
Python
Hybrid

llm_stream

Stream AI responses in real-time for long-running tasks, displaying partial output immediately while routing to the appropriate model from 20+ providers.

Instructions

Stream an LLM response for long-running tasks — shows output as it arrives.

Uses the same routing logic as llm_route but streams chunks instead of
waiting for the full response. Ideal for long-form generation, research
summaries, or any task where seeing partial output early is valuable.

Args:
    prompt: The task or question to stream.
    task_type: Task type hint — "query", "research", "generate", "analyze", "code".
    model: Optional model override (e.g. "openai/gpt-4o", "gemini/gemini-2.5-flash").
    system_prompt: Optional system instructions.
    temperature: Sampling temperature (0.0-2.0).
    max_tokens: Maximum output tokens.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
promptYes
task_typeNoquery
modelNo
system_promptNo
temperatureNo
max_tokensNo

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault
resultYes

Tool Definition Quality

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full disclosure burden. It successfully explains the streaming mechanism ('shows output as it arrives', 'streams chunks'), but omits operational details like error handling, retry behavior, rate limits, or side effects (e.g., caching, usage tracking) that would be critical for an LLM tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Well-structured with purpose front-loaded, sibling differentiation in the middle, and parameter details at the end. The Args list is necessary given schema deficiencies. Minor verbosity in phrases like 'or any task where seeing partial output early is valuable' could be tightened.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema exists (per context signals), the description appropriately focuses on inputs and behavior. All 6 parameters are documented. Minor gaps remain around error states and whether streaming affects billing/usage differently than non-streaming siblings, but core functionality is well-covered.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% (titles only). The description fully compensates by documenting all 6 parameters in the Args section, including semantic meaning (e.g., 'task_type hint'), valid ranges ('0.0-2.0' for temperature), and concrete examples ('openai/gpt-4o', 'gemini/gemini-2.5-flash').

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb ('Stream') + resource ('LLM response') combination clearly states function. Explicitly distinguishes from sibling 'llm_route' by contrasting streaming chunks vs. waiting for full response, making selection unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear when-to-use guidance ('Ideal for long-form generation, research summaries, or any task where seeing partial output early is valuable') and implicitly contrasts with llm_route. Lacks explicit 'when not to use' guidance or named alternatives for non-streaming scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Install Server

Other Tools

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/ypollak2/llm-router'

If you have feedback or need assistance with the MCP directory API, please join our Discord server