Skip to main content
Glama

LLM inference API, large model: Llama 3.3 70B or gpt-oss-120b

llm_inference
Read-only

LLM inference with a large open model: send a prompt (optional system prompt, up to 1,024 output tokens) and get an answer from Llama 3.3 70B Instruct or OpenAI gpt-oss-120b in one call, with no provider account; OpenAI-compatible at /v1/chat/completions by model name. Use when a small model is not enough: reasoning, drafting, analysis and Q&A inside an agent workflow. Example: prompt=Explain x402 in one sentence.. Price $0.01/call via x402 USDC or a card key (/credits); failed calls free. Reads or computes only and changes nothing; each call is charged, so repeats are not free.

Input Schema

NameRequiredDescriptionDefault
modelNollama-3.3-70b (default) or gpt-oss-120b
promptYesUser prompt, up to 8,000 characters
systemNoOptional system prompt, up to 2,000 characters
maxTokensNoMaximum output tokens, 1-1024 (default 400)
temperatureNo0-1.5 (default 0.6)

Schema Changelog

Changes observed during successful MCP inspections.

  1. Added

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly=true, destructive=false, openWorld=true, and idempotent=false; the description reinforces this with 'reads or computes only and changes nothing' and explains that repeats are billed, which explains the non-idempotent hint. It adds genuinely useful operational context absent from the annotations: $0.01/call via x402 USDC or a card key, failed calls free, no provider account needed, and the OpenAI-compatible route. It does not describe the response payload shape, which matters slightly since there is no output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The key facts (models, cost, when to use) are front-loaded and the whole thing is a single dense block, but there is redundancy: 'Reads or computes only and changes nothing' restates readOnlyHint, and the example 'prompt=Explain x402 in one sentence..' carries a stray double period and little illustrative value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a five-parameter, single-call inference tool with no output schema, the description covers model choice, cost, payment paths, length limits, and safety profile adequately. The main residual gap is the return value shape (plain text answer vs. structured chat completion), which is only implied by 'get an answer'.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so all five parameters already carry descriptions and enum values for model. The description only echoes the token cap ('up to 1,024 output tokens') and the optional system prompt, adding no format or tuning guidance beyond the schema. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a concrete verb and resource ('LLM inference with a large open model') and names the two exact models (Llama 3.3 70B Instruct, gpt-oss-120b) plus the compatible endpoint. It does not differentiate itself from close siblings such as ai_model_gpt_oss_120b, ai_chat_large, or ai_model_deepseek_r1, which an agent could easily confuse it with.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Use when a small model is not enough: reasoning, drafting, analysis and Q&A inside an agent workflow' gives a clear selection condition relative to small-model tools. It stops short of naming the alternative tools or stating when not to use it (e.g., when a sibling pinned to one specific model is preferable).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources