The Path of a Request
A user sees a single request; Ollama sees a sequence of decisions about identity, capability, prompt form, resources, and response state. This chapter follows that sequence so the reader can tell what Ollama guarantees at its API boundary, what the model contributes, and what the calling application still owns. The result is a practical way to trace a generate or chat call instead of treating every success or failure as an unexplained property of “the AI.”
First, Ollama decides what kind of work this is
The public request begins with a model reference and either a prompt or a message sequence. Ollama exposes separate generate and chat paths. Generate accepts a prompt and can support insertion with a suffix; chat accepts messages and can carry conversation history, images, tools, thinking fields, and tool results. Both paths can stream responses, accept runtime options, and control how long the model remains loaded. The distinction is not just a naming preference: it tells Ollama what kind of input it must prepare and what response shape it should return.
The server first parses the request and resolves the model reference. A short name may be normalized to the local model identity described by the manifest. If the name points to a cloud model, the request follows a remote path instead of local inference. For a local model, Ollama loads the artifact’s configuration and metadata, including its model family, template information, parser or renderer choices, and discovered capabilities. The server can then reject a request that asks for work the model does not advertise. A vision request needs image support; an embedding request needs an embedding model; a tool-using chat needs tool support; a request for thinking needs a thinking-capable model.
This check is an executable contract. Without it, the server would treat a request for an image-aware or tool-aware interaction as ordinary text and fail later, possibly after loading a large model. Capability validation moves the failure closer to the cause and prevents a client from assuming that every model exposes the same operations. It also explains why model names alone are insufficient. Two models may both answer text prompts while differing in vision, tools, thinking, insertion, or embedding behavior.
Next, Ollama consolidates options. It starts with defaults, applies generation defaults and model parameters, and then applies request-time overrides. A client can change settings such as temperature, token limits, or context-related options for a particular call. The server also tracks whether some runner values came from automatic defaults, because later scheduling decisions may need to adjust them if memory is tight. This separation lets a request specialize a model without silently rewriting the model artifact, while still giving the scheduler enough information to know which choices are negotiable.
Only after those checks does Ollama ask the scheduler for a runner. If the requested model is already loaded with compatible runner settings, the scheduler can reuse it. Otherwise, the request enters the model-loading path. The API handler does not need to know whether the native runner will use CPU memory, one GPU, several GPUs, or a particular acceleration library. It waits for the scheduler to return a ready runner or an error. That is the point where the request path meets the resource path.
The sequence separates control ownership. The client initiates the request and, when needed, maintains history and executes tools. Ollama resolves the model, enforces its capability contract, obtains a runner, prepares the input, and translates the runner’s response into the public protocol. The native runner performs inference and reports stream events. A cloud route changes the runner boundary, but the reason for showing the sequence remains the same: each transition has an owner and a failure condition.
Each gate can fail for a different reason, and that ordering is part of the design. If model resolution fails, there is no reason to allocate a runner. If capability validation fails, a request can be rejected before a large model is loaded. If a runner is healthy but its required context or adapter settings differ from the loaded instance, the scheduler may need a new runner rather than reusing the old one. If prompt preparation fails, the model may never receive a generation request at all. “The model did not answer” is therefore too imprecise to guide a diagnosis.
Model resolution connects the public name to local state. The server normalizes the reference, looks for the existing model identity, and reads its configuration. A missing model produces a model error rather than a generation result. A model that exists but lacks the requested capability produces a different kind of boundary failure. This is useful for clients because the remedy differs: one case concerns acquiring or naming an artifact, while the other concerns selecting a model that supports the requested operation. Treating both as generic inference failures hides the decision the caller needs to make.
Runner reuse has its own condition. Ollama can return an already loaded runner when the model, adapter and projector paths, runner options, context behavior, and health state remain compatible. A request that changes a load-time option may require a reload even if the model name is unchanged. The client still sees the same high-level API, but the next response may include a different load cost or wait for a resource transition. This is why a model name is a stable reference, not a promise that every request uses the same process state.
Prompt preparation is another boundary where model-specific rules become visible. A chat request may include the model’s stored messages and system prompt, while a generate request may construct a user message and apply a template. A raw request bypasses that formatting only under the conditions the API permits. Images require data conversion and placement into the model’s expected input form. Thinking and tool parsers may need to separate fields that arrive in one native stream. If the template, parser, image data, or message shape is wrong, changing the sampling temperature will not fix the problem; the input contract must be corrected.
Even an empty request has a defined meaning in Ollama. An empty prompt or empty message list can ask the server to load a model without generating text. Pairing that request with a zero keep-alive can ask it to unload the model. This makes lifecycle control available through the same API boundary, but it also means that an empty response is not automatically evidence of a failed model. The client must read the completion or stop reason and the surrounding request fields before interpreting the result.
Imagine an application asking a vision-capable chat model to inspect an image and return a structured record. The request must name an artifact with vision and completion support, pass image data in an accepted form, render the correct message convention, and ask for a schema the client can validate. If the model emits a tool call instead, the application must execute the relevant function and send back the result before a final answer can be produced. At every step, Ollama can transport and constrain the exchange, but the application remains responsible for supplying valid input, handling the stream, executing side effects, and deciding whether the returned record is usable.
This layered failure model is also why the API can remain broadly compatible while preserving project-specific behavior. The edge translates a familiar client request into Ollama’s internal request objects, but model capability checks, templates, runner state, and application responsibilities still govern what happens next. A successful translation at the protocol edge only means that the request entered Ollama’s path; it does not mean that the selected model can fulfill every requested feature.
One API boundary, several response contracts
After a runner is ready, Ollama still has to decide how the model should see the request. In a generate flow, a non-raw request may be assembled from a system message, model-embedded messages, the user prompt, optional images, and the model’s configured template. A suffix uses a different insertion-shaped value. A caller can explicitly provide a template, while a raw request bypasses normal formatting. These paths exist because models are trained around different message conventions. The runtime can make the choice repeatable, but it cannot make every template interchangeable.
Chat makes the state more explicit. The request supplies messages, and the model may contribute a stored system message or embedded example history. Ollama combines those messages in the appropriate order, adds a system message when the model has one and the request does not, and then filters or prepares thinking-related content when required. For models whose chat handling is native, Ollama prepares a native chat request and lets the runner apply the model’s chat template. For other models, it renders or parses the interaction in the Ollama layer. The difference is an implementation boundary with user-visible consequences: the selected route must match the model’s format and parser capabilities.
Images cross the same boundary as data rather than as a special kind of prompt prose. The REST API accepts image data in the form documented for the endpoint, while SDKs can accept paths or bytes and prepare the request. Ollama assigns stable image markers during prompt preparation and translates them for the native runner. A multimodal model can then combine image and text input, but only if the artifact carries the required projector or vision capability and the selected native path can load it. Image support is therefore a relationship among request data, model metadata, projector material, and runner behavior.
Streaming changes the response contract. Several Ollama endpoints stream newline-delimited JSON by default. A client can render partial text as it arrives, reducing perceived waiting time, or turn streaming off and receive one response object. The final streamed object carries completion state and usage information such as load duration, prompt evaluation counts, cached prompt work, and generation counts and duration. Those measurements let an application separate the cost of loading a model from the cost of evaluating a prompt or producing output.
The stream is also a state machine for richer responses. A thinking-capable model can emit a separate thinking field before ordinary content. A tool-capable model can emit a tool call rather than a final answer. The client must accumulate the partial thinking, content, and tool-call fields before constructing the assistant message that will be sent in a follow-up request. If it forwards only the last chunk, it can lose the context needed for the next turn. Streaming is consequently not just a display optimization; it changes how the client preserves conversation state.
Tool calling makes the ownership boundary especially clear. The application describes available tools to Ollama. The model may return a call with a function name and arguments. Ollama transports that structured request, but the application decides whether and how to execute the function. It then appends the assistant’s call and the tool’s result to the message history and sends the conversation back for another model turn. The documentation describes single-shot calls, parallel calls, and a multi-turn agent loop. None of those patterns transfers authority over external side effects to the model or the Ollama server.
Structured output imposes a related but different constraint. The caller can request JSON mode or provide a JSON schema. Ollama passes the format requirement toward the runner so that the generated form can be constrained. This is useful for extracting records, describing images, or keeping application responses consistent. It is not a factuality guarantee. The application should validate the returned JSON against its own data model, and it should still assess whether the values make sense. A syntactically valid object can contain an incorrect answer.
Thinking, vision, tools, and structured output therefore extend the same API boundary in different directions. Thinking separates an intermediate reasoning field from final content for supported models. Vision adds media inputs. Tools add a proposal-and-result loop controlled by the application. Schemas constrain output form. Embeddings leave the conversational path and return numeric vectors for search or retrieval. The runtime exposes these as related capabilities, but each one has a distinct state transition and a distinct limit. A model that supports vision need not support tools; a model that returns embeddings is not necessarily a completion model.
The same request can be reached through compatibility surfaces. Ollama’s OpenAI-compatible endpoints let an existing client point its base URL at the local service and use supported chat, completion, streaming, JSON, vision, tool, reasoning, and embedding features. The non-stateful Responses API supports streaming and tools but does not provide every stateful conversation feature. The Anthropic-compatible Messages API supports messages, streaming, system prompts, multi-turn conversations, vision, tools, tool results, and thinking, which is why tools such as Claude Code can use Ollama as a backend.
Compatibility is an adapter, not a promise of protocol identity. The documentation lists unsupported or partial features: OpenAI logprobs and several request fields remain unavailable; Anthropic tool choice, metadata, prompt caching, batch processing, citations, PDF blocks, URL images, and some stateful behavior are not supported in the documented surface. An API key may be accepted and ignored for local compatibility, while cloud access has separate authentication. An existing application can therefore migrate with little code in one scenario and still need changes when it depends on a missing semantic feature.
A useful way to read any response is to ask three questions. What did the model generate? What did Ollama add as transport, formatting, capability enforcement, or metrics? What did the application have to preserve, validate, or execute? This prevents a streamed JSON object from being mistaken for a complete agent protocol, a tool call from being mistaken for a completed action, or a compatible endpoint from being mistaken for an identical service.
The request path is now visible from the outside: identity and capability determine the allowed route; templates or native chat determine the input; the runner produces a stream; the API translates it; and the client owns the state needed for the next turn. But one question remains underneath every route. A runner can be reused only if its settings and resources remain compatible, and a new runner can start only if the model fits the available memory. The next boundary is the scheduler that decides whether inference can happen now, where it should happen, and what must wait or be unloaded.