When a Live Reply Fails

A NextChat reply can be correct at the provider and still look broken to the user. This chapter follows the live response from request start through server-sent events, client animation, completion, cancellation, and error handling, then uses that lifecycle to separate buffering, authentication, model policy, endpoint, and upstream causes. Readers will be able to diagnose a slow, incomplete, or absent reply by asking which boundary owns the failed transition instead of treating every symptom as a model problem.

The live response has a state machine

A streamed reply begins when the client creates an abort controller and asks the selected provider adapter to chat with streaming enabled. The controller is associated with the session and assistant message, so the user can stop one response or stop all pending responses. A request timeout provides another termination path. These controls exist before any response chunk arrives; they define the conditions under which the client will keep waiting.

The request then opens through an event-stream client. The first response check is structural. The client records the response, inspects its content type and status, and distinguishes an ordinary text response or an error response from the event-stream format it expects. A successful status alone does not prove that streaming is working. The body must also be exposed as the expected stream, and the intermediate route must preserve it.

Once the stream is accepted, each event is parsed according to the selected provider’s response format. Content fragments are placed in a temporary buffer. An animation loop moves that buffer into the visible assistant message in small pieces, so the interface remains readable even when the provider emits uneven chunks. The text the user sees is therefore produced by two rhythms: the provider’s network events and the client’s display loop. They can differ without changing the underlying answer.

The response finishes when the provider sends its end marker, the connection closes, or the client’s finish logic determines that no more work remains. The assistant message stops being marked as streaming, its content and date are stored, and the controller is removed from the pending pool. A clean finish is not limited to one exact event: the client also handles a close or a plain response as a terminal condition. This flexibility helps with providers that do not end their responses in precisely the same way, but it means the client must distinguish a legitimate close from an unexpectedly empty or malformed result.

stateDiagram-v2 [*] --> Started Started --> Streaming: valid event stream Started --> Error: non-streaming or non-200 Streaming --> Streaming: chunk parsed Streaming --> ToolPending: tool calls accumulated ToolPending --> Streaming: results appended and request restarts Streaming --> Complete: end marker or close Streaming --> Aborted: user stop or timeout Streaming --> Error: upstream or parse failure Complete --> [*] Aborted --> [*] Error --> [*]

The important states are not merely visual. Started owns the request timer and the controller. Streaming owns partial text and the connection. ToolPending temporarily stops ordinary text accumulation while the client executes the accumulated calls and prepares another provider request. Complete, Aborted, and Error determine how the message is marked and whether the pending controller can be discarded.