From Prompt to Pixels: One Generation Through the Graph

This chapter follows one checkpoint-based image-generation path through ComfyUI, from model components and text to a saved image. The focus is not a catalog of nodes; it is the relationship among the values they exchange. By the end, the reader should be able to distinguish a model from its text encoder, conditioning from a latent, sampling from decoding, and a visual preview from a durable output—and use those distinctions to predict what a change in the graph can affect.

The model does not do one job

A checkpoint can look like one selectable item in a workflow, but the built-in loader exposes several roles from it. ComfyUI's simple checkpoint loader returns a diffusion model, a CLIP text-encoding component, and a VAE. They travel through different branches of the graph because they participate in different operations. Treating them as one undifferentiated “model” makes the workflow harder to reason about: a change to the text encoder is not the same kind of change as a change to the denoiser, and neither is the same as changing the pixel-to-latent converter.

The diffusion MODEL is the component that performs the denoising work during sampling. It receives a latent representation together with conditioning and predicts how that representation should be transformed through the selected denoising process. The CLIP component has a different role. It tokenizes text and encodes it into a representation that later conditioning logic can use to guide the diffusion model. The VAE works at the representation boundary: it encodes pixels into latents for image-to-image paths and decodes latents back into pixels for viewing or saving.

This separation is more than a tidy diagram. It creates meaningful places for a person to change a workflow. Replacing the diffusion model changes the learned denoising behavior. Replacing or adjusting the text path changes how a description becomes guidance. Changing the VAE changes the conversion between the latent representation and the visible image. A workflow can keep two of those roles constant while testing the third, provided the selected components are compatible. The graph exposes the experiment's boundary.

Text enters the graph as a literal value, but it does not go directly into the denoiser. The text-encoding node receives the text and a linked CLIP component. It tokenizes the text and produces CONDITIONING. In a typical image-generation workflow there are positive and negative descriptions. The positive path describes attributes the user wants to encourage; the negative path supplies a contrasting instruction about attributes to suppress. Both are typed conditioning values that the sampler can consume. The word “prompt” therefore refers to more than the string typed into a box. The graph shows the transformation from language to a model-facing representation.

That distinction clarifies why editing text and editing conditioning are not always equivalent operations. A text encoder may apply model-specific tokenization and scheduling. A later conditioning node can combine, average, concatenate, scale, or localize already encoded values. If a user changes the raw text, the encoding path must reflect that change. If they adjust a later conditioning operation, the text encoder's result may remain the same while the guidance presented to sampling changes. The graph lets those choices be placed at different boundaries instead of collapsing them into one prompt field.

The other starting point is the latent.