One Encoder, Many Tasks

BERT's pretrained checkpoint becomes useful only at the point where a task asks for a particular kind of decision. The focus here is that boundary: how the same packed input can support a sentence label, a relationship judgment, a token tag, or an answer span, and what the task-specific layer still has to provide. The result is a practical rule for reading BERT systems: [CLS] is suited to whole-sequence judgments, token states are needed for localized decisions, and fine-tuning and feature extraction reuse the checkpoint in different ways.

The task head changes; the encoder stays

BERT's downstream recipe starts with a simple architectural decision. Keep the pretrained Transformer encoder and change the inputs and outputs around it just enough for the task. A single sentence can be represented as one packed sequence. Two related spans can be concatenated with [SEP] and marked with their segment identities. A question and a passage use that same pair format. The encoder does not need a separate internal design for each arrangement; it processes the resulting sequence with the contextual machinery already learned during pretraining.

This does not mean that every task becomes identical. The task head—the small component that turns BERT's hidden vectors into the required prediction—owns the final interpretation. A sentiment task needs a set of class scores. Natural-language inference needs labels for the relationship between a premise and a hypothesis. Named entity recognition needs one label at each token position. Question answering needs a way to identify a start and an end within the passage. The common encoder produces representations; the head defines what counts as a correct decision.

For sequence-level classification, BERT uses the final hidden vector at [CLS] as an aggregate representation. A new classification layer maps that vector to the task's label set. The input may be a single sentence, such as a movie-review sentence for sentiment, or a pair, such as a premise and hypothesis for inference. In both cases, self-attention has already allowed the positions to interact before the classifier reads the [CLS] vector. The difference is in the contents of the packed input and in the labels used for fine-tuning, not in a replacement encoder.

A sentence-pair classifier therefore does not read two independent BERT summaries and compare them only after the fact. The two spans enter one sequence, and the encoder can make their relationship part of the contextual representations. This is useful for entailment, paraphrase, and question-versus-sentence decisions, where the label depends on an interaction rather than on the meaning of either span in isolation. The segment information preserves which span supplied each token, while self-attention lets the spans exchange information.

Fine-tuning then updates all of BERT's pretrained parameters together with the newly introduced task-specific parameters. That detail prevents a common misunderstanding. “One additional output layer” does not mean that the encoder is frozen or that downstream data cannot change its behavior. It means that the task usually introduces a small amount of new structure, while the large pretrained representation is used as the starting point and adapted end to end.