Choosing What to Change: Adapters, Quantization, and Memory
Once the data and template contract is valid, LlamaFactory presents a different decision: how much of the model should actually change, and what can the available hardware sustain? This chapter treats full tuning, freeze tuning, LoRA/OFT, quantization, and memory-saving mechanisms as related choices rather than a contest with one winner. The practical result is a conditional method-selection rule: match the scope of the desired behavior to the trainable state, then check whether the resulting artifact and backend remain usable.
Fine-tuning changes a model by changing some state. The important variable is not only the number of trainable parameters, but which parameters are allowed to move, how their updates are represented, where the unchanged base weights live, and what must happen to the result afterward. A small adapter can be an excellent answer when a task calls for a bounded behavioral change. It can also be the wrong answer when the desired change reaches parts of the model that the adapter does not target. Conversely, full tuning provides broad capacity but may be impossible—or needlessly risky—under a limited memory budget.
Adaptation is a decision about scope
LlamaFactory’s four primary fine-tuning modes occupy different points in this design space. Full tuning leaves the model’s permitted parameters trainable. Freeze tuning makes only selected layers or modules trainable. LoRA and OFT add parameter-efficient modules to selected targets while leaving most of the base model unchanged. The framework exposes these as one configuration choice, but the resulting model state and operational path are materially different.
Full tuning is the broadest option. It is appropriate when the task requires substantial changes throughout the model or when the operator deliberately wants the model’s full capacity to move. Its cost is not just the memory for the weights themselves. Trainable weights need gradients, and the optimizer may need additional state; long sequences and large batches add activation pressure. The README’s hardware table labels its figures as estimates and places full training well above the memory requirements of adapter-based methods. That makes full tuning a resource decision before it becomes a quality decision.
Freeze tuning narrows the moving part of the model by selecting layers or modules. Positive layer counts can make later layers trainable; negative counts can select earlier layers. Extra modules can be included outside the hidden layers. This provides a middle ground between changing everything and adding an external adapter: the update lives inside the original model, but its scope is bounded. The benefit is a smaller trainable state and a potentially more controlled change. The cost is that the selected layers may not be the ones that express the desired capability. A setting that works for a stylistic adaptation may be insufficient for a new domain or a change that depends on embeddings, the output head, or cross-modal components.
LoRA changes the shape of the decision. Instead of updating the original weight directly, it installs low-rank trainable modules at selected targets. The rank controls the adapter’s intrinsic dimension; scaling and dropout affect how its update is applied; target selection determines which transformations can respond. LlamaFactory can discover all linear modules or accept an explicit target set. It can also save additional modules, resume from an existing adapter, create a new adapter, and later merge adapter state into the base model when the chosen path permits it.
That last point is important.