Overview
This series explains how LoRA keeps a pretrained model shared while learning compact low-rank changes for task adaptation, then follows the costs those changes move across training, storage, inference, context, and mixed-task serving. It tests rank and target placement against reported results across models and tasks, and extends the discussion to compression recovery and versioned on-device customization. The evidence supports conditional efficiency, not a universal rank or quality guarantee.
Learning outcomes
- LoRA reduces the cost of task variants by storing compact changes around one shared base, not by replacing the base with a small model.
- Merging a LoRA update can preserve ordinary dense inference, while mixed-task batches require dynamic selection or grouped execution.
- Rank and target placement are separate choices; reported results support testing both under a shared budget rather than a universal configuration.
- Across the reported experiments, LoRA was often competitive with full tuning using far fewer trainable parameters, but quality and lifecycle fit remain conditional.
Keywords
Preview every chapter · Unlock full text, audio, and video from chapter 4
PlansFAQ
What problem does LoRA solve?
LoRA reduces the cost of creating many task-specific model variants by freezing one shared base and learning a small low-rank update for each task. It does not remove the base model’s storage or computation.
How does LoRA differ from full fine-tuning?
LoRA leaves pretrained weights frozen and trains factors whose product is added to selected matrices, while full fine-tuning updates the model weights directly. The factors can be stored as a compact task artifact and merged before serving.
When is LoRA a good fit?
LoRA is a good fit when a capable shared base already contains most needed capability, the task change is compact, and the serving path can manage the chosen update. It is less sufficient when broad capability is missing or per-example task selection dominates.
Does LoRA always match full fine-tuning?
No. In the reported experiments, LoRA often matched or exceeded full fine-tuning with far fewer trainable parameters, but results varied by model, task, data regime, and configuration. A particular rank and placement still require evaluation.
Can a LoRA adapter work with any model version?
No. An adapter is tied to the base model version and parameterization it was trained against, so a new base version generally requires the adapter to be retrained and validated. A small file is not a universal patch.