Project 42
Start learning
Module 2 of 5 · 55 min

LoRA & QLoRA Parameter-Efficient Fine-Tuning

Implement Low-Rank Adaptation (LoRA) and 4-bit Quantized LoRA (QLoRA) using Hugging Face PEFT and bitsandbytes.

Core concept

By the end

You will be able to

  • Explain the mathematical foundation of low-rank matrix decomposition: W + B*A.
  • Tune LoRA rank (r), alpha scaling factor, and target module matrices (q_proj, v_proj, all-linear).
  • Deploy QLoRA with 4-bit NormalFloat (NF4) quantization and double quantization to fit on consumer GPUs.
01

The Mathematics of Low-Rank Decomposition

Full fine-tuning updates all billions of model weights, requiring massive VRAM for optimizer states and gradients. LoRA freezes original base weights W and injects trainable rank decomposition matrices A and B.

QLoRA quantizes the base model to 4-bit NF4 while computing gradients through 16-bit LoRA adapter matrices, enabling fine-tuning a 70B parameter model on a single 48GB GPU.

PEFT LoraConfig Setup in Python
python
from peft import LoraConfig, get_peft_model, TaskType

lora_config = LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type=TaskType.CAUSAL_LM
)

Practice activity

Train and Export a Domain-Specific LoRA Adapter

  1. Configure a 4-bit QLoRA training script using PEFT and TRL SFTTrainer.
  2. Train a rank-16 adapter on a domain dataset for 3 epochs.
  3. Merge adapter weights and verify loss convergence curve.

What to produce

  • Training loss curve log and exported adapter safetensors metadata.

Reflect before continuing

How does altering the LoRA alpha / r scaling ratio impact gradient updates during adapter training?

Evidence

Sources and verification

Knowledge check

Make it stick.

Pass at 80%

Choose the strongest answer for each question. Your attempts become part of your account transcript.

01What primary VRAM reduction does QLoRA achieve over standard LoRA?