LoRA vs Full Fine-Tuning vs QLoRA: A Plain-Language Comparison for Non-Technical Founders

Meta's 2023 release of LoRA-trained Llama 2 variants demonstrated that a model fine-tuned with Low-Rank Adaptation on a single A100 GPU for under USD 200 could outperform a much larger general-purpose model on a specific legal classification task. That result changed how founders and product teams think about model customisation. The three dominant approaches today (LoRA, full fine-tuning, and QLoRA) each make different trade-offs on compute cost, data requirements, output quality, and deployment flexibility. Choosing the wrong one can waste tens of thousands of dollars or produce a model that underperforms the base version it replaced.

This guide explains each approach in plain language, with specific figures on cost and data requirements, and a decision framework to help non-technical buyers choose before they engage a developer. The goal is not to make you an ML engineer; it is to make you an informed buyer who can evaluate a developer's recommendation critically.

A pre-trained language model (GPT-4, Llama 3, Mistral, Gemma) has learned patterns from hundreds of billions of tokens of text. Fine-tuning adapts that model to a specific task or domain by continuing its training on a curated, task-specific dataset. After fine-tuning, the model produces outputs that are more consistent with the target domain: a legal document classifier that has been fine-tuned on case law outperforms a general-purpose model on that task because its internal weights have been adjusted to prioritise legal language patterns.

The three approaches differ in how much of the model they adjust, which determines the cost, the required data volume, and the quality ceiling. Full fine-tuning adjusts all parameters. LoRA and QLoRA adjust a small subset, using mathematical shortcuts to achieve comparable task-specific performance at a fraction of the compute cost. The trade-off is that parameter-efficient methods have a lower quality ceiling for tasks that require deep domain shift, and that ceiling matters when selecting an approach.

Full fine-tuning updates every weight in the model during training. For a 7-billion-parameter model such as Mistral 7B, this requires a GPU cluster with sufficient VRAM to hold the full model, the gradients, and the optimiser states simultaneously, typically 80 to 160 GB of GPU VRAM. On AWS, a single p4d.24xlarge instance (eight A100 GPUs, 320 GB VRAM) costs approximately USD 32 per hour. A training run for a 7B model on a 100,000-example dataset takes four to twelve hours depending on batch size and sequence length, putting the compute cost at USD 130 to 400 per run, not including the cost of data preparation, evaluation, and iteration.

Full fine-tuning is the right choice when the domain shift is large (training a general model to behave like a specialised medical or legal model), when the training corpus is large and high quality (500,000 or more labelled examples), or when the quality ceiling of parameter-efficient methods is demonstrably insufficient for the task. As the AI predictions 2026 analysis notes, vertical LLMs trained on domain-specific corpora are increasingly the standard for regulated industries. Developers providing custom AI model training at this scale work with GPU clusters, data pipelines, and evaluation frameworks that most organisations do not maintain internally.

The other cost to account for is storage and deployment. A full fine-tuned 7B model requires approximately 14 GB in fp16 format. A 70B model requires approximately 140 GB. These weights must be hosted, versioned, and served, costs that scale with model size and request volume.

Low-Rank Adaptation, introduced in the 2021 Microsoft Research paper by Hu et al. and now the dominant parameter-efficient fine-tuning (PEFT) method, inserts small trainable weight matrices (called adapter matrices) into the transformer layers of a frozen base model. During training, only these adapters are updated. The base model weights never change. After training, the adapter weights can be stored separately (typically 10 to 100 MB versus 14 GB for the full model) and merged into the base model at inference time or applied on top.

The practical implications are significant. A LoRA training run on a 7B model requires approximately 16 to 24 GB of GPU VRAM, which fits on a single consumer-grade A10G or A100 GPU available via AWS or RunPod for under USD 2 per hour. A fine-tuning run on 10,000 to 50,000 training examples typically completes in two to six hours, putting the compute cost at under USD 20 for most projects. The adapter file produced by the run can be stored in a Git repository or an S3 bucket and applied to any instance of the base model on demand.

LoRA is the correct choice for the majority of generative AI fine-tuning projects: task-specific instruction following, domain vocabulary adaptation, tone and format alignment, and classification tasks with moderate data volume. Developers providing generative AI development services typically default to LoRA for projects where full fine-tuning is not explicitly justified by the quality requirement, because it delivers 80 to 90 percent of the quality improvement at 5 to 10 percent of the compute cost.

QLoRA (Quantised LoRA), introduced in the 2023 paper by Dettmers et al. from the University of Washington, combines LoRA with 4-bit model quantisation. Quantisation reduces the numerical precision of the base model's weights from 16-bit (fp16) or 32-bit (fp32) floating point to 4-bit integers, reducing memory requirements by approximately 75 percent. A 7B model that requires 14 GB in fp16 requires approximately 4 GB in 4-bit, and a 70B model drops from 140 GB to approximately 40 GB.

QLoRA makes it possible to fine-tune large models on a single consumer GPU. Llama 3 70B, which would require an eight-GPU cluster for full fine-tuning, can be fine-tuned with QLoRA on a single A100 80GB GPU for approximately USD 2 to 4 per hour. For startups and founders with tight compute budgets, QLoRA opens access to large-model fine-tuning that was previously only feasible for well-resourced research labs.

The trade-off is a small quality penalty from quantisation noise and a slower training speed compared to fp16 LoRA. For most commercial use cases, the quality difference between QLoRA and LoRA is not detectable in production evaluation. It becomes detectable when the task requires high numerical precision (time-series reasoning, mathematical proofs) or when the training corpus is very small (under 1,000 examples) and the model needs to learn from every example with maximum fidelity. For a broader cost comparison across training approaches, the model training approaches guide breaks down cost by model size and dataset scale.

The decision sequence for most projects: start with LoRA unless the compute budget is severely constrained (QLoRA) or the task requires a quality level that LoRA demonstrably cannot reach in evaluation (full fine-tuning). A developer working in machine learning development services at production level will run a baseline evaluation with the unmodified base model, then a LoRA-fine-tuned version, before recommending full fine-tuning. If the LoRA model meets the success metric, the more expensive approach is not justified.

Fine-tuning, in any form, adapts a model's behaviour; it does not give the model access to new information that was not in its training data. A fine-tuned model still has a knowledge cutoff. If the goal is to make a model answer questions about your specific product documentation, current pricing, or recent events, fine-tuning is the wrong tool. That is a retrieval problem, solved by RAG (Retrieval-Augmented Generation), not a training problem.

Fine-tuning also does not permanently fix hallucination. A fine-tuned model is more likely to stay within the patterns of its training data, which reduces hallucination on in-distribution inputs, but it will still produce confident incorrect outputs on out-of-distribution queries. This is why evaluation against a held-out test set is non-negotiable before any fine-tuned model reaches production.

The majority of commercial generative AI fine-tuning projects in 2026 are LoRA projects. QLoRA is the practical choice when compute budget is a binding constraint or when the target model is larger than 30B parameters. Full fine-tuning is justified for large domain shifts, very large training corpora, or quality requirements that LoRA demonstrably cannot meet in evaluation. Starting with LoRA and escalating only when evaluation shows a gap is the professional default, not the cautious one.

If you need a developer who can make this decision based on your specific data, task, and quality requirements rather than a template recommendation, the right first step is to hire an AI developer with a track record of production fine-tuning across model families and task types. Bring your data, define your success metric, and the right approach becomes a technical decision rather than a guessing game.