How Much VRAM Do You Need for a Local LLM in 2026?

The question is usually asked backwards. People ask how much VRAM a model needs, when the useful question is which models your machine can run, and the answer starts with memory you actually have free rather than what the spec sheet says.

Here is the arithmetic, the tiers, and the mistake that catches almost everyone planning around mixture-of-experts models.

The Arithmetic

A model needs memory for its weights plus overhead for the context window and activations.

Weights at four-bit quantization cost roughly 0.6GB per billion parameters. At eight bits, roughly 1.1GB per billion. At full 16-bit precision, about 2GB per billion.

Context adds on top. A modest 4096-token window costs a few hundred megabytes; a very large window can cost more than the weights, which is one reason million-token windows are impractical locally regardless of model size.

So a 4B model at four bits needs roughly 2.4GB for weights plus overhead, call it 3GB in practice. A 30B model at four bits needs roughly 18GB plus overhead, which is why 24GB is the commonly cited GPU target for that tier.

Find Your Real Budget First

Not total RAM. Free RAM while working.

Open your usual set of applications: browser with its actual number of tabs, mail client, chat, whatever else stays running. Check memory usage. Subtract from total, then leave 2GB of headroom so the machine does not start swapping, which is catastrophic for perceived speed.

On a 16GB laptop with a normal working set, the honest figure is frequently four to six gigabytes. On 8GB it may be two to three. That is your budget.

The Tiers

8GB system RAM, no discrete GPU. Models up to roughly 3B parameters at four bits. A 1.7B model is comfortable and leaves room to work. This is the configuration most corporate laptops are in, and it is sufficient for rewriting.

16GB system RAM, no discrete GPU. Up to roughly 7B or 8B at four bits, though generation will be slow enough to notice. A 4B model is the better experience: fast enough to feel instant, capable enough to hold a six-clause instruction.

32GB system RAM, no discrete GPU. Up to roughly 14B on CPU, slowly. Usually better spent on a faster smaller model unless you have a specific reason.

8GB VRAM GPU. Up to roughly 12B at four bits, fast. A meaningful upgrade over CPU inference at the same model size.

16GB VRAM GPU. Up to roughly 24B. Comfortable and quick.

24GB VRAM GPU. Up to roughly 30B, including mixture-of-experts models in that range. This is where local output becomes genuinely hard to distinguish from cloud on ordinary tasks.

Above that is enthusiast and professional territory, and for writing tasks the returns have flattened well before you get there.

The Mixture-Of-Experts Mistake

This is the single most common planning error, and it wastes a lot of downloads.

A mixture-of-experts model is described with two numbers, as in 30B-A3B: 30 billion total parameters, roughly 3 billion active per token.

Active parameters determine speed. The model computes as though it were 3B, which is why MoE models are fast for their size.

Total parameters determine memory. All 30 billion must be resident so the router can select among them.

So a 30B-A3B model needs memory for 30B, not 3B. It behaves like a small model on latency and like a large model on memory requirements. People read the active count, conclude the model will fit, download 18GB, and hit an out-of-memory error.

This scales to absurdity at the frontier. A 2.8 trillion parameter model with 40 billion active still needs memory for 2.8 trillion parameters, which is why open frontier weights are not runnable at home no matter how sparse they are. We ran that arithmetic in can you run a trillion-parameter model locally.

Partial Offload Is A Trap

Most runtimes, including llama.cpp and everything built on it, let you put some layers on the GPU and the rest in system RAM. This works and it is usually a bad deal.

The penalty for crossing the boundary is severe, because every token has to move data between GPU memory and system memory. A model fully resident in VRAM might generate at forty tokens per second; the same model with a quarter of its layers spilled might manage six.

A smaller model fully resident almost always feels better than a larger model partially offloaded. Given the choice, drop a tier.

Quantization Is The Other Lever

Before buying hardware, spend the free lever.

Q4_K_M is the default for good reason: roughly a quarter of full-precision memory, with degradation small enough to be hard to notice on writing tasks. Q5 or Q6 if you have headroom. Below Q4, quality falls off faster than the memory savings justify, and Q2 loses instruction-following entirely.

Given a choice between a bigger model at Q3 and a smaller model at Q5, take the smaller model at higher precision for writing work. Instruction-following reliability matters more than raw capability on constrained tasks. There is a fuller treatment in int4 quantization on consumer hardware.

The NPU Does Not Help

Worth stating because it affects purchasing.

As of mid-2026, the mainstream local-LLM runtimes do not route language model inference to the NPU. They use the GPU through Vulkan or similar, or the CPU. NPU execution requires vendor-specific model conversion and compilation, which is a specialist pipeline rather than a drop-in accelerator.

So an NPU-certified laptop with 8GB of RAM will not run local models better than a non-certified laptop with 32GB. RAM and VRAM are the specifications that matter. See NPU support in local LLM runtimes.

What To Buy, If You Are Buying

For rewriting and general text work: you probably do not need to buy anything. A 1.7B to 4B model on your existing machine covers it. Spend the money on nothing.

If you want the 30B tier: 24GB of VRAM, and check the model fits at Q4 with your context window before assuming.

If you are specifying a work machine: 32GB of system RAM is the highest-value line item, because it determines which models are available at all. VRAM second. NPU certification is not a factor for this workload.

Common Questions

Can I run a local model with no GPU at all?

Yes, comfortably, in the 1B to 4B range. Wrivio’s embedded engine targets exactly this case because most work laptops have no discrete GPU.

Does system RAM matter if I have a GPU?

Yes, for loading the model and for anything that spills. It also determines whether you can run a model on CPU as a fallback.

Is a Mac’s unified memory different?

Practically, yes: unified memory is shared between CPU and GPU, so a Mac with 32GB can hold larger models than a PC with 32GB of system RAM and a small GPU. The parameter arithmetic is the same.

How much disk space should I plan for?

One to three gigabytes per model at Q4. Keeping a fast model and a better model is a sensible setup and still under five gigabytes.

Download Wrivio for Windows to run a model sized for your machine, with the memory arithmetic already done.