# How much VRAM do you really need for local AI in 2026

> A 70B in Q4 calls for 40-48 GB and a 24 GB card falls short. How to calculate how much VRAM you need and how to measure it on your machine before buying.
> By José Maldonado · Artificial Intelligence · 2026-10-27

The question almost always arrives in the same shape, and almost always after watching a video in which a huge model responds on someone else's laptop. Will my card do? It is worth starting with the number that settles half the debate: a 70 billion parameter model in Q4 quantization takes up about 35 GB in weights alone, and with context and KV cache the realistic consumption rises to [40-48 GB](https://llamaperf.com/guides/vram-requirements) if you want a context window that is good for working and not just for greeting the model.

That figure breaks the intuition that local AI is about computing power. The weights have to be held in full somewhere, and when they do not fit on the card the system spreads them between VRAM and RAM with a huge speed penalty. A GeForce RTX 5090 lists 32 GB of GDDR7 in the [official NVIDIA spec sheet](https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/), and with that you do not reach a 70B in Q4 without cutting context and accepting a split with system memory.

The math takes two minutes with one multiplication and a margin, and the result almost never matches what you feel like buying. The tiers below separate what fits comfortably from what starts up and drags.

## AI's appetite for VRAM

Three things compete for the same space, and only the first is decided by the model's maker. The weights are the big, predictable part: it is calculated by multiplying the parameters in billions by the bits each weight takes up and dividing by eight to convert to gigabytes, the formula behind [the tables of 85 open models](https://willitrunllm.com/can-i-run/) reviewed on October 23, 2026. The detail almost nobody looks at is in the decimals: Q4\_K\_M does not use four bits per weight but 4.85, and Q8\_0 goes to 8.5, according to the same source.

The KV cache is the second contender and grows with context: a 70B at 4K context consumes about 3 GB in FP16 and at 32K it multiplies by eight, up to about 24 GB, again according to llamaperf. The third addend is the margin the inference engine reserves, which willitrunllm puts at 5% of the weights plus 0.6 GB, with a ceiling of 3 GB.

The 10% and 1 GB that willitrunllm reserves for a context of about 4,000 tokens is the cushion that keeps the model from crashing when the conversation gets long. With a 70B in Q4\_K\_M the math works out like this:

1.  Weights: 70 × 4.85 ÷ 8, about 42.4 GB.
2.  Margin of 10% plus 1 GB on the weights: about 47.6 GB.
3.  KV cache with 4,000 tokens in FP16: about 3 GB more.
4.  Realistic total: above 50 GB, not the 35 GB the four-bit rule suggests.

That gap explains why two people with the same model get different results: one test with the engine's default configuration and another with long context and cache at full precision are not measuring the same thing, even if both are advertised as "a 70B in Q4".

## An MoE does not save you memory, only compute

Liquid AI's LFM2.5-24B-A2B is the most confusing example: [it has 24 billion total parameters and activates only two billion per token](https://docs.liquid.ai/lfm/models/lfm25-24b-a2b), with 64 experts per block and top-4 routing over ten attention layers. The savings are in compute and latency, not in space: the file weighs as much as the full model, and in Q4\_K\_M that is about 14.6 GB before counting context.

The KV cache is not reduced either by activating fewer experts, because it depends on the attention architecture and the context length, not on the number of experts that come into play. With 131,000 tokens of window declared in that same spec sheet, the model can grow considerably more than its size on disk suggests.

That is why sparse routing changes the time bill, not the memory bill, and it is worth being clear about that before buying VRAM thinking an MoE will do you a favor it does not do.

## Which model fits in each card tier?

The figures in the table come from applying the above formula to quantizations that exist, not to the ones product sheets promise. They are estimates: actual consumption depends on the exact quantization, the context length, the batch size and the inference engine chosen.

VRAM

Model that fits comfortably

Quantization

Comfortable context

8 GB

7B to 8B

Q4\_K\_M

4K to 8K

12-16 GB

13B to 14B, or 8B in Q8\_0

Q4\_K\_M, Q8\_0

8K to 16K

24 GB

24B to 32B

Q4\_K\_M

8K to 16K

48 GB

70B

Q4\_K\_M

4K to 8K

64-96 GB

70B in Q8\_0 or 120B in Q4\_K\_M

Q8\_0, Q4\_K\_M

16K to 32K

The 8 GB tier is the deceptive one, because a 7B in Q4\_K\_M takes up about 5 GB and leaves little room for context: as soon as you open a long document the split with the CPU begins. The 12 to 16 GB band balances price and capacity better, because a 13B in Q4\_K\_M takes up about 8 GB and leaves room for 8K of context.

24 GB is the sweet spot for anyone who wants a work model without spending on two GPUs: a 32B in Q4\_K\_M is about 19.4 GB of weights, so it fits with margin for 8K of context and some cache. Above 48 GB we are already talking about a 70B in Q4 or two cards, and there the limit stops being money and becomes the power supply and the space in the case.

## When are the CPU and system RAM enough?

Splitting a model between the GPU and the processor works, but it turns bandwidth into the factor that decides your experience. Dual-channel DDR5 moves tens of gigabytes per second and a high-end card plays in another league, so every token that comes out of the non-accelerated part costs several times more time than the previous one.

There are cases where splitting makes perfect sense: models up to 14 billion in Q4, batch tasks and uses where privacy matters more than latency. If you query twice a day, the processor serves you just as well.

Splitting becomes a punishment when context grows, when there are several users at once or when the model goes past 30 billion. There the wait stops being an annoyance and becomes a change of tool, and that is the moment when GPU memory stops being a luxury.

Anyone building a machine from scratch and torn between spending on a GPU or on RAM can look at the split we describe in [what to build for self-hosting in 2026](https://observatorioblockchain.com/en/technology/autohospedaje-2026-que-montar/): system memory is bought once and is hard to upgrade, while the card is replaced entirely.

## How to measure on your machine before paying?

The definitive test is not in any third-party benchmark, but in the machine in front of you and the real use you are going to give it. If you use Ollama, [the \`ollama ps\` command](https://docs.ollama.com/faq) shows the size the loaded model takes up and the split between processor and GPU in the PROCESSOR column, with percentages like 48%/52% when part of the model lives on the CPU.

The detail that invalidates many tests is the default context: Ollama starts with 4,096 tokens, so a measurement with factory values says nothing about your case if you work with long documents. Raise the context before concluding anything, for example with \`OLLAMA\_CONTEXT\_LENGTH=8192 ollama serve\`, and repeat the same question with the same document.

If the model falls just short, the most profitable lever is KV cache quantization: \`OLLAMA\_KV\_CACHE\_TYPE=q8\_0\` stores it at 8 bits instead of 16 and halves its size, although the option is global and affects all the models you load, again according to the same FAQ. The inference engine also moves the figure, and in the comparison of [Ollama, llama.cpp, vLLM and TGI](https://observatorioblockchain.com/en/ai-agents/ollama-llamacpp-vllm-tgi-comparativa/) you can see that each one reserves memory in its own way.

To see consumption live, \`nvidia-smi\` gives the GPU's VRAM usage in real time and serves to catch the peak while the model loads into memory. Before buying, test long context for real: a 30,000 or 40,000 token document says more about your future bill than any third-party table.

## The mistakes that cost money

Buying memory for the model you are never going to run is still the most expensive mistake, and it almost always comes from the same place: a video where a 70B responds on someone else's machine. Before paying, write down which model, with which quantization and with which context you are really going to use, and check whether the number leaves you margin or forces you to cut back as soon as the first long PDF arrives.

Confusing size with quality pushes spending in the opposite direction: a 24B that activates 2B per token can answer you faster than a dense 70B, and do it with less memory, although on open-ended questions you notice the difference.

Forgetting context is the most common mistake in home comparisons, because the default value in almost any environment is short and comfortable. The KV cache multiplies by eight going from 4K to 32K on a 70B, so a test that runs fine with 4,000 tokens can blow out memory as soon as you drop in a whole report.

Trusting the tokens per second of someone else's benchmark closes the list, and it is the hardest to spot because the number looks objective. Speed depends on the inference engine, the batch size, the quantization and the memory bandwidth, so the same model can run twice as fast on two machines with the same card.

## Memory matters more than FLOPs

Marketing sold teraflops as the measure of everything, and local inference has put every figure back in its place: a machine with less compute and more memory can beat the one with the more expensive case.

The market backdrop does not invite calm decisions: DRAM contract prices [rise around 20% in the fourth quarter of 2026](https://www.trendforce.com/news/2026/10/20/news-dram-prices-seen-rising-20-qoq-in-4q26-despite-slower-gains-ai-demand-tight-supply-support-outlook/), after 80% in the second and 40% in the third, with AI inference demand squeezing supply.

With memory expensive, the tier you are not going to use is the one that pays off worst, and the one you do use holds up better when you have sized it with context and not with enthusiasm. What does not fit on the card cannot be fixed by any new driver.
