NPU vs CPU vs GPU for Local AI Writing

For local text rewriting, the CPU does the work on most machines today, the GPU does it faster if you have a capable one with enough video memory, and the NPU usually does nothing at all because the software has not been written to use it yet.

That is the short version. The longer version explains why, and why the answer will look different in a couple of years.

What Each Chip Is For

CPU. General purpose, high single-thread speed, direct access to all system memory. Modern ones have wide vector instructions, AVX2 and AVX-512 on x86, which is what makes language model inference viable at all. Its advantage is that system RAM is large and cheap, so a CPU can load a model that would never fit in a mid-range graphics card.

GPU. Massively parallel and, more importantly, attached to very fast dedicated memory. This is the fastest option by a wide margin, but only if the model fits in video memory. A card with 8 GB of VRAM comfortably holds a small quantized model. Once you exceed VRAM, performance falls off a cliff as data shuffles across the bus.

NPU. Designed for sustained inference at very low power. Excellent for things that run continuously in the background: transcription, noise suppression, camera effects, background blur. Its strength is performance per watt, not peak throughput.

Why Memory Bandwidth Decides Everything

Here is the part that surprises people. Generating text with a language model is not compute-bound on most consumer hardware. It is memory-bound.

For every single token produced, the machine has to read a large share of the model’s weights out of memory. A 2.4 GB model means moving something close to 2.4 GB per token. Do the arithmetic against a laptop with roughly 50 GB/s of memory bandwidth and you get an approximate ceiling of twenty tokens per second, no matter how fast the processor is.

This is why a GPU with 400 GB/s of dedicated bandwidth feels dramatically quicker, and why two laptops with the same CPU can differ by a factor of two purely because one has faster memory. It is also why buying more cores rarely helps.

Where the NPU Falls Down Today

NPUs have respectable throughput figures. The obstacle is not silicon, it is software.

Language model tooling grew up around llama.cpp and its descendants, which target CPU and GPU. NPU access requires vendor-specific runtimes, and Qualcomm, Intel, and AMD each expose theirs differently. An application that wants NPU acceleration has to implement and test three separate paths, then handle the machines that have none of them.

For a small text tool, that is a large amount of engineering for a benefit most users would experience as a modest battery saving. So it waits.

What This Means for Your Machine

Check three things before worrying about any of this.

Does your CPU support AVX2? On Windows, tools like CPU-Z will tell you. Anything from roughly 2013 onward does. This is a hard requirement for many local engines rather than a nice-to-have.

How much free RAM do you have? Not installed, free. A small model needs its size plus overhead available at the same time as your browser.

Do you have a discrete GPU with 8 GB or more of VRAM? If yes, tools with GPU offload will be noticeably faster. If no, CPU is the path and that is entirely normal.

The Practical Configuration

Wrivio’s Local mode runs llama.cpp in-process on the CPU. The Standard model is about 1.1 GB on disk and roughly 1.7 GB of RAM while generating. The larger option is about 2.4 GB on disk and roughly 3 GB in memory. Nothing to install separately, no server to run, no GPU required.

On a typical modern laptop that produces a rewrite of a normal email in a few seconds. On an older machine it is slower, and the app notices and tells you rather than leaving you guessing.

If the machine cannot run local inference at all, the app disables the option instead of crashing, and routes you to the cloud path.

Common Questions

Should I buy a GPU just for local AI writing?

For writing tasks specifically, no. The models involved are small and CPU inference is adequate. A GPU makes sense if you also run larger models, image generation, or fine-tuning.

Does more RAM make generation faster?

Not directly. More RAM lets you run larger models and keep more open alongside them. Faster RAM does improve speed, because bandwidth is the bottleneck.

What about Apple Silicon?

Unified memory gives it high bandwidth shared between CPU and GPU, which is why Macs punch above their weight on local models. Wrivio is Windows-only today.

Will my laptop battery suffer?

Sustained CPU inference does draw power. For occasional rewriting of emails it is negligible; for continuous generation it is noticeable.

Download Wrivio for Windows to run local rewrites on the CPU you already have, with no GPU, NPU, or extra install required.