What Is an NPU?

November 14, 2026
4 min read

In a nutshell

An NPU is the ultimate weapon, specialized for AI inference.

What else?

It doesn’t think for itself and doesn’t decide. But the one calculation it is given, it handles with power savings in a different league.

What Is an NPU?

The full name is Neural Processing Unit. It is a dedicated circuit built only to take on AI inference.

Inference here means:

  • ❌️ Not thinking things through and reasoning.
  • ⭕️ Having an already-trained AI produce an answer

Just as a weapon does no thinking of its own, an NPU makes no decisions.

It simply follows the CPU’s orders.

All inference does is repeat “multiply, add, pass it on.”

If there is only one kind of work, the idea is to line up huge numbers of circuits made just for it.

⇒

Because it doesn’t keep going back to memory for intermediate values, it is efficient and uses little power.

An NPU can only do inference; it can’t be used for training. In exchange, it keeps running on battery. An NPU is enough if you only use AI, but you need a GPU to build AI. That is the biggest difference between the two.

So, is the PC you use now in trouble without an NPU? No.

Think of ChatGPT.

It is connected to the internet, and beyond that to cloud servers.

When you type a question, the text travels down the line to a server over there. What builds the answer is the GPUs lined up in that server.

In other words, all the PC in front of you does is send and display.

Most AI services today work this way, so with or without an NPU, the speed is the same.

So, for now, there’s no need to buy a new PC just for the NPU.

The NPU starts to matter once this calculation happens on your own machine.

If you are buying a new PC anyway, there are only two things to check.

  • Whether it has an NPU. Catalogs sometimes call it something else, such as “AI engine” or “Neural Engine.”
  • The TOPS number. It shows how many trillion operations per second it can do, and the 40 TOPS Microsoft required for Copilot+ PCs is one benchmark (Introducing Copilot+ PCs).

Takeaways

I expect AI will shift toward mostly running locally, and toward a partner-like role that can handle confidential information.

The benefits:

  • No connection needed
  • It can handle things you don’t want to send outside
  • Replies come fast

Along with that trend, the NPU should become a part that is simply always there. Like the CPU or memory, before long nobody will think about it.

So there’s no need to rush out and buy one now. By your next upgrade, it’ll probably just be in there.

Trivia

  • TOPS: Short for Tera Operations Per Second. The NPU spec-sheet figure for how many trillion operations it can do per second.
  • INT8: Calculating with numbers rounded to 8-bit integers. It gives up a little accuracy in exchange for speed and power (Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference).
  • Systolic array: A circuit design that lines up calculators in a grid and has them compute while handing data to their neighbors. The approach TPU adopted.
  • Neural Engine: Apple’s name for its NPU. Only the name differs by maker; the role is the same.
  • Copilot+ PC: Microsoft’s requirements for PCs with AI features. It calls for an NPU of 40 TOPS or more.

History

In the 2010s, as photo classification and speech recognition became practical, the first thing to cry out was data-center power. The more users there were, the more inferences ran, and general-purpose parts couldn’t keep up with either the work or the electricity bill. So Google built TPU, a dedicated chip that runs only inference, and began operating it in its own data centers in 2015 (In-Datacenter Performance Analysis of a Tensor Processing Unit). It showed, with real measurements, how far efficiency rises when you go all-in on a dedicated weapon.

Next, the same problem hit smartphones. Sending every photo edit or face recognition to a server needs a connection and is slow. Computing on the device solves that, but then the battery doesn’t last. In 2017, Apple put a dedicated circuit called the neural engine into the A11 Bionic, pulling this work inside a battery-powered device (Apple Newsroom, “The future is here: iPhone X”).

The clincher was the PC. In 2024, Microsoft required an NPU of 40 TOPS or more for Copilot+ PCs (Introducing Copilot+ PCs). With features of the OS itself assuming an NPU, the NPU began moving from “an add-on that makes things faster if you have it” to a standard part alongside the CPU and GPU. All three times, the problem was the same: power.

Timeline

PeriodKey eventsKey people and organizationsCountry or regionNotes
2015TPU, a dedicated inference chip, starts running in Google’s data centersGoogleUnited StatesAs use of photo classification and speech recognition grew, general-purpose parts could no longer keep up with the work or the electricity bill. It showed in real operation how far efficiency rises when a circuit is devoted to inference alone. [The TPU paper (arXiv)](https://arxiv.org/abs/1704.04760)
2017The A11 Bionic in the iPhone X carries a “Neural Engine”AppleUnited StatesA dedicated circuit for computing things like face recognition inside a battery-powered device instead of sending them to a server. The NPU came down from the data center to the device in your hand. [Apple Newsroom](https://www.apple.com/newsroom/2017/09/the-future-is-here-iphone-x/)
2024Copilot+ PCs require an NPU of 40 TOPS or moreMicrosoftUnited StatesFeatures of the OS itself now assumed an NPU. The NPU began moving from “an add-on that makes things faster if you have it” to a standard part alongside the CPU and GPU. [Introducing Copilot+ PCs](https://blogs.microsoft.com/blog/2024/05/20/introducing-copilot-pcs/)