Lesson 2 of 7
What NVIDIA actually does
6 min read
One company sits under almost every AI model running today. It's tempting to say 'they make the chips' — but the chips are only half of why they're so hard to replace.
Two products, not one
NVIDIA is really two things stacked together. The bottom half is hardware: GPUs packed with thousands of cores and fast memory. The top half is software: CUDA — a toolkit that lets programmers actually use all those cores without hand-writing the low-level details. Every popular AI framework, the ones researchers and engineers build on, talks to the GPU through CUDA.
NVIDIA sells a chip and a software platform. The chip is fast; the platform, CUDA, is what makes the chip usable — and it's the harder half to copy.
Why the software is the moat
A competitor can build a fast chip. What's far harder is rebuilding the years of tools, libraries, and tutorials that everyone already writes their code against. Because the whole ecosystem assumes CUDA, switching to different hardware often means rewriting or re-tuning your software — so teams stay put. That lock-in, not just raw speed, is why NVIDIA is so dominant in AI.
The lock-in isn't the silicon — it's CUDA. When all the software assumes one platform, the platform is the moat.
This is changing slowly: there are efforts to run models on other vendors' chips, and portable layers that hide the hardware. But today, 'AI compute' still mostly means 'a CUDA GPU'.
The shape of it
- —NVIDIA sells hardware (GPUs) and software (CUDA) as one package.
- —CUDA lets code use the GPU's thousands of cores — every major framework relies on it.
- —The moat is the software ecosystem, not just the chip — switching hardware means rewriting code.
A rival ships a GPU as fast as NVIDIA's. Why might teams still not switch?
Continue in the app
Take the whole Inference & Hardware course — tracked
Get your personalized path, progress and streaks in the app — this lesson and every next one, in order.