In the first article in our AI series we followed the money to see “Who Gets Paid When You Ask AI a Question?“ Now let’s ask our fictional research app, “Earnings Desk,” to compare two earnings reports. A few seconds later you have something useful to read. Meanwhile, somewhere in a data center, several kinds of chips have been busy preparing that answer.

Nvidia (NVDA) gets much of the attention, understandably. But an AI computer needs more than a GPU, and understanding what else goes inside helps explain why so many other companies are getting a share of the spending.

Why Nvidia Got the Attention

GPUs, or graphics processing units, were developed to handle graphics, including the images in video games. Their ability to perform many calculations at once also proved to be a great fit for the math behind AI.

Nvidia became the poster child for AI because its GPUs were already well suited to that work and came with software that developers already knew how to use. When companies rushed to build and run bigger models, Nvidia was a known quantity.

Nvidia’s software includes CUDA, which helps programmers use Nvidia’s GPUs. Years spent building software around those tools give customers a reason to stay. Switching suppliers can mean changing software and retesting everything, so a cheaper chip must offer enough savings to make that effort worthwhile.

Advanced Micro Devices (AMD) competes here with its Instinct GPUs and ROCm software. But AMD also sells another kind of chip these computers need.

The CPU Has Plenty to Do

The central processing unit, or CPU, runs general software and coordinates tasks. Intel (INTC) sells Xeon CPUs, and AMD sells EPYC CPUs. A server can pair a CPU from either company with GPUs that handle the model’s heavy calculations.

Suppose we ask Earnings Desk to find three quarterly filings, calculate how margins changed, and build a spreadsheet. The model can help decide what to do, but other software still has to retrieve the documents, run the calculations, and create the file. CPUs do much of that work.

Now, suppose there are thousands of customers making requests like that, with each request setting off several more tasks. You can see why the CPU starts getting busy.

At this point, you’ve probably heard the term AI agents. This is the kind of work people are talking about, with AI carrying out several steps on our behalf. CPUs were already part of AI systems, but giving those systems more tasks creates more work for the CPUs that keep everything moving.

Arm Holdings (ARM) gets into these computers a little differently. A company designing a CPU can pay to use Arm’s technology, giving it a head start on the job. Nvidia does this with its Grace CPU, so even a machine full of Nvidia hardware can earn Arm licensing fees and royalties.

Arm has now introduced its own data center CPU as well. Customers can choose CPUs from Intel, AMD, or those built on Arm technology, depending on the software they run and what it costs to achieve the performance they need.

Memory Can Hold Up the Whole Operation

Even a powerful GPU can spend time waiting if memory can’t supply information quickly enough. Think of a restaurant with plenty of cooks but a kitchen that can’t get ingredients to them. Hiring more cooks won’t get dinner out any faster.

Getting enough memory is another challenge, and if you’ve been watching Micron’s (MU) rally, that helps explain the excitement. Its shares had gained nearly 270% year-to-date. In its June earnings commentary, Micron said memory demand substantially exceeded supply.

AI systems need memory to store the model and the information it uses. High-bandwidth memory, or HBM, stacks memory chips close to the processor so large amounts of information can move quickly between them.

SK Hynix (SKHY), Micron (MU), and Samsung (SSNLF) supply this memory. Faster memory helps customers get more work from their processors, while shortages can hold up new systems.

Custom Chips and the Connections Between them

Google (GOOGL) and Amazon (AMZN) have another incentive to develop chips. They run enormous cloud businesses, so reducing the cost of AI work can save them a ton of money. A chip designed around the work they do repeatedly may use less power or deliver more for each dollar spent.

Google’s TPUs, or tensor processing units, and Amazon’s Trainium chips are designed for AI calculations. They give those companies another option alongside GPUs for training models and generating answers.

Of course, deciding you want your own chip is easier than designing one. Even the biggest cloud companies bring in help, which gives Broadcom (AVGO) and Marvell (MRVL) a place in the business. They work with customers to design custom processors and supply the finished chips. Marvell’s agreement with AWS, for example, includes custom AI products.

Both companies also supply networking chips. When an AI model runs across many processors, those processors need to exchange information. Broadcom’s switch chips direct that traffic, while Marvell’s optical networking chips help move it over fiber connections.

As customers add processors, they also need better connections between them. Nobody wants expensive equipment waiting on a traffic jam.

Dell (DELL) and Super Micro Computer (SMCI) put these pieces together in servers and the tall racks that hold them, along with the cooling that keeps everything running. Customers pay them to turn a collection of parts into computers they can put to work.

Up Next

Our earnings reports still have to be stored somewhere, brought within reach, and shared between machines. There’s plenty of business in getting that information to the right place, and that’s where we’ll pick up next.

Note: This is the second article in Bob Byrne’s AI series. Read the first article, “Who Gets Paid When You Ask AI a Question?”, here.

At the time of publication, Byrne had no positions in any securities mentioned.