NodalZen

Blog · Explainer

What is an AI data center

· 6 min read · basics, power, cooling, grid

A plain-English tour of an AI data center, from a single chip to the rack, the building and the power grid that feeds it.

When you type a question into an AI chatbot, the answer is worked out in a building you will probably never see. That building is an AI data center. This post walks through it from the smallest part to the largest: the chip, the rack, the building and the power grid.

The chip

At the heart of everything is a special computer chip. Most AI work runs on a GPU (graphics processing unit), a chip first built to draw video game graphics. It turned out to be very good at doing millions of small sums at the same time, which is exactly what AI models need. Some companies design their own AI chips that do the same job, such as Google's TPUs, or tensor processing units (Google Research, November 2025).

Every one of these chips turns electricity into calculations, and almost all of that electricity ends up as heat. Think of a hair dryer that never switches off. That single fact shapes the whole building.

The server and the rack

A few chips are mounted together on a board with memory and network connections. That box is a server. Servers are stacked in a tall metal cabinet called a rack, a bit like books on a bookshelf.

Racks used for AI are far more power-hungry than the racks that hold ordinary websites or email. As one example, Nvidia's GB200 NVL72 rack design packs 72 GPUs into a single rack and draws about 120 kilowatts, which the company described as roughly double the rack designs of the time (eeNews Europe, October 2024).

To picture 120 kilowatts: the average US home bought about 10,791 kilowatt-hours of electricity in 2022 (US Energy Information Administration). A rack running flat out at 120 kilowatts for a full year would use about 1.05 million kilowatt-hours, which by simple arithmetic is roughly what 97 such homes use in a year. One rack, one small neighborhood.

The network

AI models are often too big for one rack, so hundreds or thousands of racks are wired together with fast network cables. They behave like one giant computer, called a cluster. If the network is slow, the expensive chips sit waiting, so the cabling matters as much as the chips.

The building

The building's job is to keep the racks powered, cool and safe, all day, every day.

Power inside the building. Electricity arrives at very high voltage, which is efficient for long distances but far too strong for a computer. Large devices called transformers step it down, the way a phone charger steps wall power down for your phone. Switchgear is the big set of switches and breakers that routes power and cuts it off safely if something goes wrong.

Backup power. A UPS (uninterruptible power supply) uses batteries to keep everything running for the seconds or minutes after a power cut. Generators, usually running on diesel or gas, take over for longer outages.

Cooling. All that heat has to go somewhere. Older data centers blow cold air through the racks. Newer AI racks often pump liquid right up to the hottest chips, because liquid carries heat away far better than air. Builders track how much extra power the building uses for cooling and everything else with a number called PUE (power usage effectiveness). We cover it in its own post.

The power grid

Finally, the building has to plug into the power grid, the network of power plants, high-voltage lines and substations that supplies a region. Thousands of racks like the one above add up quickly, so the local utility may have to build new lines or substations before a large site can connect.

This is now a national story. A US Department of Energy summary of a Lawrence Berkeley National Laboratory report says data centers used about 4.4% of all US electricity in 2023 (US Department of Energy, December 2024). That is why choosing a site with spare grid capacity, and waiting for equipment like transformers, have become some of the biggest questions in building one.

Putting it together

Follow the electricity and the whole picture makes sense. Power flows from the grid, through transformers and switchgear, past the backup systems, into racks full of chips. The chips turn it into answers and heat. The cooling system carries the heat away. Every part has to be sized to match the others, or the most expensive part, the chips, sits idle.

Try it in NodalZen

Open Size to see how much power, cooling and backup a GPU cluster needs, then Site to explore where in the US it could connect to the grid. If you are brand new, the Learn lessons walk through each part step by step.

Sources

Learn the basics Open the planner