After 26 years in software infrastructure, including working on EC2, I wasn’t expecting AI data centers to surprise me quite this much.

But eight months of working on new deployments at Baseten, and watching our neocloud partners build theirs, have given me a fresh education. Familiar problems of power, cooling, networking, and reliability take on different dimensions when you’re putting hundreds of kilowatts into multi-million dollar racks.

This talk covers four parts of that education:

• Financing: How do you pay the hundreds of million dollars needed?
• The physical side: power, cooling, ceilings, and even concrete depth.
• Networking: why is NVIDIA saying your GPUs need 3 different networks?
• Delivery: How to work out when a partner will actually deliver the capacity you’re paying for, instead of blindly trusting the hope-based date they will give you.

For each, I’ll describe the assumptions we started with, what we learned, and how that changed our decisions. You’ll leave with specific questions to ask when someone offers you GPU capacity, and an appreciation for how much work remains after the GPUs have been ordered.