What an Owned Architecture Looks Like

After my post about hidden costs on cloud, a director of engineering asked the only question that matters once you've seen your monthly AI bill broken into compute, egress, idle premium, and burst premium: "Okay, so what would you actually build instead?"

I'm going to walk you through this the way I'd walk a customer through it on a whiteboard — six layers, roughly in the order you'd actually build them. The point isn't whose logo goes on which box. The point is what each layer does to the four cost problems, and why owning a layer changes its economics in a way renting it never will.

The Compute Layer: Whose Idle Time Is It Anyway

Start with the GPUs, because that's where the idle premium lives. On a shared platform, the provider has to plan for everybody's peak at once. They oversubscribe, they build in headroom for noisy neighbors, and they bake the cost of that headroom into your hourly rate whether you ever touch it or not. Own the nodes and the math flips. A dedicated GPU node sitting at 40% utilization at 3am isn't a markup you're absorbing, it's spare capacity you already paid for once and get to use again for free. The idle premium doesn't disappear because you got more efficient. It disappears because there's no third party between you and the hardware whose margin depends on it existing.

A Network That Doesn't Bill You by the Byte

Egress is the cost component that confuses people the most, because it sounds like a network problem and it's actually a business model problem. A hyperscaler's network isn't expensive to operate. Moving a gigabyte across their backbone costs them a fraction of a cent. What you're paying for isn't the bandwidth, it's the fact that egress is a metered exit door on a platform designed to make leaving with your own data feel expensive. A private network design doesn't have that incentive built into it, because there's no second party charging you to leave your own building.

Power and Cooling Set the Floor, Not the Ceiling

Every GPU node you run draws real power and produces real heat, whether you rent it or own it. What changes is who's pricing the headroom. A hyperscaler prices burst capacity into your bill as a premium, because they're managing thermal and power budgets across thousands of tenants who might all spike at once. When you own the rack, burst capacity isn't something you rent from someone else's spare headroom — it's something you provisioned for when you sized the room. The burst premium goes away for the same reason the idle premium does: nobody's margin depends on it existing anymore.

Security, Backup, and Data-Readiness

Security overhead is real cost, it's just folded into the platform fee so completely that nobody asks about it. When you own the architecture, that same cost becomes visible, which sounds like a downside until you realize visible means controllable. Data protection for model weights and training data is non-negotiable — you're either backing this up and testing recovery, or you're one bad deployment away from losing weeks of training time. And none of this matters if the data going in is garbage. I've watched teams spend real money standing up exactly this kind of architecture, then feed it data nobody validated first. A data-readiness pass before you scale compute isn't a nice-to-have, it's the gate that decides whether everything above this line was worth building.

Where This Leaves the Four Costs

Compute stops being a number you can't decompose and becomes a number you sized yourself. Egress stops existing as a line item because there's no metered exit door. The idle premium and the burst premium both disappear for the same underlying reason — they were never costs in the first place, they were someone else's margin, priced as if they were costs. Owning the layers means owning the capital decision, the sizing decision, and the operational responsibility that comes with all of it. What it removes is the part where you can't explain why the bill keeps climbing, because every layer in this list is a decision you made on purpose instead of a markup you discovered after the fact.

Adapted from Mark Tuck’s essay, “What an Owned Architecture Looks Like” (LinkedIn). Mark Tuck is Private AI Cloud | Strategy & Architecture @ Cloud Ingenuity. Figures cited from public AWS, Lambda, and RunPod pricing and Cast AI production telemetry.