Building Ultrafast LLM Inference at Scale: Disaggregated Inference across GPUs and Cerebras WSEs​

QCon San Francisco 2026

Session

Building Ultrafast LLM Inference at Scale: Disaggregated Inference across GPUs and Cerebras WSEs​

Monday Nov 16 / 02:45PM PST (50 minutes), Ballroom BC at Hyatt Regency, San Francisco

Register

$3,250, Conference (3 days)
Current pricing ends November 20th

Abstract

Ultrafast inference is a balancing act: users want tokens quickly, while providers need high aggregate throughput. The hardware choices that improve one can limit the other. This talk breaks down the bottlenecks behind that tradeoff: prefill processes input tokens in parallel, while decode generates output serially and is constrained by memory bandwidth. GPUs and Cerebras Wafer-Scale Engines have different strengths, so matching each stage to specialized hardware can push the speed-throughput Pareto frontier outward.

At Cerebras, we built a heterogenous disaggregated inference cluster that pairs GPU prefill with Cerebras decode to get 15x the speed at 5x the throughput. I’ll walk through how requests move through the system, how we scale and balance the two pools, and how we handle fault tolerance and discuss the lessons we learnt along the way.

For engineers who build Kubernetes platforms, schedulers, control planes, and distributed services, this is a practical look at how those skills apply to ultrafast AI inference. No prior ML infrastructure experience is required.

Interview

My session is about disaggregated inference: splitting LLM serving into prefill and decode and running each phase where it runs best. Many inference engineers know GPU-to-GPU disaggregation as a way to boost throughput. Fewer have thought about doing it across different kinds of hardware. GPUs are great at prefill and Cerebras is built for fast decode, so pairing them unlocks speeds neither gets alone. I want to challenge people to think about inference serving as a data center design problem, not just a chip problem.

Inference is where AI demand is exploding, and the way we build for it is changing. Heterogeneous compute, meaning matching specialized hardware to each workload, is still new, but I think it's how future clusters will be designed. Anyone building with AI should understand where serving infrastructure is headed.

Most teams don't have access to specialized hardware yet, but that's starting to change. Once you do, heterogeneous clusters bring their own design questions: what ratio of GPUs to Cerebras systems you need, how to keep capacity interchangeable as load shifts, and how to connect and operate two very different kinds of hardware as one serving fleet.

Try ultra-fast inference yourself and feel the difference compared to "normal fast." That gap really unlocks imagination. Then ask what new AI experiences you could build if tokens came back almost instantly.

Register

$3,250, Conference (3 days). Current pricing ends November 20th. All pass options.

76% senior dev or higher
1:11 speaker ratio
60+ practitioners

QCon San Francisco 2026 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.

Slides & video

Slides Not available

Share

From the same track

Monday 16 November

10:35 Pacific DEKJ Session Distributed Systems Live Resharding Without Regret: Lessons from Building Valkey's Atomic Slot Migration Jacob Murphy Open Source Maintainer @Valkey & Software Engineer @Google Cloud's Memorystore Team 11:45 Pacific DEKJ Session Architecture Architecting Nubank's Global Financial Infrastructure Cat Swetel Senior Director of Engineering @Nubank - General Manager for the Foundation Business Unit 13:35 Pacific DEKJ Session AI/ML How to Build a Real-Time Voice Agent Sahil Yadav Software Engineer @Together AI - Building Real-Time Production Voice Agents 14:45 Ballroom BC Session Building Ultrafast LLM Inference at Scale: Disaggregated Inference across GPUs and Cerebras WSEs​ Nancy Zheng VP of Engineering of AI Inference @Cerebras 15:55 Ballroom BC Session Generation Got Cheap, Trust Didn’t: Scaling Agentic Changes from Prompt to Prod at Roblox Kushan Mehta, Jay Moorthi 17:05 Seacliff D Unconference Unconference: Architectures You've Always Wondered About