Abstract
Ultrafast inference is a balancing act: users want tokens quickly, while providers need high aggregate throughput. The hardware choices that improve one can limit the other. This talk breaks down the bottlenecks behind that tradeoff: prefill processes input tokens in parallel, while decode generates output serially and is constrained by memory bandwidth. GPUs and Cerebras Wafer-Scale Engines have different strengths, so matching each stage to specialized hardware can push the speed-throughput Pareto frontier outward.
At Cerebras, we built a heterogenous disaggregated inference cluster that pairs GPU prefill with Cerebras decode to get 15x the speed at 5x the throughput. I’ll walk through how requests move through the system, how we scale and balance the two pools, and how we handle fault tolerance and discuss the lessons we learnt along the way.
For engineers who build Kubernetes platforms, schedulers, control planes, and distributed services, this is a practical look at how those skills apply to ultrafast AI inference. No prior ML infrastructure experience is required.
Interview
My session is about disaggregated inference: splitting LLM serving into prefill and decode and running each phase where it runs best. Many inference engineers know GPU-to-GPU disaggregation as a way to boost throughput. Fewer have thought about doing it across different kinds of hardware. GPUs are great at prefill and Cerebras is built for fast decode, so pairing them unlocks speeds neither gets alone. I want to challenge people to think about inference serving as a data center design problem, not just a chip problem.
Inference is where AI demand is exploding, and the way we build for it is changing. Heterogeneous compute, meaning matching specialized hardware to each workload, is still new, but I think it's how future clusters will be designed. Anyone building with AI should understand where serving infrastructure is headed.
Most teams don't have access to specialized hardware yet, but that's starting to change. Once you do, heterogeneous clusters bring their own design questions: what ratio of GPUs to Cerebras systems you need, how to keep capacity interchangeable as load shifts, and how to connect and operate two very different kinds of hardware as one serving fleet.
Try ultra-fast inference yourself and feel the difference compared to "normal fast." That gap really unlocks imagination. Then ask what new AI experiences you could build if tokens came back almost instantly.
$3,250, Conference (3 days). Current pricing ends November 20th. All pass options.
QCon San Francisco 2026 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.
Slides & video
Slides Not available
Part of the track
Architectures You've Always Wondered About Hosted by Daniela Miao Co-Founder & CTO @Momento, Systems & Observability Nerd, ex-Lightstep, ex-DynamoDBFrom the same track
Monday 16 November
10:35 Pacific DEKJ Session Distributed Systems Live Resharding Without Regret: Lessons from Building Valkey's Atomic Slot Migration Jacob Murphy Open Source Maintainer @Valkey & Software Engineer @Google Cloud's Memorystore Team Sharding is easy. Resharding under heavy load is notoriously difficult. How do you move gigabytes of state across live database nodes without dropping keys, blocking the main event loop, or breaking client abstractions? 11:45 Pacific DEKJ Session Architecture Architecting Nubank's Global Financial Infrastructure Cat Swetel Senior Director of Engineering @Nubank - General Manager for the Foundation Business Unit Instant payment frameworks are transforming global finance, but few institutions have faced the infrastructure scale required by Brazil's Pix network. 13:35 Pacific DEKJ Session AI/ML How to Build a Real-Time Voice Agent Sahil Yadav Software Engineer @Together AI - Building Real-Time Production Voice Agents A voice agent looks like a chatbot with a microphone. 14:45 Ballroom BC Session Building Ultrafast LLM Inference at Scale: Disaggregated Inference across GPUs and Cerebras WSEs Nancy Zheng VP of Engineering of AI Inference @Cerebras Ultrafast inference is a balancing act: users want tokens quickly, while providers need high aggregate throughput. The hardware choices that improve one can limit the other. 15:55 Ballroom BC Session Generation Got Cheap, Trust Didn’t: Scaling Agentic Changes from Prompt to Prod at Roblox Kushan Mehta, Jay Moorthi As AI agents write more of the code Roblox engineers push, the hard part is no longer generating changes; it’s knowing which ones to trust. 17:05 Seacliff D Unconference Unconference: Architectures You've Always Wondered About