Parting the Clouds: The Rise of Disaggregated Systems

QCon San Francisco 2025

Session Architecture

Parting the Clouds: The Rise of Disaggregated Systems

Monday Nov 17 / 01:35PM PST, Pacific DEKJ at Hyatt Regency, San Francisco

Abstract

Cloud systems are undergoing an architectural shift. Traditional shared-nothing designs struggle to deliver the elasticity, availability, and operational simplicity that the cloud demands. The new generation of cloud systems (Amazon Aurora, Microsoft Socrates, Google AlloyDB, Snowflake, S3 Athena, Neon) embrace disaggregation, and decouple compute, storage, and increasingly, logs. This separation allows independent scaling, faster failover, and shared durable storage across tenants, but it also brings new performance tradeoffs and system design challenges.

In this talk, I will survey why disaggregation matters, what we have learned from the past decade of industrial systems, and where research and practice are headed next. I will explain the economic and operational motivations (elastic scaling, fault isolation, pooling) and discuss how disaggregation reshapes core database components like logging, recovery, and concurrency control. I'll discuss how design choices like "log-as-database", shared-storage replication, and caching tiers affect throughput, latency, and cost.

Finally, I’ll connect these modern architectures back to the timeless ideas in distributed systems, using Lamport's proposers, acceptors, and learners as a way to reason about how we disaggregate coordination, availability, durability, and computation. Viewed through this lens, disaggregation is our new Paxos: an architecture for separating roles so systems can scale, fail, and recover gracefully.

Topics

Architecture Distributed Systems Cloud Computing Databases
76% senior dev or higher
1:11 speaker ratio
60+ practitioners

QCon San Francisco 2025 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.

Share

From the same track

Monday 17 November

10:35 Ballroom A Session Architecture How to Build an Exchange Frank Yu Director of Engineering @Coinbase, Previously Principal Engineer and Director @FairX 11:45 Ballroom A Session Durability Compiling Workflows into Databases: The Architecture That Shouldn't Work (But Does) Jeremy Edberg, Qian Li 13:35 Pacific DEKJ Session Architecture Parting the Clouds: The Rise of Disaggregated Systems Murat Demirbas Principal Research Scientist @MongoDB Research, Previously Principal Applied Scientist @AWS and a Professor of Computer Science at the University at Buffalo (SUNY) 14:45 Ballroom A Session Platform Engineering Building Resilient Platforms: Insights from 20+ Years in Mission-Critical Infrastructure Matthew Liste Head of Infrastructure @American Express, Previously @JPMorgan Chase and @Goldman Sachs 15:55 Ballroom A Session Architecture Architecting a Centralized Platform for Data Deletion at Netflix Vidhya Arvind, Shawn Liu 17:05 Seacliff D Unconference Unconference: Architectures You've Always Wondered About