How Netflix Ensures Highly-Reliable Online Stateful Systems

Under most stateless services are stateful databases, caches, and systems which form the bedrock applications are built on. These stateful systems have to be extremely reliable as even the slightest latency or availability blip can easily cascade up to the application layer due to data in-cast and query fan-out. With careful consideration, we can design reliable inter-process-communication (IPC), software deployment, and data models to handle millions of requests per second over petabyte scale datastores at single-digit millisecond latency.

In this talk we will start with the architecture of Netflix's stateful caches and databases, including how we capacity plan, bulkhead, and deploy software to our global, full-active, data topology. Keeping petabytes of state across four AWS regions and twelve fault isolation zones consistent and cost-efficient is challenging, but it allows us to meet our millisecond service-level-objectives for both reads and write operations at the level of availability Netflix applications demand.

Then, we will cover how we make stateful systems fully idempotent for both reads and writes, and use this to build novel resilience techniques into our stateful IPC layers using our KeyValue and TimeSeries (event) use cases as concrete examples. Developers at Netflix may see simple read and write APIs, but under the hood we implement advanced load balancing, routing, hedging and other resilience techniques to make those APIs function properly every time. Finally, we will conclude with how we design these systems to, when they do fail, fail as gracefully as possible to limit the blast radius of those failures and contain the business impact to Netflix. For example, most Netflix developers would rather sacrifice properties like immediate strong consistency via caching or partial shedding in emergency situations rather than be unavailable.