How Netflix Shapes our Fleet for Efficiency and Reliability

Netflix runs on a complex multi-layer cloud architecture made up of thousands of services, caches, and databases. As hardware options, workload patterns, cost dynamics and the Netflix products evolve, the cost-optimal hardware and configuration for running our services is constantly changing. It is no longer sufficient in modern cloud computing to buy large amounts of the same shape of computer and try to pack every workload on that with large fixed buffers, both for efficiency and availability reasons. It is also no longer sufficient for platform teams to work 1:1 with every service team to optimize their hardware selection, this does not scale.

This talk shows an alternative strategy, where each workload is placed on price-optimal hardware using automated understanding of hardware performance combined with workload characterization. Furthermore, as workload patterns shift, we can continuously re-evaluate and react for every cluster to ensure business outcomes for minimal spend.

We will start with understanding how we automatically model capacity requirements, including key concepts like service buffer allocation based on business criticality. Then we will show how we marry this understanding of workload needs with a deep understanding of AWS hardware performance and pricing to place each workload on efficient hardware. Finally, we will walk through the continuously running optimization loop, which monitors, detects changes, and re-shapes our fleet to maintain business outcomes as load patterns constantly change.

Even with all this planning, our systems still face unexpected load shifts that exceed modeled bounds, so to close we will briefly cover how we manage traffic demand and compute supply to ensure we can maintain availability while intelligently and rapidly injecting capacity into the right server groups to keep Netflix up and running and our customers happily streaming.