How does Netflix maintain a seamless viewing experience for millions of users, especially during traffic spikes or when backend datastores are overloaded? Autoscaling can help during traffic spikes, but it costs money, takes a few minutes to kick in, and capacity may not always be available. Furthermore, if a downstream service or datastore is overloaded, autoscaling may exacerbate the problem. 

In this talk, we will cover service-level prioritized load shedding – our solution for prioritizing requests within a single application instance. This innovative solution ensures requests that are critical to user experience maintain high availability, and allows dynamically re-purposing non-critical capacity to serve critical traffic during times of duress. We will also discuss how we automated per-cluster tuning and validation of load shedding, enabling us to quickly deploy unique configurations to hundreds of clusters. 

Key Takeaways

1. Understand the evolution of load shedding techniques at Netflix and how service-level prioritization enhances user experience and reliability.
2. Gain insights from real-world applications and testing scenarios that demonstrate the effectiveness of prioritized load shedding.
3. Learn how platform engineering and service owners at Netflix effectively collaborated to build a generic prioritized load shedding library, efficiently roll it out, and automate tuning of load shedding thresholds.
4. Learn how we are continuing to improve the experience for service owners through our continued investments in making the load shedding capability more flexible and easier to operate.