This talk explores the application of reinforcement learning (RL) in large-scale recommendation systems to optimize user retention at scale - the true north star of effective recommendation engines. We'll discuss how RL can learn patterns and attribute future retention behavior to content consumed in current sessions, providing a more holistic approach than traditional methods.

We'll share insights from implementing this strategy in a production environment serving millions of users, highlighting significant improvements in key retention metrics. The presentation will address major challenges, including noisy attributions and proving causality within the system.

Our findings demonstrate the potential of RL in creating more sustainable, user-centric recommendation systems across various digital platforms, with important implications for the future of personalized content delivery.

Key Takeaways:

1. Using long-term rewards at different future horizons leads to incrementality in long-term metrics like Daily active users and sessions (will share results in the talk).
2. Optimal to try different horizons and approaches since duration is a tradeoff between causality of the lever and correlation with the final long-term metric.
3. High ROI of reward shaping to encode product intuition and strategy (e.g. private sharing)
4. Lessons on model architecture, co-investment in infrastructure (GPU inference, user history processing, sequence modeling) required to derive benefit at scale.