Is performance engineering more craft than machinery? How do you scale something that seems to require both domain-specific context and comprehensive knowledge across multiple levels of the software and hardware stack?
Back in 2017, I was itching for a new adventure after returning from maternity leave. The result was a brand new team and function at Twitter — the Performance Engineering, aka the IOP Team. This small but mighty assembly achieved a lot over the next 5 years, saving Twitter over 100 million dollars, creating some of the largest data processing jobs owned by infra, driving multiple company-wide initiatives, and releasing multiple open source projects and publications. And we had fun.
In this talk, I will recap that experience while summarizing some key lessons learned about effective performance engineering. Spoiler— it was all about platform building.
Platform Engineer, Distributed System Aficionado, Cache Expert, and the Founder of IOP Systems
Yao Yue is a platform engineer with specialties in caching, distributed systems, and performance engineering. She worked at Twitter for 12 years, first led the Cache Team and later created the Performance Team. After November 2022, she co-founded IOP Systems, a company that is working to improve software efficiency and reliability via smart performance engineering, with her teammates from Twitter.