Speaker
Abstract
Is performance engineering more craft than machinery? How do you scale something that seems to require both domain-specific context and comprehensive knowledge across multiple levels of the software and hardware stack?
Back in 2017, I was itching for a new adventure after returning from maternity leave. The result was a brand new team and function at Twitter — the Performance Engineering, aka the IOP Team. This small but mighty assembly achieved a lot over the next 5 years, saving Twitter over 100 million dollars, creating some of the largest data processing jobs owned by infra, driving multiple company-wide initiatives, and releasing multiple open source projects and publications. And we had fun.
In this talk, I will recap that experience while summarizing some key lessons learned about effective performance engineering. Spoiler— it was all about platform building.
Topics
QCon San Francisco 2023 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.
Part of the track
Platform Engineering Done Well Hosted by Daniel Bryant Platform Engineer, Co-Author of "Mastering API Architecture", Java Champion, and InfoQ News ManagerFrom the same track
Tuesday 3 October
10:35 Ballroom A Session Platform Engineering Building a Successful Platform: Acceleration, Autonomy & Accountability Smruti Patel SVP of Engineering @apollographql Do we build a greenfield platform or do we incrementally centralize common foundations? Do we abstract away all complexity or provide upfront knobs and handles for composability? 11:45 Ballroom A Session Platform Engineering Understanding Platforms: What They Are, Why They Work, When to Use Them, How to Build Them Hazel Weakly Head of Infrastructure & Developer Experience; Director, Haskell Foundation; Infrastructure Witch of Hachyderm Technical concepts are something that are thought of, approached, and understood differently across engineers, managers, and executives. Bridging the gaps and providing understanding to a complex and nuanced topic across all three groups can sometimes feel impossible. 13:35 Seacliff D Unconference Unconference: Platform Engineering Done Well What is an unconference? An unconference is a participant-driven meeting. Attendees come together, bringing their challenges and relying on the experience and know-how of their peers for solutions. 14:45 Ballroom A Session Platform Engineering Building Better Platforms with Empathy: Case Studies and Counter-Examples David Stenglein Solo Consultant @Missing Mass, LLC with Over 28 Years in Systems, Software, and Consulting Break out of traditional IT roles with your internal platform. Build a product based on customer empathy and real needs to achieve broad adoption. 15:55 Ballroom A Session Platform Engineering Effective Performance Engineering at Twitter-Scale Yao Yue Founder & Chief Executive Officer @IOP Systems, Platform Engineer, Distributed System Aficionado, Cache Expert Is performance engineering more craft than machinery? How do you scale something that seems to require both domain-specific context and comprehensive knowledge across multiple levels of the software and hardware stack? 17:05 Ballroom A Session Platform Engineering How to Get Tech-Debt on the Roadmap Ben Hartshorne Principal Engineer @Honeycomb, Building Operable Systems with Resilience and Business Value Only doing product-led work can lead to an unmaintainable system with lots of downtime. Unfortunately, getting time to work on the things that would prevent that can be challenging.