Functional programming embraces a kind of conceptual purity which often finds itself at odds with the demands of real world systems. Functional programming works best in a kind of ideal computing environment: heaps are endless, file descriptors don’t exist, partial failure cannot happen, and demands can never exceed a system’s capacity. Inevitably, the real world won’t accommodate: as a system scales, so does this apparent rift.
 
How do we mend this rift? Where do we have to break abstractions (slightly) so that they can deal with real world demands? Better: where can we use FP’s strengths to make the whole situation more manageable? How do we build systems that are at the same time functional — built from simple parts we can combine together, simple to reason about, and reusable — and functional — they stand a chance of being usable in our real, messy environments?
 
I’m going to talk about our experiences with functional programming (using Scala) at Twitter. Where and how we’ve made productive use of functional techniques; and also where we’ve failed. We’ll examine the development of a few core abstractions, paying particular attention to how they’ve held up to real world demands. Finally, we’ll talk a little about how the language has scaled with our organization — how our people scale informs our use of Scala.