It’s one of the hardest tradeoffs engineers have to make: build or buy. The scale of the internet and the rapid growth that it enables makes this question more difficult for many companies.

Building infrastructure components instead of buying them from vendors was once the domain of only the very largest of internet operations like Google and Facebook. Those companies not only build their own software, including low level things like compilers, they are also building their own hardware and networking. But increasingly we see non-Google, non-infrastructure companies building their own infrastructure software for their operations.

We struggle with this decision regularly at Uber. In this talk, I’d like to explore why this tradeoff is so difficult, and make some recommendations for both vendors and users.

At first the decision sounds easy, for engineers at least. When you even suggest building just about any software, you’ll get plenty of advice about how you suffer from “not invented here” syndrome and that you are re-inventing the wheel.

It’s hard for engineers because the point of being a software engineer is to write more software. Or is it? Perhaps the point of software engineering at this point in history is to endlessly debug and wrangle someone else’s software until it does what you want. And this is why build vs. buy is not an obvious tradeoff. It’s more like “build and easily support” vs. “buy and be at the mercy of a vendor”. Open source projects are vendors of a sort. The price might be $0 to install, but there are hidden costs.

The global and large scale nature of the internet has put some new requirements on software. There are no maintenance windows or acceptable downtime at all. If software breaks, no vendor is going to care as much about fixing it as you do. This makes people do surprising things, like re-invent wheels.

After enough time, most of the currently re-invented wheels will eventually become commodities that are cheap and reliable enough for everybody. But new needs will always arise that need more software written to address them. I think the future of this kind of software will look more like an open source collaboration between large users to build big important things. There are more recent examples of industry collaboration for projects like this. The role of vendors in this new world is unclear, but I think they’ll start to seem much more like development partners, rather than a helpdesk.

The Uber platform has been growing dramatically, now facilitating transportation in over 330 cities around the world. As the company has grown, the architecture that powers this platform has gone through many changes. From its simple roots as a PHP program, Uber has grown into a complex distributed system deployed across multiple datacenters using multiple databases and programming languages. This talk will cover the evolution of Uber's architecture and some of the systems we've built to handle the current scaling challenges.