This talk will present challenges, solutions, and engineering practices around building distributed systems on top of programmable network hardware through the lens of building the Oxide rack-scale computer. The programmable network processors available today in data center switches and network interface cards present the opportunity for enormous gains in flexibility, efficiency, and comprehensibility - but it's not a free lunch. Leveraging programmable network processors without losing many critical network functions that come for free with more traditional fixed-function network processing units creates new challenges in software development workflows from prototyping to continuous integration and everything in between.

In this talk, I'll cover what we've learned and open source tooling we've developed while delivering a product on top of the Tofino 2 switch processor, what we're gearing up to do at the edge with programmable network interface processors, and how specialized languages for network processing like P4 can fit into everyday development workflows for implementing network infrastructure applications.