How Databricks Solved Their Hybrid Cloud Connectivity Challenges

Hybrid cloud is the industry’s latest buzzword for client-server software architectures that allow your customers to bring their own cloud. One of the largest technical hurdles to adopting this architecture is allowing your hosted control plane to speak to APIs in the data plane. This used to be easy when the data plane ran in your own VPC, but now that the data plane runs in your customer’s network it creates a brand new set of networking challenges.

Asking every customers to set up VPNs, VPC peering across public clouds, or to poke holes in their firewalls is quickly a non-starter. How do we solve this problem across all environments with zero configuration from customers? What we really want is a way to create ingress at scale into each of our customer’s networks.

In this talk, we’ll show you how Databricks solved the problem of creating ingress at scale for all the data nodes that run in their customer’s environments. We’ll walk through creating ingress to APIs into Kubernetes clusters, no matter where it’s running, even if you’re not operating it yourself.