How Databricks Solved Their Hybrid Cloud Connectivity Challenges

QCon San Francisco 2023

Sponsored session

How Databricks Solved Their Hybrid Cloud Connectivity Challenges

Tuesday Oct 3 / 10:35AM PDT, Pacific LM

Abstract

Hybrid cloud is the industry’s latest buzzword for client-server software architectures that allow your customers to bring their own cloud. One of the largest technical hurdles to adopting this architecture is allowing your hosted control plane to speak to APIs in the data plane. This used to be easy when the data plane ran in your own VPC, but now that the data plane runs in your customer’s network it creates a brand new set of networking challenges.

Asking every customers to set up VPNs, VPC peering across public clouds, or to poke holes in their firewalls is quickly a non-starter. How do we solve this problem across all environments with zero configuration from customers? What we really want is a way to create ingress at scale into each of our customer’s networks.

In this talk, we’ll show you how Databricks solved the problem of creating ingress at scale for all the data nodes that run in their customer’s environments. We’ll walk through creating ingress to APIs into Kubernetes clusters, no matter where it’s running, even if you’re not operating it yourself.

Sponsored session

Session sponsored by

Ngrok is simplified, API-first ingress-as-a-service.

76% senior dev or higher
1:11 speaker ratio
60+ practitioners

QCon San Francisco 2023 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.

Share

From the same track

Tuesday 3 October

10:35 Pacific LM Sponsored How Databricks Solved Their Hybrid Cloud Connectivity Challenges Alan Shreve Founder and CEO @Ngrok 11:45 Pacific LM Sponsored Service Mesh for Less Louis Ryan, Alex Ly 13:35 Pacific LM Sponsored Optimizing a Query - An Optimization Tale Oren Eini CEO @RavenDB 14:45 Pacific LM Sponsored Streaming Events? You Should be Storing Them Yves Lorphelin, Matt Macchia 15:55 Pacific LM Sponsored TiDB: Inside the Journey to Build a Cloud-Native, Distributed SQL Database Li Shen SVP & Founding Engineer @PingCAP 17:05 Pacific LM Sponsored How the Largest Latin American Bank Reduced Software Delivery Lead Time on a Team with 16,000 Engineers Manuela Leoni Head of Product @Itaú Unibanco