Speaker
Abstract
Data storage and access at the edge delivers massive performance gains by reducing location-sensitive latency. Cloudflare operates a distributed cross-region database architecture at the edge, serving over 50TB of online transactional data over millions of requests across our entire security and web performance product suite. Storing and managing relational data at the edge carries a unique set of challenges dictated by timeless CAP constraints and highly variable load conditions, requiring careful tradeoffs.
How can we minimize replication lag, choose replication strategies (asynchronous, streaming, logical, custom tooling) that balance consistency and performance, optimize request routing, and even speed up split brain cluster resynchronization? Achieving high availability storage performance at the edge demands innovative techniques involving custom performance optimizations across our anycast BGP network, cluster management layer, and down to the core Postgres recovery system.
So, you still want to build your database architecture at the edge?
You’ll learn about Cloudflare’s entire edge database architecture, unique challenges and practices for data replication, failover and recovery, and custom performance techniques. We'll also peek into the exciting future landscape of edge storage.
Interview
Building a managed Postgres environment at Cloudflare, working on building edge applications, and contributing to Postgres and related open-source projects.
To help everyone understand the challenges of data at the edge and what are the ways to solve them.
Someone who is a Systems Engineer, SRE or even a developer who is curious in understanding edge computing and how to enable it through Postgres.
I would expect attendees will learn something about the limitations/challenges of data at the edge, and opportunities if done right.
Topics
QCon San Francisco 2023 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.
Part of the track
Architectures You've Always Wondered About Hosted by Wes Reisz Technical Principal @Thoughtworks, 16-Time QCon Chair, & Creator of The InfoQ PodcastFrom the same track
Monday 2 October
10:35 Ballroom BC Session Serverless AWS Lambda Under the Hood Mike Danilov Senior Principal Engineer @AWS Lambda AWS Lambda is a serverless compute service running at a massive scale! Supporting packages of up to 10GiB while allowing over 15K new containers per second (for a single customer) and serving millions of TPS across millions of unique workflows is a challenging problem. 11:45 Ballroom BC Session Database Relational Data at the Edge Justin Kwan, Vignesh Ravichandran Data storage and access at the edge delivers massive performance gains by reducing location-sensitive latency. 13:35 Ballroom BC Session K8s NIST 800-207A: Implementing Zero Trust Architecture Zack Butcher Founding Engineer @Tetrateio & NIST co-author on security, prev core services @GoogleCloud Zero Trust is all about replacing implicit trust based on perimeter security and network access with explicit trust based on identity and runtime authorization. 14:45 Ballroom BC Session Distributed Systems Managing 238M Memberships at Netflix Surabhi Diwan Senior Software Engineer @Netflix Have you ever wondered what goes on behind the scenes when you sit back, relax and watch Netflix? How does Netflix own and operate their system of record for all members making sure they continue to be in good standing and get the best experience possible? 15:55 Seacliff D Unconference Unconference: Architectures You've Always Wondered About What is an unconference? An unconference is a participant-driven meeting. Attendees come together, bringing their challenges and relying on the experience and know-how of their peers for solutions. 17:05 Ballroom BC Session Recommender Systems Unpacking how Ads Ranking Works @Pinterest Aayush Mudgal Senior Machine Learning Engineer @Pinterest, Focusing on Privacy Safe Recommender Systems, IIT Kanpur Alumnus In this session, we delve into the dynamic world of social media advertising. Facebook, Snap, Pinterest, Twitter, and many others generate the majority of their revenue from targeted ads.