Compiling Workflows into Databases: The Architecture That Shouldn't Work (But Does)

Abstract

What if everything you know about building distributed systems is backwards? What if instead of putting databases at the bottom of your architecture stack, you put them at the center—not just storing data, but actually orchestrating your application logic, managing your workflows and their state, and providing reliability guarantees?  A system that treats PostgreSQL not just as your database, but also as a durability layer for your application runtime. Instead of the typical pattern of building orchestration layers on top of databases, we compile workflow logic directly into database operations.

Drawing from our experience scaling systems at Reddit and Netflix and years of research at Stanford and MIT, this talk will discuss various use cases showing how using the open source Transact library from DBOS and durable computing concepts creates more reliable systems while reducing cost, processing time, and operational complexity.


Speaker

Jeremy Edberg

CEO @DBOS, Inventor of Chaos Engineering, Tech Editor for 'AWS for Dummies', Previously Founding Reliability Engineer @Netflix and Ops @Reddit

Jeremy is an angel investor and advisor for various incubators and startups, and the CEO of DBOS. He was the founding Reliability Engineer for Netflix and before that he ran ops for reddit as its first engineering hire. Jeremy also tech-edited the highly acclaimed AWS for Dummies, and he is one of the six original AWS Heroes. He is a noted speaker in serverless computing, distributed computing, availability, rapid scaling, and cloud computing, and holds a Cognitive Science degree from UC Berkeley.

Read more

Speaker

Qian Li

Co-founder, Architect @DBOS, Stanford CS Ph.D., Co-organizer of South Bay Systems

Qian is the co-founder of DBOS and co-leads engineering.  She completed her Ph.D. in Computer Science at Stanford University, advised by Christos Kozyrakis, and worked closely with Matei Zaharia (CTO of Databricks and creator of Spark) and Mike Stonebraker (creator of Postgres) on the DBMS-oriented Operating System (DBOS) academic project, which is the foundation of DBOS, Inc. She has broad interests in computer systems, databases, architecture, and abstractions for efficient and reliable cloud computing.  Prior to joining Stanford,  she received her B.Sc. in Computer Science and Technology from Peking University.

Read more

From the same track

Session

How to Build an Exchange: Sub Millisecond Response Times and 24/7 Uptimes in the Cloud

Monday Nov 17 / 10:35AM PST

These days it is possible to achieve fairly good performance on cloud provisioned systems. We discuss the design of a high performance, strongly consistent system which maintains constant service in the face of regular updates to core logic.

Speaker image - Frank Yu

Frank Yu

Director of Engineering @Coinbase, Previously Principal Engineer and Director @FairX

Session

Building Resilient Platforms: Insights from 20+ Years in Mission-Critical Infrastructure

Monday Nov 17 / 11:45AM PST

In this talk, Matthew will describe lessons learned from over 20+ years of building scalable, secure and stable infrastructure platforms for software in financial services (electronic trading, credit card processing etc.), the talk is relevant to anyone building platforms for mission-critic

Speaker image - Matthew Liste

Matthew Liste

Head of Infrastructure @American Express, Previously @JPMorgan Chase and @Goldman Sachs

Session

Unconference: Architectures You've Always Wondered About

Monday Nov 17 / 05:05PM PST

Session

Netflix’s Strategic Approach To Efficient, Scalable, and Resilient Data Deletion

Monday Nov 17 / 01:35PM PST

In large-scale systems, data deletion is a complex challenge that touches infrastructure, reliability, and performance. As data spreads across diverse data stores, ensuring timely and consistent deletion becomes increasingly difficult.

Speaker image - Vidhya Arvind

Vidhya Arvind

Tech Lead @Netflix Data Platform, Founding Member of Data Abstractions at Netflix, Previously @Box and @Verizon

Speaker image - Shawn Liu

Shawn Liu

Software Engineer @Netflix

Session

The Architecture of an Infinite Scroll

Monday Nov 17 / 03:55PM PST

Details coming soon.