Stripe’s Docdb: How Zero-Downtime Data Movement Powers Trillion-Dollar Payment Processing

In the presentation titled Stripe’s Docdb: How Zero-Downtime Data Movement Powers Trillion-Dollar Payment Processing, Jimmy Morzaria, a Staff Software Engineer at Stripe, delves into Stripe's custom-built document database, DocDB, and its data movement platform, which enables scalable and reliable payment processing.

Key Points:

• Scale and Reliability: Stripe processes over $1 trillion in payments annually using DocDB, handling over 5 million queries per second across more than 2000 database shards with 5.59 reliability.
• Architecture and Evolution: Originally, Stripe started with MongoDB which evolved into DocDB to handle exponential growth by introducing a database proxy service and a control plane.
• Zero-Downtime Data Movement: The engineering approach to data movement ensures zero downtime through strategies like version gating and bidirectional replication, crucial for horizontal scalability and database upgrades.
• Use Cases and Flexibility: The platform allows Stripe to add or merge database shards, facilitate version upgrades, and support transitions between multi-tenancy and single tenancy environments.

Design Principles:

• Consistency: Ensures data integrity across source and target shards during migration.
• Availability: Minimal downtime is essential due to the critical nature of payment processing.
• Performance: Preserve database performance and throughput during data migrations.
• Scalability: Adaptable to varying sizes and numbers of database shards and migrations.

Takeaways:

• Reliability is non-negotiable: Stripe prioritizes API reliability as a key differentiator, with the data movement platform designed to maintain application uptime.
• Invest in foundations: Stripe's data movement platform supports a variety of missions, from scaling to version upgrades, maintaining reliability and performance.

This is the end of the AI-generated content.

Stripe processes over $1 trillion in payments annually with industry-leading reliability, powered by its custom-built document database, DocDB, built on top of open source MongoDB. Stripe's DocDB serves over five million queries per second from Stripe’s product applications. Our deployment is also highly customized to provide low latency and diverse access, with 10,000+ distinct query shapes over petabytes of important financial data that lives in 5,000+ collections distributed over 2,000+ database shards.

This session unveils the engineering behind Stripe’s DocDB and dives deep into its Data Movement Platform, a foundational primitive that allows us to migrate terabytes of data across database shards at lightning speed and is key to provide a durable, reliable, scalable, and efficient database-as-a-service to product teams at Stripe. We’ll explore the challenges of scaling the online database backing a trillion-dollar payment infrastructure, ensuring data consistency and high reliability. Key learnings include optimizing bulk data ingestion in MongoDB and implementing a traffic switch protocol for seamless, zero-downtime data movement across database shards. Attendees will gain practical insights for building high-availability database systems and navigating complex migrations without compromising performance.