From ms to µs: OSS Valkey Architecture Patterns for Modern AI

QCon San Francisco 2025

Session Architecture

From ms to µs: OSS Valkey Architecture Patterns for Modern AI

Wednesday Nov 19 / 02:45PM PST, Ballroom A at Hyatt Regency, San Francisco

Abstract

As AI applications demand faster and more intelligent data access, traditional caching strategies are hitting performance and reliability limits.

This talk presents architecture patterns that shift from milliseconds to microseconds using Valkey Cluster, an open-source, Redis-compatible, in-memory datastore. Learn when to use proxy-based versus direct-access caching, how to avoid hidden reliability issues in sharded systems, and how to optimize for high price-performance at scale. Backed by the Linux Foundation, Valkey offers rich data structures and community-driven innovation.

Whether you’re building GenAI services or scaling existing platforms, this session delivers actionable patterns to improve speed, resilience, and efficiency.

Interview

Imagine your in-memory system is a Formula 1 engine. We've made the engine incredibly fast, but now the racetrack itself is the bottleneck. My talk is about fixing that track. We will look at how extra latency hops are hidden speed bumps, and how expensive abstractions are like adding a heavy new spoiler that just adds drag and cost. We'll dive into the real trade-offs of building for microsecond latency, using patterns you can apply with the open-source, Redis-compatible Valkey. (Only milliseconds were harmed.)

Because speed is a feature, and inefficiency is a tax. When your Redis-style operations take microseconds, your real slowdown comes from network hops, proxies, and client behavior. This is what bloats your latency tail and your cloud bill. Focusing on this now is also a key strategic move. The open governance of Valkey after the 2024 Redis license change makes it a safe, vendor-neutral base to build on, so your efficiency gains aren't locked to a single vendor.

The biggest challenge is that a system on a whiteboard and a system under pressure tell two different stories. On paper, your design looks clean. But in reality, a single busy component can create a traffic jam that backs everything up. We will talk about how to spot these expensive problems before they become a 3 a.m. emergency.

Go back and play detective with your system. Draw a map of the journey your code takes to get data. Every stop on that map, every latency hop, and every "simple" abstraction has a price tag in both time and money. Start asking what each step costs. When you see the real price, you will find amazing ways to make things faster and cheaper.

QCon is where you learn from people who have the operational scars to prove their advice works. It's built on practitioner-led content, so you get unfiltered, real world lessons without the sales pitch. You leave with insights you can actually apply on Monday morning.

Topics

Architecture Open Source Microsecond-Scale In-memory Caching Sharded System Reliability Staff Plus Engineering Platform Engineering
76% senior dev or higher
1:11 speaker ratio
60+ practitioners

QCon San Francisco 2025 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.

Share

From the same track

Wednesday 19 November

10:35 Ballroom A Session AI/ML Producing the World's Cheapest Tokens: A How-to Guide Meryem Arik Co-Founder and CEO @Doubleword (Previously TitanML), Recognized as a Technology Leader in Forbes 30 Under 30, Recovering Physicist 11:45 Ballroom A Session Capacity Planning How Netflix Shapes our Fleet for Efficiency and Reliability Joseph Lynch, Argha C 13:35 Ballroom A Session AI Architecture Realtime and Batch Processing of GPU Workloads Joseph Stein Principal Architect of Research & Development @SS&C Technologies, Previous Apache Kafka Committer and PMC Member 14:45 Ballroom A Session Architecture From ms to µs: OSS Valkey Architecture Patterns for Modern AI Dumanshu Goyal Uber Technical Lead @Airbnb Powering $11B Transactions, Formerly @Google and @AWS 15:55 Ballroom A Session Platform Engineering Write-Ahead Intent Log: A Foundation for Efficient CDC at Scale Vinay Chella, Akshat Goel