When Every Bit Counts: How Valkey Rebuilt Its Hashtable for Modern Hardware

Abstract

Ever wondered what happens when a bunch of performance-obsessed developers decide their blazing-fast database isn't quite blazing-fast enough? Join me for a deep-dive into how we threw out the beating heart of Valkey, a hashtable that was designed over 15 years ago, and replace it with something faster and more memory efficient. We'll dive into how careful attention to cache-line alignment, memory access patterns, and SIMD operations led to significant improvements in both memory efficiency and performance. Whether you're working on high-performance systems or just curious about modern hardware optimization, you'll come away with concrete insights into how subtle architectural choices can have outsized impacts on real-world performance.


Speaker

Madelyn Olson

Principal Engineer @AWS, Maintainer of the Open-Source Valkey Project

Madelyn is a maintainer of the Valkey project and a Principal Software Development Engineer at Amazon ElastiCache and Amazon MemoryDB, focusing on building secure and highly reliable features for the Valkey engine. In her free time, she enjoys taking in the natural beauty of the pacific northwest with long hikes and serene bike rides.

Read more
Find Madelyn Olson at:

Date

Wednesday Nov 19 / 01:35PM PST ( 50 minutes )

Location

Pacific DEKJ

Topics

Performance Valkey Cache System Design C

Share

From the same track

Session Rust

The High Performance Talk You Did Not Expect To Hear about Rust

Wednesday Nov 19 / 10:35AM PST

Rust makes things faster but it slows down engineers, right? This was our team’s assumption when we decided to rewrite our code from Kotlin into Rust. But, we were wrong in completely unexpected ways.

Speaker image - Ruth Linehan

Ruth Linehan

Software Engineer @Momento, Previously APIs/Webhooks @GitHub and @Puppet

Session

Instrumentation at Scale: Having Your Performance Cake and Eating It Too

Wednesday Nov 19 / 11:45AM PST

In high-performance code, a single misplaced counter increment can cost more than the operation it’s measuring. That creates a paradox: instrument too much and you slow the system down; instrument too little and you miss the insights you need to continuously deliver.

Speaker image - Brian Martin

Brian Martin

Co-founder and Software Engineer @IOP Systems, Focused on High-Performance Software and Systems, Previously @Twitter

Session Python

Python, Numba, and Algorithm Design: Building Efficient Models in Financial Services

Wednesday Nov 19 / 03:55PM PST

The popularity of Python means insurance and financial services companies have a growing body of actuaries, quantitative developers, and software engineers capable of building innovative and customized solutions for both data management and modeling.

Speaker image - Chad Schuster

Chad Schuster

Principal @Milliman Focusing on Risk Management, Modeling, and Technology Consulting Services

Session Performance

Accelerating Performance by Incrementally Integrating Rust Into Existing Codebase

Wednesday Nov 19 / 02:45PM PST

In order to improve the performance of existing applications and services, we can identify the most performance-critical pieces and reimplement them in Rust as opposed to completely rewriting the applications from scratch.

Speaker image - Lily Mara

Lily Mara

Staff Engineer @Discord, Author of "Refactoring to Rust", Previously Engineering Manager @OneSignal