In-memory caching in a distributed setting is ubiquitous, of which users usually expect both high throughput and low latencies. Meanwhile, to actually achieve them requires understanding the details and subtleties in low-level system behavior, including memory management, threading model, locking, network I/O, etc.

Pelikan is a framework to implement distributed caches such as Memcached and Redis. This talk discusses the system aspects that are important to the performance, especially tail latencies, of such services. It also covers the decisions we made in Pelikan, and how they help us reason about performance even before we run benchmarks.