A Million People want a Million Things and They Want Them Now: How to Meet Personalised User Demands in Realtime

QCon San Francisco 2022

Sponsored session Kafka

A Million People want a Million Things and They Want Them Now: How to Meet Personalised User Demands in Realtime

Tuesday Oct 25 / 10:35AM PDT, Marina

Abstract

As the amount of data in Fintech and cryptocurrency applications has increased, Kafka has become the norm for wrangling this surge at speed and scale. However, the need to send personalized data streams to users has become paramount. Your mobile day trader only wants what they can use, and neither you nor they want the data bill for everything they are not using. You also don’t want topics per user. This is where it started, but as we’ve encountered more Kafka users, the needs are increasingly similar - whether it's large-scale chat systems, betting applications, or social media. Data access patterns are not but all need to be handled fairly. In this session, I’ll walk you through designing a scaling consumer system to allow for customized user streams in high cardinality data topics, all while minimizing the data burden on user devices. I’ll be demonstrating a few use cases of the system and running through the various tradeoffs that can be made to both improve user latency and add high availability to the system. Follow along to learn about the key tradeoff of user experience vs cost, latency vs flexibility, and the increasing demands of user-facing systems.

Interview

Topics

Kafka Streaming Blockchain Database Development

Sponsored session

76% senior dev or higher
1:11 speaker ratio
60+ practitioners

QCon San Francisco 2022 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.

Share

From the same track

Tuesday 25 October

10:35 Marina Sponsored Kafka A Million People want a Million Things and They Want Them Now: How to Meet Personalised User Demands in Realtime Roman de Oliviera Solutions Engineer @Ably 11:50 Pacific LM Sponsored Reliability How to Build Reliable Systems Under Unpredictable Conditions Benjamin Wilms Co-founder & Product minded engineer @Steadybit 14:55 Marina Sponsored Architecture Architecting Your Data Platform Across Clusters, Regions, and Clouds Adit Madan Director of Product Management @Alluxio 16:10 Marina Sponsored Event-Driven Architecture A Guide to the Event-Native World of Open Standards: AsyncAPI, CloudEvents, and Shared Signals & Events Linus Hakansson Vice President of Product @ Gravitee 17:25 Marina Sponsored Sustainability Bringing Green, Sustainable Software Solutions Into the Enterprise Adam Jordan Distinguished Software Engineer @Shell