The Journey to a Million Ops / Sec / Node in Venice

Venice is an open-source derived data platform developed by LinkedIn. It is used mainly for ML feature storage, which requires the ability to refresh data at very high throughput, and to look it up with low latency.

The lion share of the project is in Java, although it also leverages RocksDB and ZSTD via JNI. On the Java side, every bit of performance which can be squeezed is fair game.

After briefly presenting Venice, this talk then deep dives into some of the tricks we have employed in our relentless pursuit to lower read latency and to reach 1M operations per second per node.

Speaker

Alex Dubrouski

Technical Lead of Server Performance Team @LinkedIn

Alex joined LinkedIn in 2020 and since then has been working on optimizing server-side performance primarily for Java based applications. Alex also contributes performance patches to OSS projects like OpenJDK, Log4J and Venice.

Before LinkedIn, Alex spent 4 years working on performance and infrastructure at Pandora Media.

Read more
Find Alex Dubrouski at:

Speaker

Gaojie Liu

Senior Staff Software Engineer @LinkedIn, Open Source Contributor @Venice, a Massive Scalable Derived Data Platform

Gaojie joined LinkedIn in 2016 and since then, he has been working on Venice, a massively scalable derived data platform. He has been working on various aspects of Venice, such as new feature development, performance tuning and architecture evolution.

Prior to LinkedIn, Gaojie had worked in Yahoo for about 5 years, where he was mainly developing Yahoo Search Gateway platform.

Read more
Find Gaojie Liu at:

Date

Tuesday Oct 3 / 11:45AM PDT ( 50 minutes )

Location

Pacific DEKJ

Topics

Distributed Systems Performance Horizontal Scalable Feature Store Java Data Platforms

Share

Share Share

From the same track

Session Java

How Netflix Really Uses Java

Tuesday Oct 3 / 10:35AM PDT

Netflix is (in)famous for the way we develop our systems. Micro Services, RxJava, Hystrix and Spring Cloud Netflix are just a few things that developers often associate with Netflix.

Speaker image - Paul Bakker
Paul Bakker

Java Platform @Netflix, Java Champion, and Co-Author of "Java 9 Modularity"

Session Java

Crafting the World's Most Advanced Computer Chips with Java

Tuesday Oct 3 / 01:35PM PDT

In today's digital age, the demand for faster, more efficient computer chips in our devices is ever-growing. The intricate process of chip manufacturing hinges on cutting-edge machines and sophisticated software, with precision down to the nanometer.

Speaker image - Johan Janssen
Johan Janssen

Software Architect @ASML

Session jvm

Optimizing JVM for the Cloud: Strategies for Success

Tuesday Oct 3 / 03:55PM PDT

Embracing cloud-native is more than just a trend; it's a strategic move for modern businesses. As organizations pivot from monolithic structures to agile micro-services, the role of the JVM becomes increasingly significant.

Speaker image - Tobi Ajila
Tobi Ajila

Eclipse OpenJ9 JVM Engineer @IBM, Optimizing JVMs for Cloud Environments

Session Java

Performance and Developer Productivity

Tuesday Oct 3 / 05:05PM PDT

Details coming soon.

Speaker image - Heather VanCura
Heather VanCura

Senior Director, Standards, Strategy & Architecture @Oracle, Director & Chairperson at Java Community Process (JCP) Program, MySQL Community/DevRel, and Board Member

Session

Unconference: JVM Trends

Tuesday Oct 3 / 02:45PM PDT

What is an unconference? An unconference is a participant-driven meeting. Attendees come together, bringing their challenges and relying on the experience and know-how of their peers for solutions.