LIquid: A Large-Scale Relational Graph Database

We describe LIquid(1 2), the graph database built to host LinkedIn. At LinkedIn, LIquid serves a ~15Tb graph at ~2M QPS. We describe the graph data model precisely in terms of the relational model and explain how to query it with Datalog. We describe how to do this in memory at scale.

Speaker

Scott Meyer

Distinguished Software Engineer @LinkedIn, Creator of the Graph Database, LIquid, Metaweb/freebase Alum

40 years of adventures in software: computer graphics, networking, GUI application development, object-oriented databases, a Java language implementation and now I seem to have settled on graph databases.  I led the storage team at Metaweb. We built freebase.com, the world-writable graph of all common knowledge which became Google's knowledge graph (and also Microsoft's).  In 2014 I moved to LinkedIn to start a next-generation relational graph database project called LIquid:

https://engineering.linkedin.com/blog/2020/liquid-the-soul-of-a-new-graph-database-part-1

https://engineering.linkedin.com/blog/2020/liquid--the-soul-of-a-new-graph-database--part-2

LIquid has been in production at LinkedIn for several years

Read more
Find Scott Meyer at:

Date

Monday Oct 2 / 10:35AM PDT ( 50 minutes )

Location

Ballroom A

Topics

Graph Databases Database Distributed Systems

Share

Share Share

From the same track

Session Stream Processing

Streaming Databases: Embracing the Convergence of Stream Processing and Databases

Monday Oct 2 / 01:35PM PDT

Streaming databases have gained significant attention in recent years. From its name, it is evident that a streaming database combines the power of stream processing and databases.

Speaker image - Yingjun Wu
Yingjun Wu

Founder and CEO @RisingWave Labs, Previously Engineer @AWS Redshift & Researcher @IBM Research Almaden

Session Distributed Systems

Redesigning OLTP for a New Order of Magnitude

Monday Oct 2 / 02:45PM PDT

The world is becoming more transactional. From colocation and server rental to serverless and usage-based billing. From coal to clean energy and smart meters that arbitrage solar prices 1440 times a month instead of monthly. Not to mention FedNow or the tsunami of instant payments.

Speaker image - Joran Greef
Joran Greef

Founder and CEO @TigerBeetle

Session Data Lakes

Incremental Data Processing with Apache Hudi

Monday Oct 2 / 03:55PM PDT

Incremental Data Processing is an emerging style of data processing gathering attention recently that has the potential to deliver orders of magnitude speed and efficiency over traditional batch processing on data lakes and data warehouses.

Speaker image - Saketh Chintapalli
Saketh Chintapalli

Software Engineer @Uber, Bringing Incremental Data Processing to Data Warehouse Models

Speaker image - Bhavani Sudha Saktheeswaran
Bhavani Sudha Saktheeswaran

Distributed Systems Engineer @Onehouse, Apache Hudi PMC, Ex-Moveworks, Ex-Uber, Ex-Linkedin

Session Architecture

Sleeping at Scale - Delivering 10k Timers per Second per Node with Rust, Tokio, Kafka, and Scylla

Monday Oct 2 / 05:05PM PDT

As a part of OneSignal’s no-code Journeys system, we knew that we would need a way to store billions of timers.

Speaker image - Lily Mara
Lily Mara

Engineering Manager @OneSignal, Author of "Refactoring to Rust"

Speaker image - Hunter Laine
Hunter Laine

Software Engineer @OneSignal

Session Data

PRQL: A Simple, Powerful, Pipelined SQL Replacement

Monday Oct 2 / 11:45AM PDT

Most databases use SQL as the interface to access relational data. Because of that, we associate SQL to be the language of relational algebra. But its affinity with the English language and unclear and inconsistent semantics leave a lot of space for improvements.

Speaker image - Aljaž Mur Eržen
Aljaž Mur Eržen

Compiler Developer @EdgeDB & PRQL Maintainer