Presentation: Parsing JSON Really Quickly: Lessons Learned

Track: Bare Knuckle Performance

Location: Pacific DEKJ

Duration: 5:25pm - 6:15pm

Day of week: Monday

Share this on:

Abstract

Our disks and networks can load gigabytes of data per second; we feel strongly that our software should follow suit. Thus we wrote what might be the fastest JSON parser in the world, simdjson. It can parse typical JSON files at speeds of over 2 GB/s on single commodity Intel core with full validation; it is several times faster than conventional parsers.

How did we go so fast? We started with the insight that we should make full use of the SIMD instructions available on commodity processors. These instructions are everywhere, from the ARM chip in your smartphone all to way to server processors. SIMD instructions work on wide registers (e.g., spanning 32 bytes): they are faster because they process more data using fewer instructions. To our knowledge, nobody had ever attempted to produce a full parser for something as complex as JSON by relying primarily on SIMD instructions. And many people were skeptical that a full parser could be done fruitfully with SIMD instructions. We had to develop interesting new strategies that are generally applicable. In the end, we learned several lessons. Maybe one of the most important lesson is the importance of a nearly obsessive focus on performance metrics. We constantly measure the impact of the choices we make.

Speaker: Daniel Lemire

Professor and Department Chair at TELUQ - Université du Québec

Daniel Lemire is a computer science professor at the Université du Québec (TELUQ).  He has written over 70 peer-reviewed publications, including more than 40 journal articles.  He has held competitive research grants for the last 15 years. He serves on the program committees of leading computer science conferences. During the 2016-2017 NSERC Discovery Grant competition, he received a rating of outstanding for the excellence of the researcher.  

He is a long-time social media user: his blog has thousands of readers and was featured on Slashdot, Reddit and Hacker News.  He was one of the first Twitter users: @lemire.

Find Daniel Lemire at

Tracks

Monday, 11 November

Tuesday, 12 November

Wednesday, 13 November