Presentation: Etsy Search: How we index and query 26 million one-of-a-kind items
In this talk talk I pull back the covers on the Etsy Search ecosystem and how we got here -- the good, the bad, and the funky.
The heart of our system is patched Solr/Lucene & ElasticSearch (and a lot of Bash) running on our boxes. I’ll dig into: signal collection from our web stack and mobile apps into Hadoop, generating machine learning ranking signals with Scalding, indexing content from 900 MySQL shards, sharding our Solr index in a novel way for performance, building a native search advertising platform, a 4-byte silver bullet for fighting JVM GC pauses, and how we continuously deploy this whole thing. I'll also talk about a bunch of things we tried that didn't work.
The goal of all this is to connect people: So our 40 million members can find unique goods that inspire them from our ever-changing set of one-of-a-kind items produced by a million individual sellers and makers around the world.
Aaron Gardner Elsewhere
Similar Talks
Tracks
Covering innovative topics
Monday, 3 November
-
Architectures You've Always Wondered about
The newest and biggest Internet architectures
-
Real World Functional
Putting functional programming concepts to work in the real world.
-
The Future of Mobile
The future of mobile and performance improvements
-
Continuous Delivery: From Heroics to Becoming Invisible
Continuous Delivery philosophies, cultures, hiccups, and best practices.
-
Unleashing the Power of Streaming Data
This track explores a variety of use-cases, platforms, and techniques for processing and analyzing stream data from the companies deploying them at scale!
-
Sponsored Solutions Track I
Tuesday, 4 November
-
Engineering for Product Success
Architectures that make products more successful
-
Reactive Service Architecture
Reactive, Responsive, Fault Tolerant and More.
-
Modern CS In the Real World
How modern CS tackles problems in the real world.
-
Applied Machine Learning and Data Science
Understand your big big data!
-
Deploying at Scale
Containerizing Applications, Discovering Services, and Deploying to the Grid.
-
Sponsored Solutions Track II
Wednesday, 5 November
-
Beyond Hadoop
Emerging Big Data Frameworks and Technology
-
Scalable Microservice Architectures
This track addresses the ways companies with hundreds of fine-grained web-services (e.g. Netflix, LinkedIn) manage complexity!
-
Java at the Cutting Edge
The latest and greatest in the Java ecosystem
-
Engineering culture
Successes and failures in creating an engineering culture.
-
Next gen HTML5 and JS
How Web Components, the Future of CSS, and more are changing the web.
-
Sponsored Solutions Track III