Warning message

  • The service having id "twitter" is missing, reactivate its module or save again the list of services.
  • The service having id "facebook" is missing, reactivate its module or save again the list of services.
  • The service having id "google_plus" is missing, reactivate its module or save again the list of services.
  • The service having id "linkedin" is missing, reactivate its module or save again the list of services.

Track: Streaming Data @ Scale


Day of week:

Stream processing at scale has become essential for many practical applications: demand and supply forecasting in a market place, fraud detection, ad-hoc experiments, and real-time recommendations, just to name a few. Building and operating a stream system that handles high volume of data also demands many areas of expertise, such as distributed systems, applied statistics, and system optimization. This track will explore interesting stories of applying stream systems to solve real-world problems with focuses on effective techniques and promising new trends.

Track Host:
Danny Yuan
Real-time Streaming Lead @Uber
Danny Yuan is a software engineer in Uber. He's currently working on streaming systems for Uber's logistics platform. Prior to joining Uber, he worked on building Netflix's cloud platform. His work includes predictive autoscaling, distributed tracing service, real-time data pipeline that scaled to process hundreds of billions of events every day, and Netflix's low-latency crypto services.
10:35am - 11:25am

Open Space
11:50am - 12:40pm

by Danny Yuan
Real-time Streaming Lead @Uber

The realtime system is the heart and soul of Uber's logistic platform. It is responsible for fulfilling requests from riders while striving to maximize driver utilization and minimize rider cost. To make our realtime system efficient and intelligent, we need to extract deep and timely insights from our carefully curated data. We also have to make the insights easily accessible for both people and machines to consume in real time. This talk will discuss how stream processing is used in Uber's...

1:40pm - 2:30pm

by Gian Merlino
Cofounder @Imply

Stream processors are used for many things, including computations, creating derived streams, and taking actions based on streaming data. But one other common use case is keeping databases up to date. This presents a number of challenges, including how to load balance, how to avoid duplicating data and dropping data, and how to handle backfills. We'll talk about some of the possible approaches, their pros and cons, and look at how they are used in real world systems like Kafka, Storm, Samza...

2:55pm - 3:45pm

by Helena Edelson
VP of Product Engineering @Tuplejump

This talk will address new architectures emerging for large scale streaming analytics. Some based on Spark, Mesos, Akka, Cassandra and Kafka (SMACK) and other newer streaming analytics platforms and frameworks using Apache Flink or GearPump. Popular architecture like Lambda separate layers of computation and delivery and require many technologies which have overlapping functionality. Some of this results in duplicated code, untyped processes, or high operational overhead, let alone the cost...

4:10pm - 5:00pm

by Jerry Chen
Lead developer of Facebook's Stylus

At Facebook, we have diversified needs for a high performance, versatile, scalable and fault-tolerant stream processing system. We built a low level Stream Processing system called Stylus to enable developers quickly build stream processing applications for these needs. Stylus is currently being used by many teams at Facebook at scale. One of such use cases is currently processing 100's of billions of events per day. In this talk, I will talk about the architecture of Stylus, and how we...

5:25pm - 6:15pm

by Karthik Ramasamy
Engineering Manager and Technical Lead for Real Time Analytics @Twitter

Storm has long served as the main platform for real-time analytics at Twitter. However, as the scale of data being processed in real- time at Twitter has increased, along with an increase in the diversity and the number of use cases, many limitations of Storm have become apparent. We need a system that scales better, has better debug-ability, has better performance, and is easier to manage – all while working in a shared cluster infrastructure. We considered various alternatives to meet...


Covering innovative topics

Monday Nov 16

Tuesday Nov 17

Wednesday Nov 18