DataEng

Past Presentations

Custom, Complex Windows @Scale Using Apache Flink

100 Million members in over 190 countries leads to more than 1 Trillion events and 3 PB of data flowing through Netflix’s real-time data infrastructure each day. We’ve built a data pipeline in the cloud that reliably collects and routes these events to a variety of sinks. The data in...

Matt Zimmer Real-time Data Infrastructure Senior Engineer @Netflix
Predictive Datacenter Analytics With Strymon

A modern enterprise datacenter is a complex, multi-layered system whose components often interact in unpredictable ways. Yet, to keep operational costs low and maximize efficiency, we would like to foresee the impact of changing workloads, updating configurations, modifying policies, or deploying...

Vasia Kalavri PMC Member of Apache Flink (Core Developer Graph Processing API) & Postdoctoral researcher at the ETH Zurich Systems group
Streaming SQL Foundation: Why I ❤ Streams+Tables

What does it mean to execute robust streaming queries in SQL? What is the relationship of streaming queries to classic relational queries? Are streams and tables the same thing conceptually, or different? And how does all of this relate to the programmatic frameworks like we’re all familiar...

Tyler Akidau Engineer @Google & Founder/Committer on Apache Beam
The Power of Distributed Snapshots in Apache Flink

Come learn how Apache Flink is handles stateful stream processing and how to manage distributed stream processing and data driven applications efficiently with Flink's checkpoints and savepoints. Over the last years, data stream processing has redefined how many of us build data pipelines....

Stephan Ewen Committer @ApacheFlink, CTO @dataArtisans
Data Decisions With Realtime Stream Processing

At Facebook, we can move fast and iterate because of our ability to make data-driven decisions. Data from our stream processing systems provide real-time data analytics and insights; the system is also implemented into various Facebook products, which have to aggregate data from many sources. In...

Serhat Yilmaz Software Engineer @Facebook
Panel: SQL Over Streams, Ask the Experts

Queries over streams are generally "continuous," executing for long periods of time and returning incremental results. Yet operations over streams must have the ability to be monotonic. New Generation of Stream Processing Engines has added support for Stream SQL. This AMA / panel features a...

Julian Hyde Original Developer @ApacheCalcite, Co-Founder SQLstream, & Architect @Hortonworks
Tyler Akidau Engineer @Google & Founder/Committer on Apache Beam
Jay Kreps Co-Founder and CEO @Confluent
Michael Armbrust Initial Author of Apache Spark SQL & Leads Streaming Team @Databricks
Stephan Ewen Committer @ApacheFlink, CTO @dataArtisans

Interviews

Matt Zimmer Real-time Data Infrastructure Senior Engineer @Netflix

Custom, Complex Windows @Scale Using Apache Flink

What's the focus of your work?

Recently, I’ve primarily been building data platforms. That is, platforms to enable Data and Software Engineers to collect and process data.

Read Full Interview
Serhat Yilmaz Software Engineer @Facebook

Data Decisions With Realtime Stream Processing

QCon: What's the focus of your work and of the team that you're on at Facebook?

Rajesh: My team is working on stream processing, and we are part of the real-time data organization which focuses on faster, simpler, and smarter delivery of data. We want to reduce the time to results for people and our data driven products and people wait on that rely on data driven. Our organization encompasses the stream...

Read Full Interview
Ville Tuulos Machine Learning Infrastructure Engineer @Netflix

Human-Centric Machine Learning Infrastructure @Netflix

Can you give an example of some of the questions you get from data scientists when you are trying to deploy models?

When it comes to common questions, as boring as it may sound, my experience is that machine learning infrastructure is much more about data than science. Most questions we get are related to data: how do I find the data I need, how do I set up the data pipeline, how do I handle the somewhat non-trivial amounts of data in python and R,...

Read Full Interview

Less than

0

weeks until QCon San Francisco 2019

Registration is $2780.00 ($0 off) for the 3-day conference if you register before Dec 31st
SAVE YOUR SEAT

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.