Real-Time Data
Past Presentations
Custom, Complex Windows @Scale Using Apache Flink
100 Million members in over 190 countries leads to more than 1 Trillion events and 3 PB of data flowing through Netflix’s real-time data infrastructure each day. We’ve built a data pipeline in the cloud that reliably collects and routes these events to a variety of sinks. The data in...
The Power of Distributed Snapshots in Apache Flink
Come learn how Apache Flink is handles stateful stream processing and how to manage distributed stream processing and data driven applications efficiently with Flink's checkpoints and savepoints. Over the last years, data stream processing has redefined how many of us build data pipelines....
Data Decisions With Realtime Stream Processing
At Facebook, we can move fast and iterate because of our ability to make data-driven decisions. Data from our stream processing systems provide real-time data analytics and insights; the system is also implemented into various Facebook products, which have to aggregate data from many sources. In...
Panel: SQL Over Streams, Ask the Experts
Queries over streams are generally "continuous," executing for long periods of time and returning incremental results. Yet operations over streams must have the ability to be monotonic. New Generation of Stream Processing Engines has added support for Stream SQL. This AMA / panel features a...
Handling Real-time Distributed Data Ingest
Software solutions, such as those for personalization, metering, IoT require processing of extremely large volumes of data in real time. High-speed data ingest and processing poses several challenges such as Managing large volume of data sometimes arriving in bursts Receiving data from...
Interviews
Custom, Complex Windows @Scale Using Apache Flink
What's the focus of your work?
Recently, I’ve primarily been building data platforms. That is, platforms to enable Data and Software Engineers to collect and process data.
Read Full InterviewData Decisions With Realtime Stream Processing
QCon: What's the focus of your work and of the team that you're on at Facebook?
Rajesh: My team is working on stream processing, and we are part of the real-time data organization which focuses on faster, simpler, and smarter delivery of data. We want to reduce the time to results for people and our data driven products and people wait on that rely on data driven. Our organization encompasses the stream...
Read Full Interview