Rajesh: The goal for this talk is to introduce what stream processing is and how we're thinking about it at Facebook. We don't want stream processing to be this entire new ecosystem people code against. We want stream processing to seamlessly integrate into the rest of Facebook's existing infrastructure. People shouldn't know that streaming is happening. The fact that people have to know that streaming is a different platform makes it difficult for people to start consuming and leveraging it. The goal of this talk is to address why stream processing is hard, then walk through three different examples of how we interact with our internal customers to give them better and more seamless access. We're obviously not finished, there is a long way to go, but it gives you a snapshot of we've done so far.

Rajesh: We have our own late-arriving data algorithm, as a result of how we deal with late-arriving data. When you see the myriad of garbage data that you see in the world, it's important to have types, to get statistical conclusions, to decide what windows you are going to keep open, how you track time. I'm going to spend some time on that.

Rajesh: I think it's more on the software engineering side. We have talked a lot about Flink and Spark and how to merge those, but how do we provide value that automatically can translate stuff? We can't just use stream processing with this and get as much value as possible. There are still some scaling challenges to this. The data scientists should focus more on what they're trying to compute not how to get it done, and we can figure out what's the best way to do that. Data scientists should focus on the accuracy and correctness of the metrics and the queries. And we should just focus about how to get things done. That's the direction that we're taking in stream processing with Facebook.