Modern businesses are pushing the limits of decision making. Advancements in stream processing and OLAP (Online Analytical Processing) technologies have enabled faster insights into the data coming in, thus powering near real time decisions. A lot of use cases such as Fraud detection, Operational dashboards, Financial Incentive pipelines and Experimentation (A/B testing) need SQL like access to such streaming data.

This talk focuses on how Uber and LinkedIn use Apache Samza, Apache Calcite and Pinot for powering such use cases. First half of the talk will go over our analytics platform: AthenaX used by data scientists and engineers for specifying data transformations (SQL on Streams) and make it available for querying by real time dashboards & maps within minutes. Second half will focus on what happens under the hood and challenges faced with respect to scale, at least once semantics, windowing, schema derivation and so on.