Speaker: Jerry Peng
He / him / his
Senior Staff Software Engineer @Databricks
Jerry Peng is currently an Senior Staff Software Engineer at Databricks extensively working Apache Spark Structured Streaming. Before joining Databricks, he was a Principal Software Engineer at Splunk working on streaming and messaging projects especially with Apache Pulsar and Apache Flink. Jerry is a committer and PMC member of Apache Pulsar, Apache Storm, and Apache Heron projects. Before Splunk, he worked at Streamlio (acquired by Splunk), Citadel, and Yahoo on distributed systems and stream processing. Jerry has been working in the area of distributed systems and stream processing since his days in grad school at the University of Illinois, Urbana-Champaign.
Session
Apache Spark: Structured Streaming Real-Time Mode
Apache Spark Structured Streaming is a distributed stream processing engine built on Spark SQL and is widely used for ETL-style workloads, where seconds- to minutes-level latency is acceptable. But millisecond-latency use cases have historically been out of reach.