Netflix customers stream over two billion hours of content each month, accounting for over a third of downstream Internet traffic during peak hours. At this scale, Netflix's systems generate and collect millions of events every second, such as request traces, streaming client activities, and system metrics. It is essential for engineers to process such data streams efficiently and reliably to support real-time monitoring and alerting, outlier detection, application diagnostics, trend prediction, and many other operations.

Mantis is a new stream processing framework in development at Netflix. It provides users with the capabilities to write scalable stream processing jobs without having to worry about hard problems such as managing continuous data flow in a distributed environment or ensuring fault tolerance.

This talk will discuss how Mantis is being used at Netflix, the unique features of Mantis, and how Mantis is implemented. In particular:

• The programming model of Mantis. Mantis allows users to specify a directed acyclic graph to process streaming data. On each computation node, Mantis allows users to specify the processing logic using a rich set of reactive programming APIs.
• The support of auto scaling. Mantis is able to scale out each computation node based on the system load. This makes each Mantis job elastic to user-defined metrics.
• The optimization on stream localities and multiple stream sources. Mantis allows its jobs to tap into various type of streams available in Netflix, and schedules the jobs so they are as close to the stream data as possible.