Big Data Platform as a Service at Netflix

Location:

Grand Ballroom A

Track:

Time:

Wednesday, 1:35pm - 2:25pm

Abstract:

Netflix is well known for being a heavily data driven company, leveraging billions of hours of subscriber viewing data to power its recommendation algorithms and refine the customer experience. Hadoop has been instrumental in unlocking the potential to process these vast quantities data. However, without tooling and services to facilitate data discoverability and usability, the potential of big data would be difficult to fully realize at Netflix.

This talk will deep dive into key services of Netflix’s “data platform as a service” architecture, including RESTful services that: provide comprehensive metadata management across data sources (Franklin); enable visualization and caching of results of Hadoop jobs (Sting); and visualize the execution plans produced by languages such as Pig and Hive (Lipstick). The presentation will show how these services can be employed in concert to solve various use cases at Netflix, and will include implementation details, demos, and our open source roadmap for these projects.

Jeff Magnusson

Jeff Magnusson heads up the Data Platform Architecture group at Netflix, a team responsible for the design and creation of services and APIs to make access to big data at Netflix as intuitive, efficient, and self-serve as possible. Prior to joining Netflix, he was developing data analytics software at a startup in Austin. Jeff holds a PhD from the University of Florida focusing on database system implementation.