Presentation: Unified Integration for Data-Driven Applications

Duration

Duration: 
4:10pm - 5:00pm

Persona:

Abstract

This talk proposes and presents a modern, unified architecture for the data stored in and applications running on Hadoop and other open source big data infrastructure.

Building and running production applications in the Hadoop ecosystem is often a long and challenging process filled with slow development cycles and painful production operations. Traditional development tools and techniques are missing from the Hadoop ecosystem. Now, as the Hadoop ecosystem continues to grow and use cases mature, these challenges are increasing as SLAs intensify and services become customer-facing and revenue-generating.

Based on my experiences as an early committer on Apache HBase, building real-time systems on Hadoop at Facebook, as well as working with customers at Cask, I would like to introduce the notion of a Data Application Platform (DAP), a modern, unified integration platform for data and applications. The DAP aims to address the challenges associated with building and operating production, real-time applications on Hadoop. Born out of the idea of traditional App Servers like Weblogic or JBoss, a DAP is an application-oriented middleware layer for big data that presents a single environment for design, operations, data science and governance. It provides higher-level APIs, reusable libraries, tools for testing, an in-memory environment for CI, a local environment for development and a distributed environment for production.

This talk will outline the requirements of building and running real-time production applications on Hadoop, the challenges, the proposal and architecture of a DAP, and addressing some of the remaining gaps in Hadoop.

Speaker: Jon Gray

Founder and CEO @Cask

Jonathan Gray, Founder & CEO of Cask, is an entrepreneur and software engineer with a background in startups, open source and all things data. Prior to founding Cask, Jonathan was a software engineer at Facebook where he drove HBase engineering efforts, including Facebook Messages and several other large-scale projects from inception to production. An open source evangelist, Jonathan was responsible for helping build the Facebook engineering brand through developer outreach and refocusing the open source strategy of the company. Prior to Facebook, Jonathan founded Streamy.com, where he became an early adopter of Hadoop and HBase and is now a core contributor and active committer in the community. Jonathan holds a bachelor’s degree in Electrical and Computer Engineering and Business Administration from Carnegie Mellon University.

Find Jon Gray at

.

Tracks

Monday Nov 7

Tuesday Nov 8

Wednesday Nov 9

Conference for Professional Software Developers