Presentation: ML in the Browser: Interactive Experiences with Tensorflow.js

Track: Machine Learning for Developers

Location: Pacific LMNO

Duration: 2:55pm - 3:45pm

Day of week: Wednesday

Share this on:

Abstract

Machine learning (ML) holds opportunity to build better experiences right in the browser! Using libraries such as Tensorflow.js, we can better anticipate user actions, reliably identify sentiment or topics in text, or even enable gesture based interaction - all without sending the user’s data to any backend server. However, the process of building an ML model and converting it to a format that can be easily imported into a front-end web application can be unclear and challenging. 

In this talk, I provide a friendly introduction to machine learning, cover concrete steps on how front-end developers can create their own ML models and deploy them as part of web applications. To further illustrate this process, I will discuss my experience building Handtrack.js - a library for prototyping real time hand tracking interactions in the browser.  Handtrack.js is powered by an object detection neural network (MobilenetV2, SSD) and allows users to predict the location (bounding box) of human hands in an image, video or canvas html tag. 

Audience

  • Front end engineers interested in using ML within their web applications.
  • Software engineers interested in training ML models
  • Data Scientists interested in deploying ML 

What you can expect

  • A friendly introduction to ML in the browser using Tensorflow.js
  • When to use ML in the browser
  • How to create a machine learning model with an example (data collection, model training, model evaluation, conversion to Tensorflow.js).
  • Practical tips and pitfalls associated with ML projects (what model to use, data validation checks, what framework to use etc.)
  • A live demo of hand gesture interaction in the browser, using a neural network model.

Speaker: Victor Dibia

Research Engineer in Machine Learning @cloudera

Qcon

Victor Dibia is a Research Engineer with Cloudera’s Fast Forward Labs. Prior to this, he was a Research Staff Member at the IBM TJ Watson Research Center, New York. His research interests are at the intersection of human computer interaction, computational social science, and applied AI. A senior member of IEEE, Victor has published work at venues like the AAAI Conference on Artificial Intelligence and ACM Conference on Human Factors in Computing Systems. His work has been featured in outlets such as the Wall Street Journal and VentureBeat. He holds an M.S. from Carnegie Mellon University and a Ph.D. from City University of Hong Kong

Find Victor Dibia at

Tracks

Monday, 11 November

Tuesday, 12 November

Wednesday, 13 November