You are viewing content from a past/completed QCon

Presentation: Multi-Modal Input Design for Magic Leap

Track: Future of Human Computer Interaction

Location: Pacific BC

Duration: 10:35am - 11:25am

Day of week: Wednesday

Level: Intermediate

Persona: Architect, Developer, Front-end Developer, General Software, Mobile Developer

Share this on:

What You’ll Learn

  1. Learn special design considerations for developing for Magic Leap One.

  2. Understand how Magic Leap is furthering and combining input modalities to improve the user experience in mixed reality.

  3. Learn about some of the use cases mixed reality can be used to address.

Abstract

As wearable spatial computing devices become more of a reality, new opportunities for interaction between humans and computers arise. Magic Leap embraces a wide range of inputs including hand and eye tracking, speech, a wireless 6DoF controller, and support for external peripherals. Learn what types of new input modalities are coming online and how they can be used and combined in different ways to surpass existing approaches in terms of throughput, discoverability, accessibility, and prediction with stories and examples from Magic Leap's Interaction Lab.

Question: 

What’s the focus of the work you do today?

Answer: 

I’m the design lead for the Interaction Lab at Magic Leap.

We are part of the Product Org and formerly a content-focused wing of Systems. Our original charter included a mix of doing first turn-ons and system tests for new platform features. Over time we have shifted more toward being an incubation team, rapid prototyping experiences and looking for new ways of interacting with our platform using the many input modalities we support. One of our customers is developers, so we build tools, iterate on our SDK, and publish sample projects with documentation as part of “Magic Kit.”

We build a lot of proof of concepts to explore the possibilities of the medium. You will find us focused on questions like: what kinds of experiences and new products can we enable with our tools and features, and are our features and KPIs good enough to be able to build the kind of products we think are valuable?

We're a hybrid team composed primarily of former game developers. We’re customers of our own tech and, in the process, also try to make sure it’s usable for external developers. We’re very design and product focused and have a mix of engineers, artists, and designers who work very closely to discover the potential of our product and feed our findings to other teams and partners. Our motto is “Fail Fast!”

Question: 

 

Question: 

What's the motivation for your talk?

Answer: 

This year we finally pulled the curtain back on Magic Leap with Magic Leap One, a dev kit and our first public offering. We want to spread the word and get people aware of our platform and see how it's different from other mixed reality and augmented reality systems.

We want to show that as we move further into the spatial computing era, we can democratize computing even further, making it more accessible by embracing natural inputs and user expectations, including for people with disabilities. For instance, we can show how when you start combining input modalities in different ways, similar to how people use a keyboard and mouse together, you can unlock a lot of new potential use cases, particularly when done in a wearable untethered mobile form factor. We can combine your gaze with gestures and speech or other modalities to accomplish things not possible with existing mediums today.

Question: 

What topics are you going to cover specific to multimodal input design for Magic Leap?

Answer: 

Over time the trajectory of HCI has been to make computers conform more to humans. Early computers were very hard to use. You had to be an expert. One topic we’ll address is how we can improve existing interactions with computers through increasing intuitiveness and breadth of inputs, making functionality more natural and thus discoverable. Another is, because Magic Leap is wearable and mobile, it opens up new possibilities for tasks focused around spatial design, either in context of real physical spaces or involving simulating real objects at scale. There’s a lot of use there for a variety of industries including interior or industrial design, but also training, education, and more.

Question: 

What is a technology problem that keeps you up at night?

Answer: 

So much of our existing technology is used for escapism. I’m particularly aware of this, having spent the majority of my career making video games prior to joining Magic Leap. When you force people to focus on a screen, I think that’s what naturally happens. Escapism certainly has its uses, but too much can cause harm as well. I think that we’re seeing humanity having an extinction psychology moment right now where things have gotten really bad, so we’re doubling down on our escapism. I came to Magic Leap, because I think there’s great value in creating a new type of computing platform that keeps us grounded in the real world and lets us use it as the foundation for our digital experiences rather than taking humans even further out of their surroundings. Now, maybe more than ever before, we need to find ways to get people back grounded in reality. I’m excited for the potential we have to do that with this new medium.

Speaker: Colman Bryant

Mixed Reality Game and Product Designer @MagicLeap

Colman Bryant is currently Lead Designer for the Interaction Lab at Magic Leap, an internal incubation team that explores interaction paradigms and prototypes new products and use cases for Magic Leap's Mixed Reality platform. In a past life, he worked in the game industry as Design Director at Hi-Rez Studios, where he lead design for hit service-based Free-to-Play games including Smite and Jetpack Fighter.

Find Colman Bryant at

Proposed Tracks

  • Delivering on the Promise of Containers

    Runtime containers, libraries and services that power microservices.

  • Evolving Java & the JVM

    6 month cadence, cloud-native deployments, scale, Graal, Kotlin, and beyond. Learn how the role of Java and the JVM is evolving.

  • Trust, Safety & Security

    Privacy, confidentiality, safety and security: learning from the frontlines.

  • Beyond the Web: What’s Next for JavaScript

    JavaScript is the language of the web. Latest practices for JavaScript development in and out of the browser topics: react, serverless, npm, performance, & less traditional interfaces.

  • Modern Operating Systems

    Applied, practical & real-world deep-dive into industry adoption of OS, containers and virtualization, including Linux on.

  • Optimizing You: Human Skills for Individuals

    Better teams start with a better self. Learn practical skills for IC.

  • Modern CS in the Real World

    Thoughts pushing software forward, including consensus, CRDT's, formal methods & probabilistic programming.

  • Human Systems: Hacking the Org

    Power of leadership, Engineering Metrics and strategies for shaping the org for velocity.

  • Building High-Performing Teams

    Building, maintaining, and growing a team balanced for skills and aptitudes. Constraint theory, systems thinking, lean, hiring/firing and performance improvement

  • Software Defined Infrastructure: Kubernetes, Service Meshes & Beyond

    Deploying, scaling and managing your services is undifferentiated heavy lifting. Hear stories, learn techniques and dive deep into what it means to code your infrastructure.

  • Practices of DevOps & Lean Thinking

    Practical approaches using DevOps and a lean approach to delivering software.

  • Operationalizing Microservices: Design, Deliver, Operate

    What's the last mile for deploying your service? Learn techniques from the world's most innovative shops on managing and operating Microservices at scale.

  • Developer Experience: Level up your Engineering Effectiveness

    Improving the end to end developer experience - design, dev, test, deploy and operate/understand.

  • Architectures You've Always Wondered About

    Next-gen architectures from the most admired companies in software, such as Netflix, Google, Facebook, Twitter, & more

  • Machine Learning without a PhD

    AI/ML is more approachable than ever. Discover how deep learning and ML is being used in practice. Topics include: TensorFlow, TPUs, Keras, PyTorch & more. No PhD required.

  • Production Readiness: Building Resilient Systems

    Making systems resilient involves people and tech. Learn about strategies being used from chaos testing to distributed systems clustering.

  • Building Predictive Data Pipelines

    From personalized news feeds to engaging experiences that forecast demand: learn how innovators are building predictive systems in modern application development.

  • Modern Languages: The Right Language for the Job

    We're polyglot developers. Learn languages that excel at very specific tasks and remove undifferentiated heavy lifting at the language level.