Designing a User Experience for an Augmented World

We have a hunch that we’re pretty close to another large leap in technology, similar to the emergence of the web or the launch of the iPhone. This time we believe computers and smartphones will fade into the background as human interactions and machines become fully integrated within the physical world.

Android Jone Electric Love Credit: Android Jone

There are eager creatives, developers and user experience specialists who are passionate about the future of technology and Human Computer Interaction. Despite this eagerness, our software development skills largely depend on hardware that isn’t quite ready yet.

For a list of currently available devices check out: State of Holographics User Interfaces

We think creating this meaningful augmented world experience requires three primary hardware components working seamlessly together.

  1. Sensors that precisely understand your location and movement in relation to the physical space you’re in.
  2. Sensors that can pick up and understand the space you are in (e.g. walls, desks, doors, etc.)
  3. A screen between your eyes and your environment that can display virtual 3D objects in your real-world space.

We decided to build a simple cost-efficient version ourselves so we could develop rapidly and test with real users. We call it the Holo UI. It uses a Leap Motion Controller to track hand movement and an Infrared camera to track head movement. A Structure 3D sensor maps the physical world, and an HD 3D projector projects the virtual world onto a 42-inch semi transparent retro screen.

With the Holo UI, we can map virtual objects onto the real world and let users interact with them. Below is a concept sketch for what we built:

Holo UI Setup Design

An important part of the experience is how minimal and unintrusive this setup is for the user. All they need to interact with this screen is a light pair of 3D glasses and a baseball cap.

For more technical details on the Holo UI, checkout Jesse's blog post.

Holo UI Credit HTML Fusion

We're just getting started, but our setup already enables us to stage various UI scenarios on the fly and test them with real users. Our vision is to design user experiences for an augmented world that can solve real life problems.

Check back for future updates on the results of our User Experience research.