How do you wish to control augmented reality? More on our HOLO UI lab

Augmented reality technology is something we at HTMLfusion have been very excited about recently. We think it's the next step in the evolution of the "screen," i.e. your laptop monitor or TV (if you’re interested in how we got into this in the first place, check out Philippe's blog post, The state of Holographics User Interfaces [Augmented World]).

We're working on a series of HOLO UI demos, short for Holographic User Interface. HOLO UI is a contraption we created in our office to demonstrate various interfaces and interactions in augmented reality (AR).

The purpose of these demos is to discover recurring elements in user behaviors when experiencing an augmented space. Ultimately, these demos help to solidify best practices for interaction design.

In our last demo, we used a reticle to activate and deactivate information through the looking glass, which is a 50" glass sheet with a semi-reflective rear projection film stuck onto it. The glass screen is effective at catching light that’s projected onto it while still being transparent enough to see real objects behind it.

For our latest demo, we will use the same hardware to focus on how a user manipulates a physical object in their space (or totem, in the AR world) to trigger a contextually accurate action.

Think of a light switch. Flipping the switch up or down dictates whether a light turns on or off. Imagine if we use the same switch to trigger something completely different, like opening the garage door or turning on a ceiling fan. We could display different UI elements on top of the same switch.

A Playstation controller is another example of an object that functions similarly to a totem. Depending on which game application you insert, the X button will initiate a pass (in the soccer game FIFA), a jump (in the fighting game Tekken), or changing stance (in Metal Gear Solid V). In HOLO UI, we don't need a fixed set of switches or buttons -- we have the freedom to explore any interaction for the application in use.

The way we interact with a seemingly simple switch device leads to an endless stream of questions about human behavior and interaction. For example, if you don't need a switch to turn on a light and had complete freedom to turn it on in any way, how would you trigger the light to turn on?

These simple questions, which consider how users would interact with devices without the impositions and limitations of our current reality, led us to ideate around the concept of augmented reality. In the process, we came up with three main questions:

  1. What information do people want about their surrounding environments?
  2. When do they want this information?
  3. What are the relevant triggers for desired information?

We wanted to augment features many people use while working to give a sense of HOLO UI's practical daily use. A music player seemed like a good place to start experimenting.

Using physical totems means we can turn any mundane object, like an empty box which was once your magic mouse's packaging, into a functional item by assigning actions to it. We thought it’d be interesting to go retro and bring back the vinyl and turntable as a music player.

A physical totem is not only fun, but useful. It helps users immediately understand the object they are supposed to interact with, even if there are virtual elements placed on top of and around it.

For this demo, we will use a 6x6 inch square box as the totem, and the box will act as the selected state while scrolling through the playlist. Once the user has selected a soundtrack, they will drag the box -- which will have the virtual vinyl on it -- over to the turntable, which triggers the track to begin playing.

Demo of a music box using totems

We also began modeling this in Blender, a 3D modeling platform, to achieve the three-dimensional feel through the looking glass.

For now, we're only focusing on the interaction within the music player in augmented reality, but as we proceed to the next steps, we'll zoom out from the music player itself and tackle the bigger picture.

More on our successes, failures, and iterations to come.