ObjectUI

Prototype

I made several sample prototypes using Unity 3D to demonstrate the plausibility and potential applications of ObjectUI, including navigation and filtering point of interest.

Unity3D prototype: navigate to destination

Unity3D prototype: filter point of interest

Application

ObjectUI extends the tangible interaction we have with physical objects, such as paper maps, books, textbooks, architecture floor plans, restaurant menus, sketchbooks, etc.

Interaction occurs upon direct gestural contact with physical objects, such as pointing and sliding.

Augmenting books: translate

Augmenting menus: picture

Introduction & Motivation

There are two main categories of AR technology nowadays: eyewear and camera. AR eyewears focus interaction on air gestures and remotes, whereas AR cameras rely on interacting with touchscreens.

However, these types of interaction lack the direct contact physicality and tactile relationship we develop with our world and reality.

New paradigm

Each object has its UI. If we touch an object, this touching action carries an intent. For example, when we point to an item name on a restaurant menu, it could mean we want to direct others’ attention to it, explain, or ask what it is.

The AR layer is triggered upon direct hand contact with objects, such as pointing, tapping, and sliding.

Future Work

ObjectUI interaction pattern, a set of intuitive gestures for tangible interaction to trigger AR layer.

A refined version with occlusion removal.

ObjectUI v2, extending direct contact with voice.