Augmented Everyday Objects

Enhance and extend tangible interaction with AR

ObjectUI is an object-based interface through augmented reality (AR). Each facet of an object itself is a user interface (UI). As people interact with these UI's on objects, their interaction intent would be embodied through the AR display.


I made several sample prototypes using Unity 3D to demonstrate the plausibility and potential applications of ObjectUI, including navigation and filtering point of interest.

Unity3D prototype: navigate to destination
Unity3D prototype: filter point of interest


ObjectUI extends the tangible interaction we have with physical objects, such as paper maps, books, textbooks, architecture floor plans, restaurant menus, sketchbooks, etc.

Interaction occurs upon direct gestural contact with physical objects, such as pointing and sliding.

Augmenting books: translate
Augmenting menus: picture

Introduction & Motivation

There are two main categories of AR technology nowadays: eyewear and camera. AR eyewears focus interaction on air gestures and remotes, whereas AR cameras rely on interacting with touchscreens.

However, these types of interaction lack the direct contact physicality and tactile relationship we develop with our world and reality.

As we interact with physical objects, can AR enhance, rather than replace, the rich materiality we are already familiar with?

New paradigm

Each object has its UI. If we touch an object, this touching action carries an intent. For example, when we point to an item name on a restaurant menu, it could mean we want to direct others’ attention to it, explain, or ask what it is.

The AR layer is triggered upon direct hand contact with objects, such as pointing, tapping, and sliding.

Future Work

ObjectUI interaction pattern, a set of intuitive gestures for tangible interaction to trigger AR layer.

A refined version with occlusion removal.

ObjectUI v2, extending direct contact with voice.

Date: Fall 2016 - Present
Client: Studio Project
State: Prototype
Keyword: AR, Augmented Reality, Mixed Reality, tangible interaction, everyday objects, maps, books