Build Your World and Play In It

Build Your World and Play In It

ISMAR 2010 Best Student Paper

Brett R. Jones, Rajinder Sodhi, Roy Campbell, Guy Garnett, Brian Bailey, “Build your world and play in it: Interacting with surface particles on complex objects,” IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2010.


We explore interacting with everyday objects by representing content as interactive surface particles. Users can build their own physical world, map virtual content onto their physical construction and play directly with the surface using a stylus. A surface particle representation allows programmed content to be created independent of the display object and to be reused on many surfaces. We demonstrated this idea through a projector-camera system that acquires the object geometry and enables direct interaction through an IR tracked stylus. We present three motivating example applications, each displayed on three example surfaces. We discuss a set of interaction techniques that show possible avenues for structuring interaction on complicated everyday objects, such as Surface Adaptive GUIs for menu selection. Through a preliminary informal evaluation and interviews with end users, we demonstrate the potential of interacting with surface particles and identify improvements necessary to make this interaction practical on everyday surfaces.

I design and prototype new technologies to create magical interactive experiences. I graduated from UIUC. I've worked for Walt Disney Imagineering, Microsoft Research and EA. I've co-created things like IllumiRoom and RoomAlive. I also co-run the #1 website on projection mapping.