Invoked Computing: spatial audio and video AR invoked through miming

Summary

Direct interaction with everyday objects augmented with artificial affordances may be an approach to HCI capable of leveraging natural human capabilities. Rich Gold described in the past ubiquitous computing as an "enchanted village" in which people discover hidden affordances in everyday objects that act as "human interface "prompt[s]" (R. Gold, "This is not a pipe." Commun. ACM 36, July 1993.). In this project we explore the reverse scenario: a ubiquitous intelligence capable of discovering and instantiating affordances suggested by human beings (as mimicked actions and scenarios involving objects and drawings). Miming will prompt the ubiquitous computing environment to "condense" on the real object, by supplementing it with artificial affordances through common AR techniques. An example: taking a banana and bringing it closer to the ear. The gesture is clear enough: directional microphones and parametric speakers hidden in the room would make the banana function as a real handset on the spot.

In other words, the aim of the "invoked computing" project is to develop a multi-modal AR system able to turn everyday objects into computer interfaces / communication devices on the spot. To "invoke" an application, the user just needs to mimic a specific scenario. The system will try to recognize the suggested affordance and instantiate the represented function through AR techniques (another example: to invoke a laptop computer, the user could take a pizza box, open it and "tape" on its surface). We are interested here on developing a multi-modal AR system able to augment objects with video as well as sound using this interaction paradigm.

Movie

References

1. ZERROUG, A., CASSINELLI, A., ISHIKAWA, M., Invoked computing: Spatial audio and video AR invoked through miming, Proceedings of Virtual Reality International Conference (VRIC 2011), 6-8 April 2011, Laval, France (Jury Grand Prize) [PDF-2MB] [PPT-4MB][Poster-4MB].

Media

1. Wired: Augmented Reality: University of Tokyo “invoked computing”, by Bruce Sterling 11.24.11
2. The Guardian/The Observer: Alvaro Cassinelli: you really can use a banana as a telephone, interview by Sam Price, 1 Jan. 2012.
3. DiGinFo: Banana Phone And Pizza Box Laptop PC - Invoked Computing For Ubiquitous AR, Nov.18, 2011
4. CNET: New concept turns pizza boxes, bananas into computers, Nov. 2011
5. Business Weekly, INVOKED COMPUTING IS TOP BANANA 26, Feb 2012
6. Techcrunch, Ubergizmo, etc...