Visual and Tactile Cues for High-Speed Interaction

Summary

This is a new system which integrates two subsystems. One subsystem can extract a 3-dimensional position of a moving object every 2ms, and project pictures (e.g. a screen of a video game or a computer, a movie etc.) on the moving object at the same time, using two "1ms Auto Pan-Tilt" systems we developed which can track an object in 3-dimensional space without delay by high-speed vision and two rotational mirrors. Another subsystem can display tactile sensation on an object depending on its 3-dimensional position, especially a paticular position on a palm of a hand, using "Airborne Ultrasound Tactile Display (AUTD)" by an array of ultrasonic oscillators, which is developed by the Shinoda Laboratory, Department of Complexity Science and Engineering Graduate School of Frontier Sciences, Department of Information Physics and Computing, the University of Tokyo. This time, we realized a demonstration that papers around us and our hands are transformed into a screen of a computer or a smartphone, and we can feel even tactile sensation. In a sense, this system can be regarded as a moving object version and a tactile sensation version of projection mapping technology.

This system recognizes our hands and objects existing in the environment at a high-speed beyond human's ability of recognition by high-speed image processing technology, and it is possible to use the system to display and input information without uncomfortable feelings such as a delay. Thus, this system shows that we can use a object as a tool for human interfaces even if the object is moving. While we aim to embed intelligent function into objects such as conventional computers and smartphones, this system embeds information into existing environments and objects; it points the way toward future dramatic changes in our information environment.

Fig.1 System Configuration.		Fig.2 1ms AutoPan/Tilt and Airborne Ultrasound Tactile Display.

Fig.3 Picture displayed on a moving paper.		Fig.4 Picture dislayed on a moving hand. (Tactile sensation is also displayed.)

Movie

Reference

1. Kohei Okumura, Hiromasa Oku and Masatoshi Ishikawa: Lumipen: Projection-based Mixed Reality for Dynamic Objects, 2012 IEEE International Conference on Multimedia and Expo (ICME 2012) (Melbourne,2012.7.11) / Conference Proceedings, pp.699-704
2. Kohei Okumura, Hiromasa Oku and Masatoshi Ishikawa: High-Speed Gaze Controller for Millisecond-order Pan/tilt Camera, 2011 IEEE International Conference on Robotics and Automation (ICRA 2011) (Shanghai, 2011.5.12) / Conference Proceedings, pp.6186-6191 [PDF (1.3MB)]^*IEEE