Dynamic projection mapping onto deforming non-rigid surface using a high-speed projector
Summary
Projection mapping attracts attentions as an emerging technology to extend the real world. However, almost realized examples have been limited to static or quasi-static environments. This research aims at overcoming this limitation and realizes dynamic projection mapping in which dynamically-changing real-world and virtual visual information are completely merged in the level of human visual perception. This high-speed dynamic projection mapping requires a high-speed projector enabling high-frame-rate and low-latency projection. In order to meet this demand, we have developed a high-speed projector "DynaFlash" that can project 8-bit images up to 1,000fps with 3ms delay.
In particular, as a challenging target for the dynamic projection mapping, we focus on a non-rigid surface. Sensing of non-rigid surface deformation is difficult to be achieved at high speed because it has high degrees-of-freedom and involves self-occlusions as well as external occlusions. Our newly proposed method overcomes this limitation. Our method can obtain the deformation robustly at 1,000 fps by using an originally proposed marker "Deformable Dot Cluster Marker", even when the target causes large deformation and occlusions.
Using these base technologies including DynaFlash and Deformable Dot Cluster Marker, we realize a new dynamic projection mapping onto deforming non-rigid surface. In this demonstration, by drawing the marker on the target with IR ink, we allow the marker to be invisible to human and enable robust sensing independently of the projected images. In our technology, both the projection and sensing are operated at a speed of 1,000 fps. Therefore, it is possible to keep the projection consistent with the deformation and extend the real world as if the projected image is printed or existed as an original (digital) texture on the target. Especially, focusing on new paradigms in the field of user interface and fashion, we have demonstrated dynamic projection mapping onto a deformed sheet of paper and T-shirt. Also we show that projection to multiple targets can be controlled flexibly by using multiple markers.
Movie
If you want to use the original video, please send an e-mail for copyright permission to contact .
Reference
- Gaku Narita, Yoshihiro Watanabe, and Masatoshi Ishikawa: Dynamic Projection Mapping onto Deforming Non-rigid Surface using Deformable Dot Cluster Marker, IEEE Transactions on Visualization and Computer Graphics, 2016.
- Gaku Narita, Yoshihiro Watanabe, and Masatoshi Ishikawa: Dynamic Projection Mapping onto a Deformable Object with Occlusion Based on High-speed Tracking of Dot Marker Array, The 21st ACM Symposium on Virtual Reality Software and Technology (VRST2015) (Beijing, 2015.11.14)/Proceedings, pp.149-152, 2015.
