High-speed Three-dimensional Tracking of Non-Rigid Surface for Dynamic Projection Mapping Using Multiple Cameras

Summary

Dynamic projection mapping has attracted attention for its usefullness to extend the reality. To realize dynamic projection mapping, it is necessary to track the motion of the object at high speed. In this research, we, in particular, focus on the high-speed three-dimensional tracking of the deforming non-rigid surface. Although various researches for tracking non-rigid deformation in three dimensions have been published, because of the large computational complexity, it is difficult to track the motion of the object at a speed in which human does not perceive geometric inconsistency between the object and the projected image. In addition, although the conventional system with Deformable Dot Cluster Marker succeeds in tracking deformation of non-rigid curved surface at high speed, information of non-rigid deformation can be obtained only two-dimensionally, and the sensing range is not so wide.

Therefore, in this research, we propose a method of finding 3D position of each point on a marker affixed to the projection target by using a system with multiple cameras based on Deformable Dot Cluster Marker. This high-speed non-rigid three-dimensional tracking is coordinated with a new high-speed projector DynaFlash v2 (3-LED + 1-DMD) capable of switching 8-bit color images at 947 fps, resulting in a new dynamic projection mapping that adaptively projects images has been realized. In the images and a movie below, it can be seen that the image in which gloss and irregularities are adaptively changed is projected according to the target posture and motion.

Movie

Reference

1. Noriaki Endo，Hikaru Amano，Yoshihiro Watanabe，Masatoshi Ishikawa, "High-Speed Three-Dimensional Tracking of Non-Rigid Surface for Dynamic Projection Mapping Using Multiple Cameras," .
The Institute of Electronics, Information and Communication Engineers, PRMU2017-122/Technical report of IEICE, vol. 117, no.391, pp. 87-92, Jan. 2018.