Deformable Dot Cluster Marker: Marker for high-speed non-rigid surface tracking
Summary
In this research, we call a sensing technology to observe the deformation of a non-rigid surface continuously as non-rigid surface tracking. This technology is still a challenging problem in the field of computer vision. This is because the deformation of a non-rigid surface has high degrees-of-freedom and involves self-occlusions as well as external occlusions. These complicate the tracking problem.
In particular, as an application of the non-rigid surface tracking, we focus on dynamic projection mapping. This is the type of projection mapping which utilizes a moving object and dynamically-changing environment as the projected target. In this application, a speed is required to be sufficiently high that a human does not perceive any misalignment between the target object and the projected images. In order to achieve this requirement, non-rigid surface tracking needs to have millisecond-order computational speed. However, any conventional methods do not have such performance.
Based on this background, we propose a new method of the marker-based high-speed non-rigid surface tracking. In this type of method, a marker is drawn on the target surface and we obtain the deformation through the distortion of the marker in a captured image. We call the proposed marker as Deformable Dot Cluster Marker. This marker design is optimized under the condition that a local position in the marker is not incorrectly identified as other positions. Also the marker is easy to be drawn on the target because it is a binary pattern composing only black and white. With this marker design, we propose an original image processing method to obtain the deformation. This method executes detection of a marker from a captured image and frame-by-frame tracking utilizing high-frame-rate imaging independently and in parallel in separate threads. In addition, even when multiple markers exist in an image, the method can recognize them individually. We have implemented on CPU and achieved 1,000-fps performance. Also we have shown dynamic projection mapping onto deforming non-rigid surface at 1,000 fps in which the system generates the images based on the observed deformation and projects them in real-time by using a high-speed projector "DynaFlash".
Reference
- Gaku Narita, Yoshihiro Watanabe, and Masatoshi Ishikawa: Dynamic Projection Mapping onto Deforming Non-rigid Surface using Deformable Dot Cluster Marker, IEEE Transactions on Visualization and Computer Graphics, 2016.
- Gaku Narita, Yoshihiro Watanabe, and Masatoshi Ishikawa: Dynamic Projection Mapping onto a Deformable Object with Occlusion Based on High-speed Tracking of Dot Marker Array, The 21st ACM Symposium on Virtual Reality Software and Technology (VRST2015) (Beijing, 2015.11.14)/Proceedings, pp.149-152, 2015.
