VarioLight 2: Dynamic Projection Mapping for a Sports Sphere using Circumferential Markers

Summary

Version 1

In conventional high-speed projection mapping, a dot marker has been used to recognize the object's posture. However, it is often necessary to manually start tracking because a single dot marker has no geometric information. A cluster marker that combines multiple dots has a feature of automatic tracking start, but can suffer from problems of low resolution and blur. Such dot markers are not robust against random occlusion by hands or legs because the number of measurement points in an image can often be limited. Due to these problems, a widely projection mapping system corresponding to the object's posture (VarioLight) is inconvenient in practical situations such as sports.

Our laboratory has newly developed "circumferential markers" as tracking markers, focusing spheres that are actively used in sports (Fig. 1). To distinguish them from the version 2 described below, we call them uniform circumferential markers (UCM). The circle is observed as an ellipse in the perspective projection of the camera, and its shape includes geometric information of the sphere (marker position on the sphere from its inclination and collapse condition, and depth position of the sphere from its size). This marker has notable features of automatic tracking start, strong against low resolution and blurring, and a wide projection range in the depth direction. In addition, due to the large number of measurement points on the circumference, it is strong against random occlusion by hands and feet. With the 500 fps projection mapping system shown in Fig. 2, rotation-oriented projection for the sphere with circumferential markers is realized. Note that due to the uniformity of the markers, only relative posture can be measured with UCM.

By using this marker, as shown in Fig. 3(a), it is possible to project onto a playing ball among multiple people which has random occlusion by hands. Moreover, as shown in Fig. 3(b), thanks to circumferential makrers, rotation information is measured with high spatiotemporal resolution and can be projected immediately, and then real-time rotational visualization is realized. As shown in Fig. 3(c), texture projection whose speed is lower than that of the sphere itself, by which users can understand both texture and the sphere's speed, is also realized. This projection system is useful in sports such as soccer, volleyball, and tabletennis. It is expected to be used for sports practice by rotational visualization and wide-area entertainment using balls.

Fig.1 Uniform circumferential markers (UCM) arranged on a sphere.	Fig.2 Schematic diagram of a 500 fps projection mapping system for a widely moving sphere with circumferential markers．
Fig.3 (a) Playing catch with random occlusion by hands. (b) Real-time rotational expression using an arrow. (c) Texture projection whose speed is lower than that of the sphere itself, by which users can understand both texture and the sphere's speed.

Version 2

Uniform circumferential Markers (UCM) in the aforementioned version 1, as the name implies, are uniformly distributed on the spherical surface, making it impossible to acquire absolute posture. In order to obtain it, it is necessary to embed unique code into the markers by making the markers non-uniformly distributed. Conventional methods use complex rectangular shapes such as QR codes or dense point cloud distributions for coding, but these methods are susceptible to blurring and occlusion.

In this study, we propose biased circumferential markers (BCM) (Fig. 4). While the inner circle of each marker has the same distribution as in the UCM, the size of the outer circle and its bias against the inner circle are varied to achieve coding. As in the UCM, the shape of the outer circle is strong against blur because it utilizes a large circle. The proposed markers are also strong against occlusion because it can utilize the interrupted circumference of the circle by using the same image measurement algorithm as in the UCM. This enables fast and accurate measuring of the absolute posture of the sphere while maintaining the tracking performance of the UCM.

These markers are not only useful for dynamic projection mapping of a sphere with a wide area and robustness, but also shows the possibility of a new type of encoded marker that uses the circular shape and is resistant to blur and occlusion.

By knowing the absolute posture of the sphere, the projection does not shift after the return to occlusion. In addition, the rotation information before and after occlusion is accurate, so the motion information during occlusion can be easily estimated. In the future, these markers can be used to realize seamless all-around projection mapping using surrounding multiple projectors.

Fig.4 Biased circumferential markers (BCM) arranged on a sphere.

Movie

VarioLight 2: Dynamic Projection Mapping for Ball Sports
If you want to use the original video, please send an e-mail for copyright permission to contact.



VarioLight 2: Wide Dynamic Projection Mapping in Rhythmic Gymnastics
If you want to use the original video, please send an e-mail for copyright permission to contact.

Reference

1. Yuri Mikawa, Tomohiro Sueishi, Yoshihiro Watanabe, and Masatoshi Ishikawa: Dynamic Projection Mapping for Robust Sphere Posture Tracking Using Uniform/Biased Circumferential Markers, IEEE Transaction on Visualization and Computer Graphics, 2021 (Early Access).
2. Yuri Mikawa, Tomohiro Sueishi, Yoshihiro Watanabe, and Masatoshi Ishikawa: Projection mapping system to a widely dynamic sphere with circumferential markers, IEEE International Conference on Multimedia and Expo (ICME2020) (London, 2020.7.7-9), pp. 1-6
3. Yuri Mikawa, Tomohiro Sueishi, Yoshihiro Watanabe, and Masatoshi Ishikawa: VarioLight: Hybrid Dynamic Projection Mapping Using High-speed Projector and Optical Axis Controller, The 11th ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (SIGGRAPH ASIA 2018) (Tokyo, 2018.12.5-7)/Emerging Technologies, Article No. 17