Ellipses Ring Marker for High-speed Finger Tracking

Summary

The hand and fingers are typical input interfaces for information-related devices such as displays and robots. There are not only contact interfaces such as keyboards, but also non-contact interfaces such as gesture recognition. However, markerless systems have a significant problem of measurement latency in order to achieve a comfortable interaction with the displays and robots. Marker-based systems, such as motion capture systems, are capable of high-speed and low-latency tracking. However, it is necessary to attach convex dot markers to the hand or body surface, which has an issue of the marker wearability. The low latency tracking of hand posture using markers with high wearing continuity is expected to expand the application scenarios of hand tracking such as prolonged measurement.

We focus on a finger-ring as a marker with high wearability, and propose an ellipses ring marker, shown in Fig. 1. Circles and ellipses in three-dimensional space always appear as ellipses by perspective projection transformation (i.e., camera observation with a two-dimensional image). We actively utilize the projective invariance for high-speed marker tracking. High-speed image processing including binarization, contour extraction, and optimization process of ellipses fitting can run less than 2 ms. Then, we can realize dynamic projection mapping onto a palm (Fig. 2) using a high-speed low-latency projector as a low-latency feedback application. In addition to such image projection, low-latency feedback for fast and complex hand motions will be possible, leading to a wide range of applications in human-machine cooperation, sports, and medical operation.

Fig.1 Ellipses ring marker.	Fig.2 Dynamic projection mapping onto a palm.

Reference

1. Tomohiro Sueishi and Masatoshi Ishikawa: Ellipses Ring Marker for High-speed Finger Tracking, The 27th ACM Symposium on Virtual Reality Software and Technology (VRST2021) (Osaka, 2021.12.10)/Proceedings, Article No. 31