High-Speed Focal Tracking Projection Based on Liquid Lens

Summary

Projection mapping (PM) is attractive as a fundamental technology for the advancement of various subjects, such as media art, entertainment, and augmented reality. However, conventional projectors have a shallow depth of field (DOF); therefore, sharp images are only visible in the limited depth range. In the case of dynamic projection mapping (DPM), which can project images on the surface of the moving objects, the shallow DOF limits the permissible motion of the object, because the projected images become blurred when the object is outside the DOF.

Our laboratories have developed a high-speed focal tracking projection system, which includes the technologies of high-speed vision, high-speed projector, and high-speed variable focus optics. In this system, the variation of the object's distance and posture was captured using the high-speed vision technology that served as immediate feedback to the liquid lens and high-speed projector. As a result, the focal distance is compensated, and the projected images are updated in real-time to fit the moving object. Therefore, a well-focused image projection was achieved even when the motion involved large depth range movement.

This system could ensure that the projected images were sharp and clear at variable distances, while the object was moving dynamically in a large three-dimensional area. Hence, this approach can be effectively applied to applications such as Volume Slicing Display. Furthermore, it can turn any physical surface into an interactive display, and enable the manipulation of their appearance to provide detailed information. Our system provides the essential technology for expanding such applications.

This work is the result of collaborative research between our laboratory, Watanabe Laboratory (Tokyo Institute of Technology), and Wang Laboratory (Institute of Semiconductor, Guangdong Academy of Sciences, China).

Fig.1 The conventional projectors have a shallow depth of field.	Fig.2 The proposed technology can project sharp images in large depth range.

Version 1

The first version of the high-speed focal tracking projection system has been developed. In this version, we have developed a high-speed variable focus optics unit, which can change the focus at high-speed. In this system, the object's distance was captured using the depth sensor that served as feedback to the liquid lens. As a result, the focal distance is compensated. Moreover, by updating the projected images, volume slicing display has been achieved.

Fig.3 Prototype of dynamic focus projector using a liquid lens. (Ver.1)

Fig.4 Projection images are kept in-focus, while target (white board) moves from 0.5m to 2.0m.

Version 2

The second version of the high-speed focal tracking projection system has been developed. In this version, we incorporated the technologies of high-speed vision and high-speed projector in order to increase in speed of feedback and update the projected images. Moreover, high-speed marker tracking method was incorporated to estimate the object's distance and posture. As a result, a well-focused dynamic projection mapping was achieved even when the motion involved large depth range movement.

Fig.5 Prototype of projection system using a liquid lens. (Ver.2)	Fig.6 System configuration.

Fig.7 In-focus projection in the large depth range.


Fig.8 Volume Slicing Display: change images based on the distance.

Movie

References

1. Lihui Wang, Hongjin Xu, Satoshi Tabata, Yunpu Hu, Yoshihiro Watanabe, and Masatoshi Ishikawa: High-Speed Focal Tracking Projection Based on Liquid Lens, ACM SIGGRAPH 2020 Emerging Technologies (SIGGRAPH '20) (Virtual Event, USA, 2020.8.24-28) [DOI: 10.1145/3388534.3408333]
2. Lihui Wang, Hongjin Xu, Yunpu Hu, Satoshi Tabata, Masatoshi Ishikawa, Dynamic Depth-of-Field Projection for 3D Projection Mapping, ACM CHI Conference on Human Factors in Computing Systems (CHI'19) (Glasgow, Scotland, UK. 2019.05.05-09) [DOI: 10.1145/3290607.3313246]
3. Lihui Wang, Yunpu Hu, Hongjin Xu, Masatoshi Ishikawa, Dynamic focal tracker display, SPIE Photonics West 2019 (San Francisco, California, USA. 2019.02.07)/ (Oral Session) [DOI: 10.1117/12.2506958]