Wide High-resolution Projection System Using High-speed Gaze Point Estimation

Summary

In image presentation devices with a fixed angle of view, such as liquid crystal displays and projectors, it is difficult to achieve both high resolution and a wide angle of view with a limited number of pixels. If the image presentation area can move in accordance with the observer's gaze point, it is possible to achieve both resolution and angle of view, but this requires not only accurate positioning of the dynamically changing image presentation area but also fast estimation of the observer's gaze point.

In this study, we propose a wide high-resolution projection system that combines a tracking projection mosaicing system developed in our laboratory and a high-speed gaze point estimation by capturing the pupil and corneal surface reflection images alternately and at high speed using near-infrared light (Fig. 1). The near-infrared illumination placed at the four corners of the projection screen and at the same optical axis as the camera that captures the observer's eyes blink alternately at 500 Hz, and a high-speed camera (1,000 fps) synchronized with the illumination captures the eyes to estimate the gaze point. Positional relationship between the pupil and the corneal reflection of each illumination enables fast gaze point estimation with almost no prior calibration (Fig. 2). In addition, while dynamically switching the direction of the projector's direction with galvanometer mirrors in accordance with the position of the gaze point, the projector can display a single large image from the observer's viewpoint even if the gazing point changes (Fig. 3), in conjunction with the tracking projection mosaicing system that prevents misalignment of the projected image by appropriately synchronizing the galvanometer mirrors and a high-speed projector. The third-person view can also be a form of visualization of the observer's view (Fig. 4). Since this measurement and projection algorithm does not require prior positioning accuracy between the screen and the observer, it is expected to be applied to situations where people can move around freely in a wide area in augmented reality systems.

Fig.1 Concept.	Fig.2 High-speed gaze point estimation algorithm.
Fig.3 First person view.	Fig.4 Third person view.

Reference

1. Ayumi Matsumoto, Tomohiro Sueishi, and Masatoshi Ishikawa: High-speed Gaze-oriented Projection by Cross-ratio-based Eye Tracking with Dual Infrared Imaging, 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW2022) (Online, 2022.3.14)/Proceedings, pp.594-595 (2022) Nomination Best Poster Award
2. Ayumi Matsumoto, Masashi Nitta, Tomohiro Sueishi, and Masatoshi Ishikawa: Wide High-resolution Projection System Using High-speed Gaze Point Estimation, Transactions of the Society of Instrument and Control Engineers, Vol.58, No.1, pp.42-51 (2022) (Japanese)