Robotic Assistance for Extended Sensing, Locomotion and Manipulation by Gaze Control

Summary

For people with severe sensory-based motor disorder or musculoskeletal disorders, robotic assistance becomes a promising solution to improve their daily living standards. In this study we propose a robotic assistance method to realize extended sensing, locomotion as well as manipulation for a user to interact with the environment utilizing a 2D/3D gaze interface. Specifically, the proposed method focus on the situation where a user desire to explore the environment beyond their physical perception capability, requiring the robotic assistance of realizing extended sensing. The proposed 2D/3D gaze interface is wearable, unrestrained to users, and it is straightforward to give commands for robot control. Primary studies showed the effectiveness of the method in an indoor environment to fetch an invisible target from a box on a shelf.

Fig.1 Proposed concept

Fig.2 Developed system

Fig.3 Snapshots of experimental process

Reference

1. Y. Cao, S. Huang, S. L. Sørensen, Y. Yamakawa and M. Ishikawa: A Wearable Real-Time 2D/3D Eye-Gaze Interface to Realize Robot Assistance for Quadriplegics. IEEE Access, vol. 13, pp. 155259-155276, 2025, doi: 10.1109/ACCESS.2025.3603617
2. Chunxin Yang, Shouren Huang, Yuji Yamakawa, Masatoshi Ishikawa: Design of a Robotic Assistance System for Individuals with Disabilities Based on a Wearable 2D/3D Gaze Control Interface. Proceedings of the 2025 IEEE International Conference on Cyborg and Bionic Systems (CBS), pp.662-668, 2025.
3. Shouren Huang, Sune Lundø Sørensen, Yongpeng Cao, Masatoshi Ishikawa, Kjærgaard Mikkel, Yuji Yamakawa: Robotic Assistance for Extended Sensing, Locomotion and Manipulation by Gaze Control,2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)