masterhead masterhead  masterhead

Combining visual and haptic modalities for human-robot interaction and collaboration


In this research, we present the study of combining human visual and haptic perception in human-robot interaction to realize collaborative manipulation. In our human-robot collaboration scheme, cognitive capabilities of human and accurate motion control capabilities of robot are well integrated. Human operator is designated for coarse global-motion under feedback interfaces utilizing human visual as well as haptic modalities. Fine local-motion in an active manner without involving human intention-aware is realized by a dynamic compensation robotic module with high-speed visual feedback. Primary experimental results verified effectiveness of the proposed method.

Combinations of visual and haptic modalities for human-robot interaction and collaboration

Experimental system for evaluation


  1. Shouren Huang, Masatoshi Ishikawa, and Yuji Yamakawa. 2019. Human-Robot Interaction and Collaborative Manipulation with Multimodal Perception Interface for Human. In Proceedings of the 7th International Conference on Human-Agent Interaction (HAI ’19), October 6–10, 2019, Kyoto,Japan. ACM, New York, NY, USA, 3 pages.
  2. Shouren Huang, Chaoyue Hu, Yuji Yamakawa, Masatoshi Ishikawa: Combining Human Visual and Haptic Perception in Human-Robot Interaction for Collaborative Manipulation,2019 JSME Conference on Robotics and Mechatronics(ROBOMECH2019),(Hiroshima,2019.6.7)/Proceedings, 2P1-L03 (2019)
Ishikawa Group Laboratory, Data Science Research Division, Information Technology Center, University of Tokyo
Ishikawa Group Laboratory WWW admin: contact
Copyright © 2008 Ishikawa Group Laboratory. All rights reserved.