Utilizing High-Speed 3D Vision for a Commercial Robotic Arm

Summary

Camera-based 2D and 3D vision technologies, as well as visual feedback control, are already widely applied as the "eyes" that enable robots to observe their environment and target objects and perform intelligent motions. Currently, general 2D and 3D vision systems operate at 30-100 fps, limiting their application to low-speed tasks where motion blur does not occur. In this study, we developed a high-speed 3D measurement system (1,000 fps) using an active stereo method and aim to integrate it with a widely used robotic arm. Specifically, we constructed a system using the UR10e as shown in Fig. 1 and investigated its performance under various feedback rate.

In this study, we employed an active stereo 3D measurement system to capture objects with high temporal resolution. For the structured light pattern, we used a parallel bus pattern that enables high-speed 3D shape acquisition in a single shot. The measurement system was constructed using a high-speed camera (OMRON SENTECH, STC-MBV133U3V-SP, 648×474 @ 1,000 fps) and a projector (TEXAS INSTRUMENTS, DLPDLCR230NPEVM, 1920×1080, fixed projection pattern). A higher feedback rate offers advantages over a lower feedback rate, including improved responsiveness, finer control, and enhanced stability in high-speed systems. Assuming that factors such as computational load and noise are not considered, the following hypotheses can be formulated regarding the effect of feedback rate on control error, as illustrated in Fig. 2: 1) Within the saturation feedback rate, a higher feedback rate results in smaller control error. 2) Providing feedback information with a temporal resolution exceeding the saturation feedback rate does not lead to an improvement in control error. 3) Robots with a wider operational bandwidth exhibit a higher saturation feedback rate.

In the evaluation experiment, the measurement target was driven by a speed-adjustable stepping motor, performing irregular and high-speed circular reciprocating motion. The feedback rate was set to six levels: 50 Hz, 100 Hz, 200 Hz, 250 Hz, 333 Hz, and 500 Hz, and the tracking accuracy of the UR10e was compared. Additionally, to simulate robots with different operational bandwidths, the velocity slider of the UR10e was set to 50%, 60%, 70%, and 80% at each frequency, and tracking experiments were conducted under different output limit conditions. As an evaluation metric for tracking performance, the Euclidean distance between the end-effector position of the UR10e and the 3D position of the measurement target was integrated, and its average value was calculated. Fig. 3 shows the average Euclidean distance error between the UR10e end-effector and the measurement target. For each velocity scale, it was confirmed that the tracking error tended to decrease as the robot's speed increased. Furthermore, when the velocity scaler was constant, the error tended to decrease as the camera feedback rate increased. Moreover, when the feedback rate exceeded 200 Hz, the error became nearly constant, suggesting that the robot reached its saturation feedback rate within this range.

Fig.1 The developed high-speed 3D vision integrated with a UR10e

Fig.2 The proposed hyposis

Fig.3 Evaluation results

Reference

1. S. Huang, W. Wang, L. Miyashita, K. Murakami, Y. Yamakawa, and M. Ishikawa, “Utilizing High-Speed 3D Vision for a Commercial Robotic Arm: Direct Integration and the Dynamic Compensation Approach,” J. Robot. Mechatron., Vol.37 No.2, pp. 424-433, 2025.