High-speed Depth-normal Measurement and Fusion Based on Multiband Sensing and Block Parallelization

Summary

A wide range of research areas have high expectations for the technology to measure 3D shapes, and to reconstruct the shape of a target object in detail. In this study, we consider a high-speed shape measurement technology that realizes accurate measurements in dynamic scenes. We propose a measurement method that sacrifices neither measurement density nor accuracy while realizing high speed. Many conventional 3D shape measurement systems employ only depth information to reconstruct a shape, which makes it difficult to capture the irregularities of an object's surface in detail. Meanwhile, methods that measure the surface normal to capture 3D shapes can reconstruct high-frequency components, although lowfrequency components tend to include integration errors. Thus, depth information and surface normal information have a complementary relationship in 3D shape measurements. This study proposes a novel optical system that simultaneously measures the depth and normal information at high speed by waveband separation as shown in Fig.1, 2, and a method that reconstructs the high-density, high-accuracy 3D shape at high speed from the two obtained data types by block division. The results confirm that the high-speed measurement was conducted at 400 fps with pixel-wise measurement density, and a measurement accuracy with an average error of 1.61 mm (Fig.3-5).

Fig.1 Optical system for high-speed depth-normal measurement	Fig.2 Actual system

Fig.3 Measured point clouds of a moving target

Fig.4 Depth measurement by a commercial system	Fig.5 Depth-normal fusion by the proposed system

References

1. Leo Miyashita, Yohta Kimura, Satoshi Tabata, Masatoshi Ishikawa: High-speed simultaneous measurement of depth and normal for real-time 3D reconstruction, SPIE Optical Engineering + Applications, Oral, 11842-52, San Diego, California, USA (hybrid with virtual conference), 1-5 Aug. (2021)
2. Leo Miyashita, Yohta Kimura, Satoshi Tabata, Masatoshi Ishikawa: High-speed Depth-normal Measurement and Fusion Based on Multiband Sensing and Block Parallelization, Journal of Robotics and Mechatronics (JRM), Vol.34, No.5, pp.1111-1121 (2022)