3D Motion Recognition with a Set of Differently Focused Images
Summary
Our laboratory has developed a Dynamorph Lens (DML), which is a high-speed liquid lens with an ability to control a focal length in milliseconds. A focus-based new image processing technique has become possible with multi focus image sequences which can be obtained by combining the DML and high frame rate camera.
Based on the image sequences captured by shifting the focus quickly using the DML, we especially focus on an estimation of 3D motion of feature points in the images, and have developed an algorithm for that characteristics. In our method, we regard several images at different focus position in a specific scene as one 3D data or a 'focal depth image set'. Then we estimate the 3D motion of an object by calculating an optical flow between differentials of the 3D data in a depth direction.
We used the DML and the high-speed camera and did a tracking experiment against the image set captured at 2000 fps.
We obtained 250 images per second as the focal depth image set by adding a 125 Hz triangle wave to a piezoelectric actuator.
In this experiment we did the tracking task using a free-swimming shrimp as a target,
and we succeeded the 3D motion estimation of the object in high precision over 316 frames.
We show a movie of the image set in this experiment and a movie of the tracking result below.
The movie of the tracking result shows that the shrimp is always in focus, which indicates the success of 3D motion estimation including the depth direction.
Figure 1. Method summary
