High-resolution Focused Tracking of Freely Swimming Fish

Summary

Individual identification and health management is desirable for multiple dynamic living beings in three dimensions unrestrictedly, such as freely swimming fish. Techniques by high-resolution image measurement are known for extracting individual identification information of a static object and heart rate estimation of fish. In the image measurement, however, continuous high-resolution observation for dynamic targets is difficult from the viewpoint of the angle of view, resolution, and depth of field. To solve a trade-off between the angle of view and resolution in a fixed camera, continuous high-resolution imaging can be expected using a mirror-based high-speed tracking system (1ms Auto Pan-tilt). However, correct tracking for a specific target is difficult with similar appearances (i.e., school of fish), and the mirror-based tracking system suffers from shallow depth of field of a telescopic lens. In this research, we propose an ellipse self-window method and high-resolution imaging system. Individual health management applications in aquaculture and aquarium are expected for the school of fish, and individual identification is also applied to robotics picking and drones.

In continuous detection of the position of a specific individual in a school of fish, occlusion including crossing with other individuals is a major issue. We extend a self-window method in high frame rate image processing and propose an ellipse self-window method (Fig. 1). By limiting the object recognition processing such as binarization in each image to the elliptical area to be updated, the ellipse converges near the fish head (heart brightness changes of Medaka fish can be observed from below) and high-speed tracking robust against individual intersection can be possible. In the mirror-based tracking system, the ellipse self-window method can improve the efficiency of continuous high-resolution imaging of a specific individual.

To solve the shallow depth of field in the mirror-based tracking system, we propose a high-resolution focused imaging method for multiple 3D dynamic targets by effectively combining the high-speed optical axis control system, a high-speed liquid variable focus lens, and a wide-angle camera (Fig. 2). While observing the whole targets with a wide-angle camera, the direction of a high-resolution camera is dynamically controlled for a specific individual by the mirror-based optical axis control system. The liquid lens is also controlled at high speed by triangulation of a stereo system consisting of the wide-angle camera and the mirror control system, and focused imaging against three-dimensional motion can be achieved (Fig. 3).

Fig. 1 Ellipse self-window method.	Fig. 2 High-resolution imaging system.
Fig. 3 Tracking results of Medaka fish.

Movie

Reference

1. Murtuza Petladwala, Tomohiro Sueishi, Shoji Yachida, and Masatoshi Ishikawa: High-Speed Occlusion Recovery Method for Multiple Fish Visual Tracking, 42nd Annual International Conferences of the IEEE Engineering in Medicine and Biology Society (EMBC2020) (Montreal, 2020.7.20)/Proceedings, MoAT14.12, p.6185
2. Tomohiro Sueishi, Takuya Ogawa, Shoji Yachida, and Masatoshi Ishikawa: Continuous high-resolution observation system using high-speed gaze and focus control with wide-angle triangulation, SPIE Photonics West 2020 (San Francisco, 2020.2.2)/Proceedings of SPIE, Vol.11250, pp.1125012-1-10
3. Tomohiro Sueishi, Takuya Ogawa, Shoji Yachida, Yoshihiro Watanabe, and Masatoshi Ishikawa: High-resolution Observation Method for Freely Swimming Medaka Using High-speed Optical Tracking with Ellipse Self-window, 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC2018) (Honolulu, 2018.7.20)/Proceedings, FrPoS-32.41