SCIENCE CHINA Information Sciences, Volume 59 , Issue 9 : 092208(2016) https://doi.org/10.1007/s11432-015-5497-1

Precise planar motion measurement of a swimming multi-joint robotic fish

More info
  • ReceivedMay 17, 2015
  • AcceptedSep 6, 2015
  • PublishedAug 23, 2016


This paper presents a method for planar motion measurement of a swimming multi-joint robotic fish. The motion of the robotic fish is captured via image sequences and a proposed tracking scheme is employed to continuously detect and track the robotic fish. The tracking scheme initially acquires a rough scope of the robotic fish and thereafter precisely locates it. Historical motion information is utilized to determine the rough scope, which can speed up the tracking process and avoid possible ambient interference. A combination of adaptive bilateral filtering and k-means clustering is then applied to segment out color markers accurately. The pose of the robotic fish is calculated in accordance with the centers of these markers. Further, we address the problem of time synchronization between the on-board motion control system of the robotic fish and the motion measurement system. To the best of our knowledge, this problem has not been tackled in previous research on robotic fish. With information about both the multi-link structure and motion law of the robotic fish, we convert the problem to a nonlinear optimization problem, which we then solve using the particle swarm optimization (PSO) algorithm. Further, smoothing splines are adopted to fit curves of poses versus time, in order to obtain a continuous motion state and alleviate the impact of noise. Velocity is acquired via a temporal derivative operation. The results of experiments conducted verify the efficacy of the proposed method.

Funded by

National Natural Science Foundation of China(61375102)

National Natural Science Foundation of China(61333016)

National Natural Science Foundation of China(61421004)

Beijing Natural Science Foundation(3141002)

State Key Laboratory of Alternate Electrical Power System with Renewable Energy Sources(LAPS16006)



This work was supported by National Natural Science Foundation of China (Grant Nos. 61375102, 61333016, 61421004), Beijing Natural Science Foundation (Grant No. 3141002), and State Key Laboratory of Alternate Electrical Power System with Renewable Energy Sources (Grant No. LAPS16006).


[1] Su Z S, Yu J Z, Tan M, et al. Implementing flexible and fast turning maneuvers of a multijoint robotic fish. IEEE/ASME Trans Mechatron, 2014, 19: 329-338 CrossRef Google Scholar

[2] Kopman V, Laut J, Acquaviva F, et al. Dynamic modeling of a robotic fish propelled by a compliant tail. IEEE J Oceanic Eng, 2014, 40: 209-221 Google Scholar

[3] Swain D T, Couzin I D, Leonard N E. Real-time feedback-controlled robotic fish for behavioral experiments with fish schools. Proc IEEE, 2012, 100: 150-163 CrossRef Google Scholar

[4] Takada Y, Koyama K, Usami T. Position estimation of small robotic fish based on camera information and Gyro sensors. Robotics, 2014, 3: 149-162 CrossRef Google Scholar

[5] Wang W, Xie G M. Online high-precision probabilistic localization of robotic fish using visual and inertial cues. IEEE Trans Ind Electron, 2014, 62: 1113-1124 Google Scholar

[6] Fontaine E, Lentink D, Kranenbarg S, et al. Automated visual tracking for studying the ontogeny of zebrafish swimming. J Exp Biol, 2008, 211: 1305-1316 CrossRef Google Scholar

[7] Butail S, Paley D A. Three-dimensional reconstruction of the fast-start swimming kinematics of densely schooling fish. J R Soc Interface, 2012, 9: 77-88 CrossRef Google Scholar

[8] Porez M, Boyer F, Ijspeert A J. Improved Lighthill fish swimming model for bio-inspired robots: modeling, computational aspects and experimental comparisons. Int J Robot Res, 2014, 33: 1322-1341 CrossRef Google Scholar

[9] Marchese A D, Onal C D, Rus D. Autonomous soft robotic fish capable of escape maneuvers using fluidic elastomer actuators. Soft Robot, 2014, 1: 75-87 CrossRef Google Scholar

[10] Yu J Z, Wang S, Tan M. A parallel algorithm for visual tracking of multiple free-swimming robot fishes based on color information. In: Proceedings of IEEE International Conference on Intelligent Systems and Signal Processing, Changsha, 2003. 359--364. Google Scholar

[11] Kopman V, Laut J, Polverino G, et al. Closed-loop control of zebrafish response using a bioinspired robotic-fish in a preference test. J R Soc Interface, 2013, 10: 20120540-87 Google Scholar

[12] Butail S, Bartolini T, Porfiri M. Collective response of zebrafish shoals to a free-swimming robotic fish. PLoS ONE, 2013, 8: e76123-87 CrossRef Google Scholar

[13] Huck T, Westenberger A, Fritzsche M, et al. Precise timestamping and temporal synchronization in multi-sensor fusion. In: Proceedings of IEEE Intelligent Vehicles Symposium, Baden, 2011. 242--247. Google Scholar

[14] Romer K. Time synchronization and localization in sensor networks. Dissertation for Ph.D. Degree. Zurich: Swiss Federal Institute of Technology Zurich, 2005. 97--137. Google Scholar

[15] Liu T, Hu Y, Hua Y, et al. Study on autonomous and distributed time synchronization method for formation UAVs. In: Proceedings of IEEE International Joint Conference of FCS, Denver, 2015. 293--296. Google Scholar

[16] Nakadai K, Nakajima H, Murase M, et al. Real-time tracking of multiple sound sources by integration of in-room and robot-embedded microphone arrays. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, 2006. 852--859. Google Scholar

[17] Lim A, Mizumoto T, Cahier L K, et al. Robot musical accompaniment: Integrating audio and visual cues for real-time synchronization with a human flutist. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, 2010. 1964--1969. Google Scholar

[18] Nilsson O, Handel P. Time synchronization and temporal ordering of asynchronous sensor measurements of a multi-sensor navigation system. In: Proceedings of IEEE/ION Position Location and Navigation Symposium, Indian Wells, 2010. 897--902. Google Scholar

[19] Wu Z X, Yu J Z, Su Z S, et al. Towards an Esox lucius inspired multimodal robotic fish. Sci China Inf Sci, 2015, 58: 052203-87 Google Scholar

[20] Tomasi C, Manduchi R. Bilateral filtering for gray and color images. In: Proceedings of IEEE International Conference on Computer Vision, Bombay, 1998. 839--846. Google Scholar

[21] Hartigan J A, Wong M A. A k-means clustering algorithm. J Roy Stat Soc C-App, 1979, 28: 100-108 Google Scholar

[22] Wu Z X, Yu J Z, Tan M, et al. Kinematic comparison of forward and backward swimming and maneuvering in a self-propelled sub-carangiform robotic fish. J Bionic Eng, 2014, 11: 199-212 CrossRef Google Scholar

[23] Kennedy J. Particle swarm optimization. In: Sammut C, Webb G, eds. Encyclopedia of Machine Learning. Berlin: Springer, 2010. 760--766. Google Scholar

[24] Ebbesen S, Kiwitz P, Guzzella L. A generic particle swarm optimization Matlab function. In: Proceedings of American Control Conference, Montreal, 2012. 1519--1524. Google Scholar

[25] Reinsch C H. Smoothing by spline functions. Numer Math, 1967, 10: 177-183 CrossRef Google Scholar

[26] Comaniciu D, Meer P. Mean shift: a robust approach toward feature space analysis. IEEE Trans Patt Anal and Mach Intell, 2002, 24: 603-619 CrossRef Google Scholar

[27] Rother C, Kolmogorov V, Blake A. Grabcut: interactive foreground extraction using iterated graph cuts. ACM Trans Graph, 2004, 23: 309-314 CrossRef Google Scholar

Copyright 2020 Science China Press Co., Ltd. 《中国科学》杂志社有限责任公司 版权所有

京ICP备17057255号       京公网安备11010102003388号