SCIENCE CHINA Information Sciences, Volume 60, Issue 6: 062102(2017) https://doi.org/10.1007/s11432-015-5450-3

Adaptive multiple video sensors fusion based on decentralized Kalman filter and sensor confidence

More info
  • ReceivedApr 25, 2016
  • AcceptedMay 18, 2016
  • PublishedFeb 8, 2017


The fusion of multiple video sensors provides an effective way to improve the robustness and accuracy of video surveillance systems. In this paper, an adaptive fusion method based on a decentralized Kalman filter (DKF) and sensor confidence is presented for the fusion of multiple video sensors. The adaptive scheme is one of the approaches used for preventing the divergence problem of the filter when statistical values of the measurement noises of the system models are not available. By introducing the sensor confidence, we can adaptively adjust the measurement noise covariance matrix of the local DKFs and thus, determine the weight of each sensor more correctly in the fusion procedure. Also, the DKF applied here can make full use of redundant tracking data from multiple video sensors and give more accurate fusion results in an efficient manner. Finally, the fusion result with improved accuracy is obtained. Experimental results show that the proposed adaptive decentralized Kalman filter fusion (ADKFF) method works well in the case of real-world video sequences and exhibits more promising performance than single sensors and comparative fusion methods.

Funded by

"source" : null , "contract" : "2012CB821206"

National Basic Research Program of China(973 Program)

National Natural Science Foundation of China(61532006)

National Natural Science Foundation of China(61502042)

National Natural Science Foundation of China(61320106006)



This work was supported by National Basic Research Program of China (973 Program) (Grant No. 2012CB821206) and National Natural Science Foundation of China (Grant Nos. 61320106006, 61532006, 61502042).


[1] Aghajan H, Cavallaro A. Multi-Camera Networks: Principles and Applications. Pittsburgh: Academic Press, 2009. Google Scholar

[2] Jia Y. Alternative proofs for improved LMI representations for the analysis and the design of continuous-time systems with polytopic type uncertainty: a predictive approach. IEEE Trans Automat Contr, 2003, 48: 1413-1416 CrossRef Google Scholar

[3] Snidaro L, Visentini L, Foresti G L. Intelligent Video Surveillance: Systems and Technology. Boca Raton: CRC Press, 2009. 363--388. Google Scholar

[4] Li B, Yan W. A sensor fusion framework using multiple particle filters for video-based navigation. IEEE Trans Intell Trans Syst, 2010, 11: 348-358 CrossRef Google Scholar

[5] Denman S, Lamb T, Fookes C, et al. Multi-spectral fusion for surveillance systems. Comput Electr Eng, 2010, 36: 643-663 CrossRef Google Scholar

[6] Loreto S, Jose M M, Ander A, et al. RGB-D, laser and thermal sensor fusion for people following in a mobile robot. Int J Adv Robot Syst, 2013, 10: 271-663 CrossRef Google Scholar

[7] Federico C. A review of data fusion techniques. Sci World J, 2013, 2013: 704504-663 Google Scholar

[8] Christoph S, Fernando P L, Marco K. Information fusion for automotive applications---an overview. Inform Fusion, 2011, 12: 244-252 CrossRef Google Scholar

[9] Chan A L, Schnelle S R. Fusing concurrent visible and infrared videos for improved tracking performance. Opt Eng, 2013, 52: 017004-252 CrossRef Google Scholar

[10] Jia Y. Robust control with decoupling performance for steering and traction of 4WS vehicles under velocity-arying motion. IEEE Trans Contr Syst Tech, 2000, 8: 554-569 CrossRef Google Scholar

[11] Snidaro L, Visentini I, Foresti G L. Fusing multiple video sensors for surveillance. ACM Trans Multim Comput Commun Appl, 2012, 8: 7-569 Google Scholar

[12] Chong C Y, Mori S. Optimal fusion for non-zero process noise. In: Proceedings of the 16th International Conference on Information Fusion, Istanbul, 2013. 365--371. Google Scholar

[13] Xu J, Song E B, Luo Y T, et al. Optimal distributed Kalman filtering fusion algorithm without invertibility of estimation error and sensor noise covariances. IEEE Signal Process Lett, 2012, 19: 55-58 CrossRef Google Scholar

[14] Li Z G, Tian X Y. The application of federated Kalman filtering in the information fusion technique. In: Cross Strait Quad-Regional Radio Science and Wireless Technology Conference, Harbin, 2011, 2: 1228--1230. Google Scholar

[15] Zhang H, Sang H S, Shen X B. Adaptive federated Kalman filtering attitude estimation algorithm for double-FOV star sensor. J Comput Inf Syst, 2010, 6: 3201-3208 Google Scholar

[16] Qi W J, Zhang P, Deng Z L. Weighted fusion robust steady-state Kalman filters for multisensor system with uncertain noise variances. J Appl Math, 2014, 2014: 369252-3208 Google Scholar

[17] Julier S J, Uhlmann J K. General decentralized data fusion with covariance intersection. In: Handbook of Multisensor Data Fusion. Boca Raton: CRC Press, 2009. 319--342. Google Scholar

[18] Markus S S, Kristian K. Performance analysis of decentralized Kalman filters under communication constraints. J Adv Inf Fusion, 2007, 2: 65-75 Google Scholar

[19] Deng Z L, Zhang P, Qi W J, et al. Sequential covariance intersection fusion Kalman filter. Inform Sciences, 2012, 189: 293-309 CrossRef Google Scholar

[20] Deng Z L, Zhang P, Qi W J, et al. The accuracy comparison of multisensor covariance intersection fuser and three weighting fusers. Inform Fusion, 2013, 14: 177-185 CrossRef Google Scholar

[21] Ibarra-Bonilla M N, Escamilla-Ambrosio P J, Ramirez-Cortes J M, et al. Pedestrian dead reckoning with attitude estimation using a fuzzy logic tuned adaptive kalman filter. In: Proceedings of the IEEE 4th Latin American Symposium on Circuits and Systems, Cusco, 2013. 1--4. Google Scholar

[22] Li J, Lei Y H, Cai Y Z, et al. Multi-sensor data fusion algorithm based on fuzzy adaptive Kalman filter. In: Proceedings of the 32nd Chinese Control Conference, Xi'an, 2013. 4523--4527. Google Scholar

[23] Correia P L, Pereira F. Objective evaluation of video segmentation quality. IEEE Trans Image Process, 2003, 12: 186-200 CrossRef Google Scholar

[24] Jia Y. General solution to diagonal model matching control of multi-output-delay systems and its applications in adaptive scheme. Progress Nat Sci, 2009, 19: 79-90 CrossRef Google Scholar

[25] Xu T, Cui P. Data fusion of integrated navigation system based on confidence weighted. Acta Aeronaut Et Astronaut Sin, 2007, 28: 1389-1394 Google Scholar

[26] Snidaro L, Foresti G L, Niu R X, et al. Sensor fusion for video surveillance. In: Proceedings of the 7th International Conference on Information Fusion, Stockholm, 2004, 2: 739--746. Google Scholar

[27] Hartley R, Zisserman A. Multiple View Geometry in Computer Vision. 2nd ed. New York: Cambridge Univercity Press, 2004. Google Scholar

[28] Shen X J, Luo Y T, Zhu Y M, et al. Globally optimal distributed Kalman filtering fusion. Sci China Inf Sci, 2012, 55: 512-529 CrossRef Google Scholar

Copyright 2020 Science China Press Co., Ltd. 《中国科学》杂志社有限责任公司 版权所有