logo

SCIENTIA SINICA Informationis, Volume 49, Issue 5: 507-519(2019) https://doi.org/10.1360/N112018-00316

Review of scene matching visual navigation for unmanned aerial vehicles

More info
  • ReceivedNov 29, 2018
  • AcceptedFeb 28, 2019
  • PublishedMay 14, 2019

Abstract

High precision location and navigation is one of the key techniques of autonomous flight, efficient reconnaissance, and precision strike of unmanned aerial vehicles (UAVs). Combined with inertial system, scene matching visual navigation technology can be used in a highly precise autonomous navigation system because of its simple structure, passive working mode, and high locating precision. Many previous studies on visual navigation systems of UAVs focus on extracting information of aircraft attitude and navigation using visual information, or the combination of the navigation system and IMU sensors. In China, numerous researches have focused on the technologies of the inertial-combined navigation, 3D map reconstruction with image or laser, and visual navigation landing application. In this study, we summarize the characteristics, categories, and analytical methods of scene matching visual navigation for UAVs. Then, we conclude the primary techniques in high precision, strong real-time, robust and continuous visual navigation tasks under different time, visual angle, illumination, resolution, platform, and sensor.


Funded by

国家自然科学基金(61473230,61603303)

陕西省自然基金(2017JM6027,2017JQ6005)

地理信息工程国家重点实验室开放基金(SKLGIE2015-M-3-4)


References

[1] Jordan M J. Merging the tribes: streamlining DoD's acquisition of unmanned aerial systems. In: Proceedings of Us Defense Technology Report: ADA449383, 2006. 1--213. Google Scholar

[2] Dong J W. Analysis on Inertial Navigation Technology. Instrum Tech, 2017, (01): 41--43. Google Scholar

[3] Gu Y T. Research on navigation system of unmanned aircraft vehicle. Modern Navigation, 2013, (3): 198--201. Google Scholar

[4] Carr J R, Sobek J S. Digital scene matching area corre-lator (DSMAC). In: Proceedings of SPIE Image Processing for Missile Guidance, 1980. 238: 36--41. Google Scholar

[5] Reed C G, Hocan J J. Range Correlation Guidance for Cruise Missiles. IEEE Trans Aerosp Electron Syst, 1979, AES-15: 547-554 CrossRef Google Scholar

[6] Soundararaj S P, Sujeeth A K, Saxena A. Autonomous indoor helicopter flight using a single onboard camera. In: Proceedings of International Conference on Intelligent Robots and Systems (IROS), 2009. 1--7. Google Scholar

[7] Byrne J, Cosgrove M, Mehra R. Stereo based obstacle detection for an unmanned air vehicle. In: Proceedings of IEEE Interna-tional Conference on Robotics and Automation (ICRA), 2006. 2594--2599. Google Scholar

[8] Thrun S, Diel M, Hahnel D. Scan align-ment and 3-D surface modeling with a helicopter plat-form. In: Proceedings of the 4th International Conference on Field and Service Robotics, 2003. Google Scholar

[9] Oh P Y, Green W E, Barrows G. Neural nets and optic flow for autonomous micro-air-vehicle navigation. In: Proceedings of ASME International Mechanical Engineering Congress and Exposition, Anaheim, 2004. 1279--1285. Google Scholar

[10] Metni N, Hamel T, Derkx F. Visual tracking control of aerial robotic systems with adaptive depth estimation. Int J Control Autom Syst, 2007, 5: 51--60. Google Scholar

[11] Conroy J, Gremillion G, Ranganathan B. Erratum to: Implementation of wide-field integration of optic flow for autonomous quadrotor navigation. Auton Robot, 2009, 27: 199-200 CrossRef Google Scholar

[12] Stowers J, Bainbridge-Smith A, Hayes M. Optical Flow for Heading Estimation of a Quadrotor Helicopter. Int J Micro Air Vehicles, 2009, 1: 229-239 CrossRef Google Scholar

[13] Weiss S, Achtelik M, Kneip L. Intuitive 3D Maps for MAV Terrain Exploration and Obstacle Avoidance. J Intell Robot Syst, 2011, 61: 473-493 CrossRef Google Scholar

[14] Kleinert M, Schleith S. Inertial aided mo-nocular SLAM for GPS-denied navigation. In: Proceedings of 2010 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, Salt Lake City, 2010. 20--25. Google Scholar

[15] Michael Sobers JrD, Johnson Eric N. Indoor naviga-tion for micro aerial vehicles. In: Proceedings of AIAA Infotech at Aerospace, Atlanta, 2010. 1--29. Google Scholar

[16] CelikK, Chung S-J, Somani A. Mono-vision corner SLAM for indoor navigation. In: Proceedings of IEEE International Conference Ames IA: Electro/Information Technology, 2008. 343--348. Google Scholar

[17] Kong W Y, Egan G K, Cornall T. Feature based naviga-tion for UAVs. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, 2006. Google Scholar

[18] Dong-Gyu Sim , Sang-Yong Jeong , Doh-Hyeong Lee . Hybrid estimation of navigation parameters from aerial image sequence. IEEE Trans Image Process, 1999, 8: 429-435 CrossRef PubMed ADS Google Scholar

[19] Dong-Gyu Sim , Rae-Hong Park , Rin-Chul Kim . Integrated position estimation using aerial image sequences. IEEE Trans Pattern Anal Machine Intell, 2002, 24: 1-18 CrossRef Google Scholar

[20] Conte G, Doherty P. An integrated UAV navigation system based on aerial image matching. In: Proceedings of IEEE Aerospace Conference, 2008. Google Scholar

[21] Zingg S, Scaramuzza D, Weiss S. MAV navigation through indoor corridors using optical flow. In: Proceedings of 2010 IEEE International Conference on Robotics and Automation Anchorage Convention District, Anchorage, 2010. 3361--3368. Google Scholar

[22] Vasconcelos J F, Silvestre C, Oliveira P. Embedded UAV model and LASER aiding techniques for inertial navigation systems. Control Eng Practice, 2010, 18: 262-278 CrossRef Google Scholar

[23] Diao C, Wang Y X, Wang J T, Miao M. Computer vision assisted autonomous landing of USV. J Syst Simulat, 2008, S1: 495-498. Google Scholar

[24] Zhou F Q, Zhang G J. Vision guiding method with monocular camera. Chin J Sci Instrum, 2006, 27: 1129--1132. Google Scholar

[25] Zhang B H, Cai Z H, Wang Y X. Binocular stereo vision navigation for electric VTOL aircraft. In: Proceedings of 2010 Chinese Guidance, Navigation and Control Conference, Shanghai, 2010. 298--303. Google Scholar

[26] Fang J C, Zhang X. Development of autopilot technology of small UAV. J Chin Inertial Tech, 2007, 15: 658--663. Google Scholar

[27] Xu C, Fan Y Z, Sheng X R, et al. Vision based navigation system of UAV and improvements of the corresponding filtering algorithm. J Beijing Univ Aeronaut Astronautics, 2010, 36: 1000--1004. Google Scholar

[28] Pan X, Ma D Q, Wu Y J. Estimating pitch attitude and altitude of unmanned aerial vehicle vision-based landing. J Zhejiang Univ, 2009, 43: 692--696. Google Scholar

[29] Ren Q Y, Li P, Han B. Vision-based mini unmanned helicopter pose and position estimation. J Zhejiang Univ (Eng Sci), 2001, 43: 18--22. Google Scholar

[30] Xiong R, Chu J, Wu J. Incremental mapping based on dot-line congruence for robot. Control Theory Appl, 2007, 24: 170--176. Google Scholar

[31] Wu X L, Shi Z Y, Zhong Y S. An overview of vision-based UAV navigation. J Syst Simulat, 2010, S1: 62-65. Google Scholar

[32] Li Y J, Pan Q, Zhao C H, et al. Scene matching navigation for UAV based on spatial relationship geometric constraints. Appl Res Comput, 2010, 27: 3822--3846. Google Scholar

[33] Yu Q F, Li Q, Lei Z H, et al. Methods and experiments of velocity measurement of UAV from image sequence. Acta Aeronaut ET Astronaut Sin, 2009, 30: 1503--1507. Google Scholar

[34] Wu Q Y, Zhou D X, Cai X P. Construction of virtual environment from image sequences. J Image Graph, 2006, 11: 113--118. Google Scholar

[35] Guo Q. Scene matching technology development outline. Infrared Laser Eng (Supplement), 2007, 36: 57--61. Google Scholar

[36] Zhang Y Y, Su J, Li B, et al. Research on scene matching flexibility of scene matching navigation. In: National Security Geophysics Series. 2015. Google Scholar

[37] Zitova B, Flusser J. Image registration methods: a sur-vey. Image Vision Comput, 2003, 21: 9771000. Google Scholar

[38] Meth R, Chellappa R. Stability and sensitivity of topo-graphic features for SAR target characterization. In: Proceedings of IEEE International Conference on Image Processing, 1997. 3: 464--467. Google Scholar

[39] Cave R G, Harley P J, Quegan S. Registering SAR images to digital map data using a template matching technique. In: Proceedings of Remote Sensing: Global Monitoring for Earth Management, Espoo, 2002. 1429--1431. Google Scholar

[40] Liu Z H, Wang H, Chen B G. Research on selection for scene matching area. Comput Tech Develop, 2013, (12): 128--133. Google Scholar

[41] Johnson M. Analytical development and test results of acquisition probability for terrain correlation devices used in navigation systems. In: Proceedings of Aerospace Sciences Meeting, San Diego, 2013. Google Scholar

[42] Liu Y, Zhao F W, Jin S L. New method of selecting scene matching reference map. Infrared Laser Eng, 2001, 30: 168--229. Google Scholar

[43] An R, Jin X L, Wang H L, et al. Image ability to obtain correct matching based on feature matching. Infrared Laser Eng, 2005, 34: 469--473. Google Scholar

[44] Guo W P, Sun J Y, He F F, et al. Platform for adapta-bility analysis of scene matching based on database. Infrared Laser Eng, 2006, 35: 738--741. Google Scholar

[45] Jiang B, Chen Y. A Rule of Selecting Scene Matching Area. J Tongji Univ, 2007, 35: 830--833. Google Scholar

[46] Huang J L, Tan X Z, Zhang X Y. Selection of adaptive area in geomagneticnavigation based on the principal component analysis. J Naval Aeronaut Astronaut Univ, 2018, 159: 17--23. Google Scholar

[47] Xu X S, Tang J J, Zhang T, et al. Selection for matching area in terrain aided navigation based on entropy-weighted grey correlation decision-making. J Chin Inertial Tech, 2015, 23: 201--206. Google Scholar

[48] Li J, Yang X, Yang L. Mumford-Shah model based critical subset segmentation from navigation reference images. Acta Automat Sin, 2004, 30: 45--56. Google Scholar

[49] Zhang Y Y, Su J, Li B, et al. SAR scene matching area selection based on multi-attribute comprehensive analysis. J Projectiles, Rockets, Missiles Guidance, 2016, 36: 104--108. Google Scholar

[50] Luo H B, Chang Z, Yu X R, et al. Automatic suitable-matching area selection method based on multi-feature fusion. Infrared Laser Eng, 2011, 40: 2037--2041. Google Scholar

[51] Li J, Yang X. Approach to selection of suitable-matching area from reference image. J Data Acquisition Process, 2000, 15: 495--499. Google Scholar

[52] Su K, Guan S Y, Liu J. A practical normalized cross correlation algorithm for scene matching systems. J Astronaut, 1997, (3): 1--7. Google Scholar

[53] An R, Wang H L. Investigation of image matching algorithm for aircraft guidance. In: Proceedings of the 13th National Conference on Image and Graphics, Nanjing, 2006. Google Scholar

[54] Huttenlocher D P, Klanderman G A, Rucklidge W J. Comparing images using the Hausdorff distance. IEEE Trans Pattern Anal Machine Intell, 1993, 15: 850-863 CrossRef Google Scholar

[55] Leng X F, Liu J Y, Xiong Z. A real-time image matching algorithm for navigation system based on bifurcation extraction. Act Automat Sin, 2007, 33: 678--682. Google Scholar

[56] Yu S S, Tsai W H. Relaxation by the Hopfield neural network. Pattern Recognition, 1992, 25: 197-209 CrossRef Google Scholar

[57] Shen W M, Zhang Z X, Zhang J Q. Image matching by neural network based relaxation. J Wuhan Tech Univ Surv Mappiing, 1996, 21: 247--251. Google Scholar

[58] Sang N, Zhang T X, Wei L G. Relaxation Matching by the Neural Network. J Commun, 1996, 17: 52--56. Google Scholar

[59] Taha A A, Hanbury A. An efficient algorithm for calculating the exact Hausdorff distance.. IEEE Trans Pattern Anal Mach Intell, 2015, 37: 2153-2163 CrossRef PubMed Google Scholar

[60] Chen Y, He F, Wu Y. A local start search algorithm to compute exact Hausdorff Distance for arbitrary point sets. Pattern Recognition, 2017, 67: 139-148 CrossRef Google Scholar

[61] Deng W, Zou H, Guo F. Point-pattern matching based on point pair local topology and probabilistic relaxation labeling. Vis Comput, 2018, 34: 55-65 CrossRef Google Scholar

[62] Medioni G, Nevatia R. Matching Images Using Linear Features. IEEE Trans Pattern Anal Mach Intell, 1984, PAMI-6: 675-685 CrossRef Google Scholar

[63] Cucka P, Rosenfeld A. Region matching and depth finding for 3D objects in stereo aerial photographs. Pattern Recogn, 1990, 23: 81-94 CrossRef Google Scholar

[64] Su J, Lin X G, Liu D Z. A Multi-sensor Image Registration Algorithm Based on Structure Feature Edges. Acta Automatica Sin, 2009, 35: 251-257 CrossRef Google Scholar

[65] Wang W H, Chen Y C. Image registration by control points pairing using the invariant properties of line segments. Pattern Recognition Lett, 1997, 18: 269-281 CrossRef Google Scholar

[66] Yang J, Qiu J, Wang Y F, et al. Feature-based image regis-tration algorithm using invariant line moments. Acta Photonica Sin, 2003, 32: 1114--1117. Google Scholar

[67] Ou Y H, Fan D Z, Ji S, et al. Line matching based on discrete description and conjugate point constraint. Acta Geod Cartograph Sin, 2018, 47: 74--82. Google Scholar

[68] Liu G X, Liu D M, Liu F P, et al. A robust image registration algorithm based on feature points matching. Acta Opt Sin, 2008, 28: 455--461. Google Scholar

[69] An M, Jiang Z G. Fast downward-looking target matching technology on missile in large field of view. Syst Eng Electron, 2008, 30: 114--117. Google Scholar

[70] Lowe D G. Distinctive Image Features from Scale-Invariant Keypoints. Int J Comput Vision, 2004, 60: 91-110 CrossRef Google Scholar

[71] Bentoutou Y, Taleb N, Kpalma K. An Automatic Image Registration for Applications in Remote Sensing. IEEE Trans Geosci Remote Sens, 2005, 43: 2127-2137 CrossRef ADS Google Scholar

[72] Alhichri H S, Kamel M. Virtual circles: a new set of features for fast image registration. Pattern Recognition Lett, 2003, 24: 1181-1190 CrossRef Google Scholar

[73] Liu X J, Yang J, Ling J G, et al. Scene matching approach by increment sign correlation. Infrared Laser Eng, 2006, 35: 732--737. Google Scholar

[74] Tao T, Zhang Y. Detection and description of scale-invariant keypoints in log-polar space. J Image Graph, 2018, 20: 1639--1651. Google Scholar

[75] Li F Y. Application of non - uniform mapping transform technique in scene matching correlator. Dissertation for Ph.D. Degree. Xi'an: Northwestern Polytechnical University, 2007. Google Scholar

[76] Fu G, Zhao H R, Li C, et al. A Method by Improved Circular Projection Matching of Tracking Twisty Road from Remote Sensing Imagery. Acta Geodaetica et Cartographica Sinica, 2014, 43(7):724-730 DOI: 10.13485/j.cnki.11-2089.2014.0097. Google Scholar

[77] Zhao F, Huang Q M, Gao W. Image matching by multiscale oriented corner correlation. In: Proceedings of the 7th Asian Conference on Computer Vision. Berlin: Springer, 2006. 928--937. Google Scholar

[78] Qin Q, Chai Z H, Fu Y H, et al. Infrared scene matching based on ternary amplitude-phase filter. J Infrared Millimeter Waves, 2005, 24: 477--480. Google Scholar

[79] Wong L B, Tian Y, Wang Y Q, et al. Novel scene matching method for complex terrain based on three-view constraints. Acta Aeronaut Astronaut Sin, 2008, 29: 203--211. Google Scholar

[80] Qu Y, Guo X W, He P, et al. Matching extension based on colinearity and epipolar geometry constraint. Opt Tech, 2017, 43: 450--454. Google Scholar

[81] Wang Y M. A Post-processing Algorithm for Continuous Scene Matching. J Astronaut, 2004, 25: 535--540. Google Scholar

[82] Wang K P, Shang Y, Yu Q F. An image matching approach based on the invariant moment of slope angle histogram. J Comput-Aided Design Comput Graph, 2016, 21: 389--393. Google Scholar

[83] Sjahputera O, Keller J M. Particle swarm over scene matching. In: Proceedings of Swarm Intelligence Sym-posium, 2005. Google Scholar

[84] Zhang Z J, Hao X Y, Liu S L, et al. Application of improved genetic algorithm in image pattern matching. J Geomat, 2013, 38: 52--53. Google Scholar

[85] Zhang Y S, Zhu Q, Wu B, et al. A hierarchical stereo line matching method based on a triangle constraint. Geomat Inf Sci Wuhan Univ, 2013, 38: 522--527. Google Scholar

[86] Zhang X C, Zeng P, Zhang Y S. Scene matching by dual-tree complex wavelet structural similarity measurement. Modern Defense Tech, 2017, 45: 87--92. Google Scholar

[87] Sun S Y, Zhang Y, Li J Z, et al. Two-dimensional location of aerial photography based on scene matching. Syst Eng Electron, 2018, 40: 2415--2419. Google Scholar

[88] Zhang W Y, Li Z, Wang Y J. UAV scene matching algorithm based on CenSurE-star feature. Chin J Sci Instrument, 2017, 38: 462--470. Google Scholar

[89] Jin Z, Wang X, Moran B. Multi-Region Scene Matching Based Localisation for Autonomous Vision Navigation of UAVs. J Navigation, 2016, 69: 1215-1233 CrossRef Google Scholar

[90] Ling Z G, Pan Q, Zhang S W, et al. A scene matching method with weighted hausdorff distance based on edge measure. J Astronaut, 2009, 30: 1633--1638. Google Scholar

  • Figure 1

    Diagram of UAV scene matching navigation system

  • Table 1   Principles and characteristics of three types of image matching methods
    Algorithm type Principle Characteristic
    Based on gray
    Gray information for
    similarity measuremet matching
    No image segmentation and feature extraction
    Poor resistance to geometric deformation and interference
    The computation increases with the increase of image size
    Feature-based
    Extract points, lines
    and regions as features
    Feature construction and extraction are complex
    Good adaptability to deformation and occlusion
    Incomplete feature extraction leads to low accuracy
    Explanation-based
    The relational feature and
    semantic network for matching
    Semantic features are difficult to extract
    Closer to human cognitive ability
    Can improve the matching accuracy

Copyright 2019 Science China Press Co., Ltd. 《中国科学》杂志社有限责任公司 版权所有

京ICP备18024590号-1