logo

SCIENTIA SINICA Informationis, Volume 49, Issue 5: 520-537(2019) https://doi.org/10.1360/N112018-00318

UAV sense and avoidance: concepts, technologies, and systems

More info
  • ReceivedNov 30, 2018
  • AcceptedFeb 28, 2019
  • PublishedMay 15, 2019

Abstract

Unmanned aerial vehicle (UAV) sense and avoidance is one of the core technologies for integrating UAV into airspace, and it is a significant symbol of UAV autonomy and intelligence. Critical technologies and systems have been gradually developed over the past few years. We review studies related to the concept, techniques, and systems on UAV sense and avoidance, which is based on policies, regulations, and techniques. Moreover, we review the environment sensing technology related to collision avoidance path planning and maneuver control. We also propose two different systematic levels of sense-and-avoid (SAA) considering the differences in platform properties, application scenarios, and operational manners of the large/middle-sized and small/micro-sized UAVs. Furthermore, two different system paradigms, i.e., the hierarchical paradigm and reactive paradigm, are proposed to guide the system design of SAA. Finally, our study provides the prospect of sense and avoidance based on policies and regulations, technologies, and system design.


Funded by

国家自然科学基金(61603303,61473230)


References

[1] Hu Z P, Tian B C. Battlefield application and development trend of drones under informatization conditions. Aerod Missile J, 2011, 10: 63--65. Google Scholar

[2] Li D R, Li M. Research advance and application prospect of unmannedaerial vehicle remote sensing system. Geomat Inform Sci Wuhan Univ, 2014, 39: 505--513. Google Scholar

[3] Guo S Y. Research on earth observation coverage for multiple unmanned rotorcraft. Dissertation for Master Degree. Hangzhou: Zhejiang University, 2014. Google Scholar

[4] Gao J S, Zou Q Y, Chen S D. Research on anti-terrorism of foreign UAV system. In: Proceedings of 2006 China UAV Congress, 2006. Google Scholar

[5] Chen T. Design and research on visual positioning and tracking system for the small unmanned helicopter. Dissertation for Master Degree. Hangzhou: Zhejiang University, 2013. Google Scholar

[6] Huerta M. Integration of civil unmanned aircraft systems (UAS) in the national airspace system (NAS) roadmap. Federal Aviation Administration, Retrieved Dec. 2013, 19: 2013. Google Scholar

[7] Li Y J, Pan Q, Yang F, et al. Multi-source information fusion for sense and avoidance of UAV. In: Proceedings of the 29th Chinese Control Conference, Beijing, 2010. Google Scholar

[8] Part FAR. 91, General Operating and Flight Rules. Federal Aviation Administration. 1991. Google Scholar

[9] Weatherington D, Deputy U. Unmanned aircraft systems roadmap, 2005-2030. Deputy, UAV Planning Task Force, OUSD (AT&L). 2005. Google Scholar

[10] Verstraeten J, Stuip M, van Birgelen T. Assessment of detect and avoid solutions for use of unmanned aircraft systems in nonsegregated airspace. In: Handbook of Unmanned Aerial Vehicles. Berlin: Springer, 2015. 1955--1979. Google Scholar

[11] Angelov P. Sense and Avoid in UAS: Research and Applications. Hoboken: Wiley, 2012. Google Scholar

[12] Billingsley T B. Safety analysis of TCAS on Global Hawk using airspace encounter models. Cambridge: Massachusetts Institute of Technology, 2006. Google Scholar

[13] Sahawneh L R, Duffield M O, Beard R W. Detect and Avoid for Small Unmanned Aircraft Systems Using ADS-B. Air Traffic Control Q, 2015, 23: 203-240 CrossRef Google Scholar

[14] Fasano G, Accardo D, Moccia A. Multi-Sensor-Based Fully Autonomous Non-Cooperative Collision Avoidance System for Unmanned Air Vehicles. J Aerospace Computing Inf Communication, 2008, 5: 338-360 CrossRef Google Scholar

[15] Lyu Y, Pan Q, Zhao C H, et al. Autonomous stereo vision based collision avoid system for small UAV. In: Proceedings of AIAA Information Systems, 2017. Google Scholar

[16] Ozaslan T, Loianno G, Keller J. Autonomous Navigation and Mapping for Inspection of Penstocks and Tunnels With MAVs. IEEE Robot Autom Lett, 2017, 2: 1740-1747 CrossRef Google Scholar

[17] Lester T, Cook S, Noth K. USAF Airborne Sense and Avoid (ABSAA) Airworthiness and Operational Approval Approach. 2014. http://www.mitre.org/sites/default/files/publications/usaf-airborne-sense-avoid-13-3116.pdf. Google Scholar

[18] Dalamagkidis K, Valavanis K P, Piegl L A. Current Status and Future Perspectives for Unmanned Aircraft System Operations in the US. J Intell Robot Syst, 2008, 52: 313-329 CrossRef Google Scholar

[19] Zeitlin A, Lacher A, Kuchar J, et al. Collision avoidance for unmanned aircraft: proving the safety case. JAIDS J Acq Immun Def Synd, 2006, 21: 49--57. Google Scholar

[20] Accardo D, Fasano G, Forlenza L. Flight Test of a Radar-Based Tracking System for UAS Sense and Avoid. IEEE Trans Aerosp Electron Syst, 2013, 49: 1139-1160 CrossRef ADS Google Scholar

[21] Owen M P, Duffy S M, Edwards M W. Unmanned aircraft sense and avoid radar: surrogate flight testing performance evaluation. In: Proceedings of IEEE Radar Conference, 2014. Google Scholar

[22] Newmeyer L, Wilde D, Nelson B, et al. Efficient processing of phased array radar in sense and avoid application using heterogeneous computing. In: Proceedings of the 26th International Conference on Field Programmable Logic and Applications (FPL), 2016. Google Scholar

[23] Yu H B. Research on the key technology of MMW radar for power line detection. Dissertation for Master Degree. Nanjing: Nanjing University of Science and Technology, 2015. Google Scholar

[24] Geyer C M, Dey D, Singh S. Prototype Sense-and-Avoid System for UAVs. Technical Report CMU-RI-TR-09-09. 2009. Google Scholar

[25] Li J P, Liu K, Li J M, et al. The invention relates to a multi-rotor uav ultrasonic anti-collision system. CN205353698U, 2016-06-29. Google Scholar

[26] Dey D, Geyer C, Singh S, et al. Passive, long-range detection of aircraft: towards a field deployable sense and avoid system. In: Field and Service Robotics. Berlin: Springer, 2010. 113--123. Google Scholar

[27] Lai J, Mejias L, Ford J J. Airborne vision-based collision-detection system. J Field Robotics, 2011, 28: 137-157 CrossRef Google Scholar

[28] Song K T, Huang J H. Fast optical flow estimation and its application to real-time obstacle avoidance. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), 2001. 2891--2896. Google Scholar

[29] Mori T, Scherer S. First results in detecting and avoiding frontal obstacles from a monocular camera for micro unmanned aerial vehicles. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), 2013. 1750--1757. Google Scholar

[30] Fu C, Olivares-Mendez M A, Suarez-Fernandez R. Monocular Visual-Inertial SLAM-Based Collision Avoidance Strategy for Fail-Safe UAV Using Fuzzy Logic Controllers. J Intell Robot Syst, 2014, 73: 513-533 CrossRef Google Scholar

[31] Heng L, Meier L, Tanskanen P, et al. Autonomous obstacle avoidance and maneuvering on a vision-guided MAV using on-board processing. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), 2011. 2472--2477. Google Scholar

[32] Hart P, Nilsson N, Raphael B. A Formal Basis for the Heuristic Determination of Minimum Cost Paths. IEEE Trans Syst Sci Cyber, 1968, 4: 100-107 CrossRef Google Scholar

[33] Li J, Sun X X. A route planning's method for unmanned aerial vehicles based on improved A-Star algorithm. Acta Armament Arii, 2008, 29: 788--792. Google Scholar

[34] Van T J. Development of an autonomous avoidance algorithm for UAVs in general airspace. In: Proceedings of the 1st CEAS European Air and Space Conference, 2007. Google Scholar

[35] Wang X, Yadav V, Balakrishnan S N. Cooperative UAV Formation Flying With Obstacle/Collision Avoidance. IEEE Trans Contr Syst Technol, 2007, 15: 672-679 CrossRef Google Scholar

[36] Saunders J B, Call B, Curtis A, et al. Static and dynamic obstacle avoidance in miniature air vehicles. In: Proceedings of AIAA Infotech, 2005. Google Scholar

[37] Lin Y, Saripalli S. Sense and avoid for unmanned aerial vehicles using ADSB. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), 2015. 6402--6407. Google Scholar

[38] Tu J, Yang S X. Genetic algorithm based path planning for a mobile robot. In: Proceedings of IEEE International Conference on Robotics and Automation, 2003. 1221--1226. Google Scholar

[39] Durand N. Neural nets trained by genetic algorithms for collision avoidance. Appl Intelligence, 2000, 13: 205-213 CrossRef Google Scholar

[40] Ma Y H, Zhou D Y. A chaotic genetic algorithm (CGA) for path planning of UAVs. J Northwestern Polytechnical Univ, 2006, 24: 468--471. Google Scholar

[41] Duan H, Zhang X, Wu J. Max-Min Adaptive Ant Colony Optimization Approach to Multi-UAVs Coordinated Trajectory Replanning in Dynamic and Uncertain Environments. J Bionic Eng, 2009, 6: 161-173 CrossRef Google Scholar

[42] Liu S, Mao L, Yu J. Path planning based on ant colony algorithm and distributed local navigation for multi-robot systems. In: Proceedings of IEEE International Conference on Mechatronics and Automation, 2006. 1733--1738. Google Scholar

[43] Lu L, Gong D. Robot path planning in unknown environments using particle swarm optimization. In: Proceedings of 4th International Conference on Natural Computation, 2008. 422--426. Google Scholar

[44] Richards A, How J P. Aircraft trajectory planning with collision avoidance using mixed integer linear programming. In: Proceedings of American Control Conference, 2002. 1936--1941. Google Scholar

[45] Wang Y, Zhu X P, Zhou Z, et al. UAV path following in 3-D dynamic environment. Robot, 2014, 36: 83--91. Google Scholar

[46] Neunert M, de Crousaz C, Furrer F, et al. Fast nonlinear model predictive control for unified trajectory optimization and tracking. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), 2016. 1398--1404. Google Scholar

[47] Zhu Y, Zhang T, Song J Y. Path planning for nonholonomic mobile robots using artificial potential field method. Control Theory Appl, 2010, 27: 152--158. Google Scholar

[48] Khatib O. Real-time obstacle avoidance for manipulators and mobile robots. In: Autonomous robot vehicles. Berlin: Springer, 1986. 396--404. Google Scholar

[49] Dong Z N, Chen Z L, Zhou R, et al. A hybrid approach of virtual force and $A^*$ search algorithm for UAV path replanning. In: Proceedings of the 6th IEEE Conference on Industrial Electronics and Applications, 2011. 1140--1145. Google Scholar

[50] Li W. Behavior based control of a mobile robot in unknown environments using fuzzy logicp. Control Theory Appl, 1996, 2: 153--162. Google Scholar

[51] Han S C, Bang H, Yoo C S. Proportional navigation-based collision avoidance for UAVs. Int J Control Autom Syst, 2009, 7: 553-565 CrossRef Google Scholar

[52] Mujumdar A, Padhi R. Nonlinear geometric guidance and differential geometric guidance of UAVs for reactive collision avoidance. In: Proceedings of AIAA Guidance, Navigation, and Control Conference, 2010. Google Scholar

[53] Chakravarthy A, Ghose D. Obstacle avoidance in a dynamic environment: a collision cone approach. IEEE Trans Syst Man Cybern A, 1998, 28: 562-574 CrossRef Google Scholar

[54] Carbone C, Ciniglio U, Corraro F, et al. A novel 3D geometric algorithm for aircraft autonomous collision avoidance. In: Proceedings of IEEE Conference on Decision and Control, 2006. 1580--1585. Google Scholar

[55] Shanmugavel M, Tsourdos A, White B. Co-operative path planning of multiple UAVs using Dubins paths with clothoid arcs. Control Eng Practice, 2010, 18: 1084-1092 CrossRef Google Scholar

[56] Sutton R S, Barto A G. Reinforcement Learning: An Introduction. Cambridge: MIT Press, 1998. Google Scholar

[57] La H M, Lim R, Sheng W. Multirobot Cooperative Learning for Predator Avoidance. IEEE Trans Contr Syst Technol, 2015, 23: 52-63 CrossRef Google Scholar

[58] Hung S M, Givigi S N. A Q-Learning Approach to Flocking With UAVs in a Stochastic Environment.. IEEE Trans Cybern, 2017, 47: 186-197 CrossRef PubMed Google Scholar

[59] Long P, Liu W, Pan J. Deep-Learned Collision Avoidance Policy for Distributed Multiagent Navigation. IEEE Robot Autom Lett, 2017, 2: 656-663 CrossRef Google Scholar

[60] van den Berg J, Lin M, Manocha D. Reciprocal velocity obstacles for real-time multiagent navigation. In: Proceedings of IEEE International Conference on Robotics and Automation, 2008. 1928--1935. Google Scholar

[61] Chen Y F, Liu M, Everett M, et al. Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), 2017. 285--292. Google Scholar

[62] Lyu Y, Pan Q, Hu J W, et al. Multi-vehicle flocking control with deep deterministic policy gradient method. 2018,. arXiv Google Scholar

[63] Liang B, Hong B R, Shu G. Research on vision-action model of autonomous robot and obstacle avoiding. Acta Electron Sin, 2003, 31: 2197--2200. Google Scholar

[64] Aguilar W G, Casaliglla V P, Pólit J L, et al. Obstacle avoidance for flight safety on unmanned aerial vehicles. In: Proceedings of International Work-Conference on Artificial Neural Networks, 2017. 575--584. Google Scholar

[65] Wilson M. Ground-based sense and avoid support for unmanned aircraft systems. In: Proceedings of Congress of the International Council of the Aeronautical Sciences (ICAS), 2012. Google Scholar

[66] Chamberlain L J, Scherer S, Singh S. Self-Aware Helicopters: Full-Scale Automated Landing and Obstacle Avoidance in Unmapped Environments. In: Proceedings of AHS Forum, 2011. Google Scholar

[67] Utt J, McCalmont J, Deschenes M. Development of a sense and avoid system. In: Proceedings of AIAA, 2005. Google Scholar

[68] Zarandy A, Nagy Z, Vanek B, et al. A five-camera vision system for UAV visual attitude calculation and collision warning. In: Proceedigns of International Conference on Computer Vision Systems, 2013. 11--20. Google Scholar

[69] Miller P C. General atomics successfully tests UAS sense-and-avoid system. Unmanned Aircraft System Magazine, 2014. Google Scholar

[70] Chaumette F, Hutchinson S. Visual servo control. IEEE Robot Automat Mag, 2006, 13: 82-90 CrossRef Google Scholar

[71] Mnih V, Kavukcuoglu K, Silver D. Human-level control through deep reinforcement learning. Nature, 2015, 518: 529-533 CrossRef PubMed ADS Google Scholar

[72] Tian J, Shen L C. Research on multi-base multi-UAV cooperative reconnaissance problem. Acta Aeronaut Et Astron Autica Sin, 2007, 28: 913--921. Google Scholar

[73] Ye X F. A HRI method based on stereo vision and deep learning for UAV. Dissertation for Master Degree. Tianjin: Tianjin University, 2017. Google Scholar

[74] Alvarez H, Paz L M, Sturm J, et al. Collision avoidance for quadrotors with a monocular camera. In: Experimental Robotics. Berlin: Springer, 2016. 195--209. Google Scholar

[75] Dey D, Shankar K S, Zeng S, et al. Vision and learning for deliberative monocular cluttered flight. In: Field and Service Robotics. Berlin: Springer, 2016. 391--409. Google Scholar

[76] Lyu Y, Pan Q, Zhao C. Vision-based UAV collision avoidance with 2D dynamic safety envelope. IEEE Aerosp Electron Syst Mag, 2016, 31: 16-26 CrossRef Google Scholar

[77] Oleynikova H, Honegger D, Pollefeys M. Reactive avoidance using embedded stereo vision for MAV flight. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), 2015. 50--56. Google Scholar

[78] McGuire K, de Croon G, De Wagter C. Efficient Optical Flow and Stereo Vision for Velocity Estimation and Obstacle Avoidance on an Autonomous Pocket Drone. IEEE Robot Autom Lett, 2017, 2: 1070-1076 CrossRef Google Scholar

[79] Iacono M, Sgorbissa A. Path following and obstacle avoidance for an autonomous UAV using a depth camera. Robotics Autonomous Syst, 2018, 106: 38-46 CrossRef Google Scholar

[80] Hu J, Niu Y F, Wang Z C. Obstacle avoidance methods for rotor UAVs using RealSense camera. In: Proceedings of Chinese Automation Congress (CAC), 2017. 7151--7155. Google Scholar

[81] Upton E, Halfacree G. Raspberry Pi User Guide. Hoboken: John Wiley and Sons, 2014. Google Scholar

[82] Weiss A, Rosenthal M, Mazloumian A. Realtime signal processing on NVIDIA TX2 using CUDA. In: Proceedings of Nvidia GPU Technology Conference (GTC), 2018. Google Scholar

[83] Rabl A, Salner P, Büchi L, et al. Implementation of a capacitive proximity sensor system for a fully maneuverable modular mobile robot to evade humans. In: Proceedings of Austrian Robotics Workshop, 2018. Google Scholar

[84] Grzonka S, Grisetti G, Burgard W. A Fully Autonomous Indoor Quadrotor. IEEE Trans Robot, 2012, 28: 90-100 CrossRef Google Scholar

[85] Mohta K, Watterson M, Mulgaonkar Y. Fast, autonomous flight in GPS-denied and cluttered environments. J Field Robotics, 2018, 35: 101-120 CrossRef Google Scholar

[86] Gageik N, Benz P, Montenegro S. Obstacle Detection and Collision Avoidance for a UAV With Complementary Low-Cost Sensors. IEEE Access, 2015, 3: 599-609 CrossRef Google Scholar

  • Figure 1

    (Color online) SAA function flow

  • Figure 2

    UAV SAA environment sensing technology

  • Figure 3

    (Color online) The hierarchical SAA paradigm for large & middle size UAV

  • Figure 4

    (Color online) The reactive SAA paradigm for small & micro size UAV

  • Table 1   UAV SAA sensor configuration
    Platform T-CAS ADS-B Ground radar Airborne radar Lidar Man eye EO IR IRST Ultra-Sonic Stereo MWR
    HALE √* √* √*
    MALE √* √* √*
    TUAV √*
    SUAV √*
    MAV √* √*

    a) represents the sensor applicable to the UAV; * represents the sensors that should be mounted according to the policies and regulations in the future application of UAV.

  • Table 2   Global path planning & reactive collision avoidance
    Global path planning Reactive collision avoidance
    Platform HALE & MALE UAV Small & micro UAV
    Operation space Sparse mid-air scenario Complex low-altitude scenario
    Sensing information Global, wide range target status Local, limited range target measurements
    Control output Global way-points Local control command
    Sensing & control Almost decoupled Coupled
    Loop frequency Low High
  • Table 3   Large & middle size UAV safety capacity level
    Safety level Description
    0 Does not have any security assurance capability and technical means
    1
    Local flight safety support under specific mission functions include:
    (a) limited perceptionability and partial communication ability in VMC environment;
    (b) decision rules and operation methodsin simple task environment;
    (c) and simple flight control operation method and command realization.
    2
    Wide range and long endurance flight safety support in a simple and sparse flightenvironment:
    (a) wide range sensing and multi-link communication capabilities in a VMC condition;
    (b) situation assessment and decision making in a wide range of mission environments;
    (c) and fully controllable task execution.
    3
    Safety flight insurance in complex meteorological conditions and flight environment:
    (a) perception ability and communication link under VMC and IMC;
    (b) environmental situation analysis and fault diagnosis under complex flight conditions;
    (c) and controllable task instruction execution under man's supervision function.
    4
    All-weather flight safety support under complex air traffic control system:
    (a) large-scale perception and cooperative air traffic information under VMC and IMC;
    (b) multi-information support for flight situation analysis and platform health management functions;
    (c) and access to ATC system, complete flight status acquisition and flight command execution.
    5
    Equivalent safe flight in shared airspace with man-machine:
    (a) large-scale perception andhighly reliable information interaction under VMC and IMC;
    (b) highly reliable situationanalysis and decision-making punder the support of multiple information
    sources (more thanpeople in the loop);
    (c) and seamless access to the ATM (air traffic management) system to achieve the mission flight
    under the ATCoperating rules.
  • Table 4   Small & micro UAV safety capacity level
    Safety level Description
    0 Visual line of sight flight with man in the loop remote control (100% man in the loop).
    1
    Partial perception and auxiliary control functions:
    (a) auxiliary navigation andenvironment perception in sparse environment;
    (b) low level auxiliary judgment and warning ability;
    (c) and people in the loop control, with auxiliary functions of the automatic driving system.
    2
    Man in the loop decision making (partial man in the loop):
    (a) auxiliary navigation, environment and target modeling in dense environment;
    (b) auxiliary decision-making function and safety warning ability;
    (c) man in the loop path planning and maneuver control.
    3
    Automatic mission execution with man in the loop supervision:
    (a) reliable navigation and data processing in the task environment;
    (b) supervised online data analysis and decision-making function;
    (c) online path planning and control.
    4
    Autonomous task execution:
    (a) intelligent information processing and environmental perception;
    (b) autonomous analysis and decision-making function;
    (c) optimal online path planning and maneuver control.
    5
    Collaborative task execution ability with multiple platforms:
    (a) multi-platform collaborative perceptionand data analysis;
    (b) collaborative situational awareness and decision-making;
    (c) distributed multi-platform path planning and control.

Copyright 2019 Science China Press Co., Ltd. 《中国科学》杂志社有限责任公司 版权所有

京ICP备18024590号-1