SCIENTIA SINICA Informationis, Volume 48 , Issue 8 : 1083-1096(2018) https://doi.org/10.1360/N112018-00028

Interactive cognition in self-driving

More info
  • ReceivedFeb 3, 2018
  • AcceptedMar 13, 2018
  • PublishedMay 22, 2018


In recent years, self-driving approaches, such as Tesla, Waymo, and Intel, have focused on computational awareness, including cognitive perception, planning, and decision making. However, little effort has been concentrated on interactive cognition. This problem is important for unmanned vehicles to be accepted by society. Targeting on this urgent issue, we analyze the richness and difficulties of the interactive cognition of self-driving vehicles. The interactive cognition of self-driving can be divided into the vehicle intelligent speaker dialogue based on natural language interaction, body language interaction, and vehicle body language interaction. Through the interactive cognition of self-driving, it can employ intelligent speakers and the Internet to achieve the interaction between self-driving vehicles and owners, crews, operation and maintenance personnel, developers, remote service requests, etc. This paper solves the gesture recognition and understanding of pedestrians and road traffic police, and also overcomes two typical body interaction problems, including the car-meeting task at narrow roads and the car overtaking/merging task. Finally, based on many self-driving interactive cognitions, we provide an interactive bus architecture that is independent of a decision bus, and apply it in various types of smart cars.

Funded by






本文撰写过程中, 得到清华大学戴琼海院士、北京联合大学李学伟教授、鲍泓教授、杨鹏教授、杜煜教授、刘元盛教授、程光教授、刘宏哲教授等的大力支持和帮助, 在此表示最诚挚的感谢. 北京联合大学徐歆恺老师、张欢老师、郑颖、逄桂林参与论文资料整理, 在此一并感谢.


[1] Li D Y. Opening, innovation and industrial transformation, starting with the intelligent driving. Sci-tech Innov Brands, 2014, 6: 24. Google Scholar

[2] Li D Y, Zheng S Y. Practice and prospect of wheeled robots. Sci Technol Rev, 2015, 33: 52--54. Google Scholar

[3] Berawi M A. Stimulating Innovation and Creativity: The Way Forward. IJTech, 2017, 8: 1-19 CrossRef Google Scholar

[4] Zhang X Y, Gao H B, Guo M. A study on key technologies of unmanned driving. CAAI Trans Intel Technol, 2016, 1: 4-13 CrossRef Google Scholar

[5] Bojarski M, Testa D D, Dworakowski D, et al. End to end learning for self-driving cars. 2016,. arXiv Google Scholar

[6] Kesting A. Microscopic Modeling of Human and Automated Driving: Towards Traffic-Adaptive Cruise Control. Saarbrücken: Vdm Verlag Dr Müller, 2008. Google Scholar

[7] Urmson C, Anhalt J, Bagnell D. Autonomous driving in urban environments: boss and the urban challenge. J Field Robot, 2008, 25: 425-466 CrossRef Google Scholar

[8] SAE On-Road Automated Vehicle Standards Committee. Taxonomy and definitions for terms related to on-road motor vehicle automated driving systems. SAE Standard J3016, 2014. Google Scholar

[9] Liberty E, Mazzae E, Garrott R, et al. NHTSA driver distraction research: past present and future. In: Proceedings of the 17th International Technical Conference on the Enhanced Safety of Vehicles, Amsterdam, 2001. Google Scholar

[10] Hörl S, Ciari F, Axhausen K W. Recent perspectives on the impact of autonomous vehicles. 2016. https://www.research-collection.ethz.ch/bitstream/handle/20.500.11850/121359/2/ab1216.pdf. Google Scholar

[11] Gu Y P. Intelligent voice assistant system based on intelligent voice communication. Dissertation for Master Degree. Nanjing: Nanjing University of Posts and Telecommunications, 2015. Google Scholar

[12] Huisman D, Freling R, Wagelmans A P M. A robust solution approach to the dynamic vehicle scheduling problem. Transport Sci, 2004, 38: 447-458 CrossRef Google Scholar

[13] Google. Determining pickup and destination locations for autonomous vehicles. P20160370194, 2016. Google Scholar

[14] Mitra S, Acharya T. Gesture recognition: a survey. IEEE Trans Human Mach Syst, 2007, 37: 311-324 CrossRef Google Scholar

[15] Hasan H S, Kareem S A. Human computer interaction for vision based hand gesture recognition: a survey. In: Proceedings of the 2012 International Conference on Advanced Computer Science Applications and Technologies, Kuala Lumpur, 2012. Google Scholar

[16] Rautaray S S, Agrawal A. Vision based hand gesture recognition for human computer interaction: a survey. Artif Intel Rev, 2015, 43: 1-54 CrossRef Google Scholar

[17] Candamo J, Shreve M, Goldgof D B. Understanding transit scenes: a survey on human behavior-recognition algorithms. IEEE Trans Intel Transport Syst, 2010, 11: 206-224 CrossRef Google Scholar

[18] Poppe R. A survey on vision-based human action recognition. Image Vision Comput, 2010, 28: 976-990 CrossRef Google Scholar

[19] Wang J, Liu Z C, Wu Y. Learning actionlet ensemble for 3D human action recognition. IEEE Trans Pattern Anal Mach Intel, 2014, 36: 914-927 CrossRef PubMed Google Scholar

[20] Vijayan I, Laur M H, Absmeier J P. Automated vehicle operation based on gesture to pedestrian. US20170190336, 2017. Google Scholar

[21] Liu C, Tomizuka M. Enabling safe freeway driving for automated vehicles. In: Proceedings of the American Control Conference, Boston, 2016. Google Scholar

[22] Xu H B. Single track vehicle closed-loop system modeling simulation analysis. Dissertation for Master Degree. Jilin: Jilin University, 2012. Google Scholar

[23] Li D Y, Han W, Zheng S Y, et al. Intelligent vehicle distributed process virtual switch communication method. CN 104994166 A, 2015. Google Scholar

  • Table 1   SAE J3016 for automated driving systems levels

Copyright 2020 Science China Press Co., Ltd. 《中国科学》杂志社有限责任公司 版权所有

京ICP备17057255号       京公网安备11010102003388号