logo

SCIENCE CHINA Information Sciences, Volume 63 , Issue 9 : 190202(2020) https://doi.org/10.1007/s11432-019-2844-3

Investigating the dynamic memory effect of human drivers via ON-LSTM

More info
  • ReceivedNov 29, 2019
  • AcceptedFeb 29, 2020
  • PublishedAug 13, 2020

Abstract

It is a widely accepted view that considering the memory effects of historical information (driving operations) is beneficial for vehicle trajectory prediction models to improve prediction accuracy. However, many commonly used models (e.g., long short-term memory, LSTM) can only implicitly simulate memory effects, but lack effective mechanisms to capture memory effects from sequence data and estimate their effective time range (ETR). This shortage makes it hard to dynamically configure the most suitable length of used historical information according to the current driving behavior, which harms the good understanding of vehicle motion. To address this problem, we propose a modified trajectory prediction model based on ordered neuron LSTM (ON-LSTM). We demonstrate the feasibility of ETR estimation based on ON-LSTM and propose an ETR estimation method. We estimate the ETR of driving fluctuations and lane change operations on the NGSIM I-80 dataset. The experiment results prove that the proposed method can well capture the memory effects during trajectory prediction. Moreover, the estimated ETR values are in agreement with our intuitions.


Acknowledgment

This work was supported in part by National Key Research and Development Program of China (Grant No. 2018AAA0101400), National Natural Science Foundation of China (Grant No. 61790565), Science and Technology Innovation Committee of Shenzhen (Grant No. JCYJ20170818092931604), and Intel Collaborative Research Institute for Intelligent and Automated Connected Vehicles (Grant No. ICRI-IACV).


References

[1] Wang X, Jiang R, Li L. Long memory is important: A test study on deep-learning based car-following model. Physica A-Statistical Mech its Appl, 2019, 514: 786-795 CrossRef ADS Google Scholar

[2] Pei X, Pan Y, Wang H. Empirical evidence and stability analysis of the linear car-following model with gamma-distributed memory effect. Physica A-Statistical Mech its Appl, 2016, 449: 311-323 CrossRef ADS Google Scholar

[3] Sipahi R, Niculescu S I. Stability of car following with human memory effects and automatic headway compensation. Proc R Soc A, 2010, 368: 4563-4583 CrossRef PubMed ADS Google Scholar

[4] Treiber M, Helbing D. Memory effects in microscopic traffic models and wide scattering in flow-density data. Phys Rev E, 2003, 68: 046119 CrossRef PubMed ADS arXiv Google Scholar

[5] Ammoun S, Nashashibi F. Real time trajectory prediction for collision risk estimation between vehicles. In: Proceedings of IEEE 5th International Conference on Intelligent Computer Communication and Processing, 2019. 417--422. Google Scholar

[6] Hillenbrand J, Spieker A M, Kroschel K. IEEE Trans Intell Transp Syst, 2006, 7: 528-540 CrossRef Google Scholar

[7] Batz T, Watson K, Beyerer J. Recognition of dangerous situations within a cooperative group of vehicles. In: Proceedings of IEEE Intelligent Vehicles Symposium, 2009. 907--912. Google Scholar

[8] Lefèvre S, Vasquez D, Laugier C. A survey on motion prediction and risk assessment for intelligent vehicles. Robomech J, 2014, 1: 1 CrossRef Google Scholar

[9] Tay M K C, Laugier C. Modelling smooth paths using gaussian processes. In: Field and Service Robotics. Berlin: Springer, 2008. 381--390. Google Scholar

[10] Tran Q, Firl J. Online maneuver recognition and multimodal trajectory prediction for intersection assistance using non-parametric regression. In: Proceedings of IEEE Intelligent Vehicles Symposium Proceedings, 2014. 918--923. Google Scholar

[11] Aoude G, Joseph J, Roy N, et al. Mobile agent trajectory prediction using Bayesian nonparametric reachability trees. Infotech@ Aerospace 2011, 2011, 1512. Google Scholar

[12] Lee J D, Moeckli J, Brown T L, et al. Distraction detection and mitigation through driver feedback. 2013. Google Scholar

[13] Zheng Y, Sathyanarayana A, Hansen J. Non-Uniform time window processing of in-vehicle signals for maneuvers recognition and route recovery. SAE Technical Paper, 2015. Google Scholar

[14] Li G, Li S E, Liao Y, et al. Lane change maneuver recognition via vehicle state and driver operation signals---Results from naturalistic driving data. In: Proceedings of IEEE Intelligent Vehicles Symposium, 2015. 865--870. Google Scholar

[15] Hochreiter S, Schmidhuber J. Long short-term memory.. Neural Computation, 1997, 9: 1735-1780 CrossRef PubMed Google Scholar

[16] Sak H, Senior A, Beaufays F. Long short-term memory recurrent neural network architectures for large scale acoustic modeling. In: Proceedings of the 15th Annual Conference of the International Speech Communication Association, 2014. Google Scholar

[17] Xing Y, Lv C, Cao D. Personalized Vehicle Trajectory Prediction Based on Joint Time-Series Modeling for Connected Vehicles. IEEE Trans Veh Technol, 2020, 69: 1341-1352 CrossRef Google Scholar

[18] Altché F, de La Fortelle A. An LSTM network for highway trajectory prediction. In: Proceedings of IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), 2017. 353--359. Google Scholar

[19] Kim B, Kang C M, Kim J, et al. Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network. In: Proceedings of IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), 2017. 399--404. Google Scholar

[20] Alahi A, Goel K, Ramanathan V, et al. Social lstm: Human trajectory prediction in crowded spaces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016. 961--971. Google Scholar

[21] Deo N, Trivedi M M. Multi-modal trajectory prediction of surrounding vehicles with maneuver based lstms. In: Proceedings of IEEE Intelligent Vehicles Symposium (IV), 2018. 1179--1184. Google Scholar

[22] Xing Y, Lv C. Dynamic State Estimation for the Advanced Brake System of Electric Vehicles by using Deep Recurrent Neural Networks. IEEE Trans Ind Electron, 2019, : 1-1 CrossRef Google Scholar

[23] Wang X, Jiang R, Li L. Capturing Car-Following Behaviors by Deep Learning. IEEE Trans Intell Transp Syst, 2018, 19: 910-920 CrossRef Google Scholar

[24] Shen Y, Tan S, Sordoni A, et al. Ordered neurons: integrating tree structures into recurrent neural networks. 2018,. arXiv Google Scholar

[25] Dai S, Li L, Li Z. Modeling Vehicle Interactions via Modified LSTM Models for Trajectory Prediction. IEEE Access, 2019, 7: 38287-38296 CrossRef Google Scholar

  • Figure 1

    (Color online) The illustration of the hierarchical updating logic of ON-LSTM. (a) The hierarchical update when $\boldsymbol{\omega}_t$ does not approximate to zero vector; (b) the hierarchical update when $\boldsymbol{\omega}_t$ approximates to zero vector; (c) the hierarchical updating process and the corresponding constituency tree of a specific sequence.

  • Figure 2

    (Color online) The variations of $f_{\rm~mas}$ and $i_{\rm~mas}$ before and after a lane change operation is performed. Note that the values in the figure have been normalized.

  • Figure 3

    (Color online) The statistics of the captured memory effects in the testing set. (a) The proportion of significant memory effects of different behavior change relationships. The coordinate values in the figure respectively indicate different driving behaviors, specifically, MLL: $-2$; LLC: $-1$; GS: 0; RLC: 1; MRL: 2. (b) The distribution of the time deviation between the labeled behavior change point and the estimated start time of the memory effect.

  • Figure 4

    (Color online) The statistics of the estimated ETR of the captured driving fluctuations (grouped by the $F_{\rm~mas}$ at the beginning of the memory effect). (a) $F_{\rm~mas}\in~(0,0.67]$, 1760 cases in total; (b) $F_{\rm~mas}\in(0.67,0.73]$, 3639 cases in total; (c) $F_{\rm~mas}\in(0.73,0.78]$, 1240 cases in total; (d) $F_{\rm~mas}\in(0.78,0.83]$, 477 cases in total; (e) $F_{\rm~mas}\in(0.83,1]$, 359 cases in total; (f) the boxplot of the descent rate of $f_{\rm~mas}$ during the memory effect.

  • Figure 5

    (Color online) The statistics of the estimated ETR of the captured motion change operations.

  • Table 1  

    Table 1RMS values of prediction error

    Prediction horizon (s) RMS value (m)
    C-LSTM ON-LSTM
    1 0.54 0.48
    2 1.17 1.04
    3 1.90 1.71
    4 2.73 2.50
    5 3.67 3.43
    6 4.70 4.40

Copyright 2020  CHINA SCIENCE PUBLISHING & MEDIA LTD.  中国科技出版传媒股份有限公司  版权所有

京ICP备14028887号-23       京公网安备11010102003388号