ISSN 0253-2778

CN 34-1054/N

open
Free to Publish. Free to Read.
Open AccessOpen Access JUSTC Info. & Intelligence 16 September 2022

The surrounding vehicles behavior prediction for intelligent vehicles based on Att-BiLSTM

  • 1.

    Department of Automation, University of Science and Technology of China, Hefei 230022, China

  • 2.

    Institute of Advanced Technology, University of Science and Technology of China, Hefei 230088, China

Cite this:
https://doi.org/10.52396/JUSTC-2021-0155
More Information
  • Author Bio:

    Yunqing Gao received his B.S. degree in Automation from the University of Science and Technology of China (USTC) in 2020. He is currently pursuing the M.S. degree in Control Engineering at USTC. His research interests include automatic control and trajectory prediction

    Hongbo Gao received his Ph.D. degree from Beihang University in 2016. He is currently an associate professor at the Department of Automation in the University of Science and Technology of China. He is the author or coauthor of over 40 journal papers, and the co-holder of 10 patent applications. His research interests include unmanned system platform, robotics, machine learning, decision support system, and intelligent driving

  • Corresponding author: E-mail: ghb48@ustc.edu.cn
  • Received Date: 30 June 2021
  • Accepted Date: 25 May 2022
    • Available Online: 16 September 2022
    Abstract

  • Abstract

    A surrounding vehicles behavior prediction method was presented for intelligent vehicles. The surrounding vehicles’ behavior is hard to predict since the significant uncertainty of vehicle driving and environmental changes. This method adopts bidirectional long short-term memory (BiLSTM) model combined with an encoder to ensure the memory of long-time series training. By constructing an attention mechanism based on BiLSTM, we consider the importance of different information which could guarantee the encoder’s memory under long sequence. The designed attention-bidirectional LSTM (Att-BiLSTM) model is adopted to ensure the surrounding vehicles’ prediction accuracy and effectiveness.

    Graphical abstract

    Flowchart of the presented Att-BiLSTM algorithm.

    Abstract

    A surrounding vehicles behavior prediction method was presented for intelligent vehicles. The surrounding vehicles’ behavior is hard to predict since the significant uncertainty of vehicle driving and environmental changes. This method adopts bidirectional long short-term memory (BiLSTM) model combined with an encoder to ensure the memory of long-time series training. By constructing an attention mechanism based on BiLSTM, we consider the importance of different information which could guarantee the encoder’s memory under long sequence. The designed attention-bidirectional LSTM (Att-BiLSTM) model is adopted to ensure the surrounding vehicles’ prediction accuracy and effectiveness.

    • Public Summary

    • BiLSTM model combined with encoder is adopted to ensure the memory of long-time series training.
    • The importance of different states problem can be solved by considering the attention mechanism.
    • The Att-BiLSTM model is designed to ensure the surrounding vehicles prediction effectiveness and accuracy.

    FullText(HTML)
  • loading
    • References(24)
  • [1]
    Li D, Gao H. A hardware platform framework for an intelligent vehicle based on a driving brain. Engineering, 2018, 4 (4): 464–470. doi: 10.1016/j.eng.2018.07.015
    [2]
    Lefèvre S, Vasquez D, Laugier C. A survey on motion prediction and risk assessment for intelligent vehicles. Robomech Journal, 2014, 1 (1): 1–14. doi: 10.1186/s40648-014-0001-z
    [3]
    Brännström M, Coelingh E, Sjöberg J. Model-based threat assessment for avoiding arbitrary vehicle collisions. IEEE Transactions on Intelligent Transportation Systems, 2010, 11 (3): 658–669. doi: 10.1109/TITS.2010.2048314
    [4]
    Ortiz M G, Fritsch J, Kummert F, et al. Behavior prediction at multiple time-scales in inner-city scenarios. In: 2011 IEEE Intelligent Vehicles Symposium (IV). Baden-Baden, Germany: IEEE, 2011: 1068−1073.
    [5]
    Xie G, Gao H, Qian L, et al. Vehicle trajectory prediction by integrating physics-and maneuver-based approaches using interactive multiple models. IEEE Transactions on Industrial Electronics, 2018, 65 (7): 5999–6008. doi: 10.1109/TIE.2017.2782236
    [6]
    Zheng X, Zhang D, Gao H, et al. A novel framework for road traffic risk assessment with HMM-based prediction model. Sensors, 2018, 18 (12): 4313. doi: 10.3390/s18124313
    [7]
    Gao H, Cheng B, Wnag K, et al. Object classification using CNN-based fusion of vision and LIDAR in autonomous vehicle environment. IEEE Transactions on Industrial Informatics, 2018, 14 (9): 4224–4231. doi: 10.1109/TII.2018.2822828
    [8]
    Gao H, Su H, Cai Y, et al. Trajectory prediction of cyclist based on dynamic Bayesian network and long short-term memory model at unsignalized intersections. Science China Information Sciences, 2021, 64: 172207. doi: 10.1007/s11432-020-3071-8
    [9]
    Pu J, Zhou W, Hu H, et al. Boosting continuous sign language recognition via cross modality augmentation. In: Proceedings of the 28th ACM International Conference on Multimedia. Seattle, USA: Association for Computing Machinery, 2020: 1497–1505.
    [10]
    Ondruska P, Posner I. Deep tracking: Seeing beyond seeing using recurrent neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, 2016, 30 (1): 3361–3367. doi: 10.1609/aaai.v30i1.10413
    [11]
    Khosroshahi A, Ohn-Bar E, Trivedi M M. Surround vehicles trajectory analysis with recurrent neural networks. In: 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC). Piscataway, USA: IEEE, 2016: 2267–2272.
    [12]
    Jeong Y, Kim S, Yi K. Surround vehicle motion prediction using LSTM-RNN for motion planning of autonomous vehicles at multi-lane turn intersections. IEEE Open Journal of Intelligent Transportation Systems, 2020, 1: 2–14. doi: 10.1109/OJITS.2020.2965969
    [13]
    Park S H, Kim B D, Kang C M, et al. Sequence-to-sequence prediction of vehicle trajectory via LSTM encoder-decoder architecture. In: 2018 IEEE Intelligent Vehicles Symposium (IV). Jiangsu, China: IEEE, 2018: 1672–1678.
    [14]
    Benterki A, Judalet V, Maaoui C, et al. Multi-model and learning-based framework for real-time trajectory prediction. In: 2020 28th Mediterranean Conference on Control and Automation. Akko, Israel: IEEE, 2020: 776–781.
    [15]
    Gao H, Guo F, Zhu J, et al. Human motion segmentation based on structure constraint matrix factorization. Science China Information Sciences, 2022, 65: 119103. doi: 10.1007/s11432-020-2967-3
    [16]
    Dai S, Li L, Li Z. Modeling vehicle interactions via modified LSTM models for trajectory prediction. IEEE Access, 2019, 7: 38287–38296. doi: 10.1109/ACCESS.2019.2907000
    [17]
    Xin L, Wang P, Chan C Y, et al. Intention-aware long horizon trajectory prediction of surrounding vehicles using dual LSTM networks. In: 2018 21st International Conference on Intelligent Transportation Systems (ITSC). Hawaii, USA: IEEE, 2018: 1441–1446.
    [18]
    Guan D, Zhao H, Zhao L, et al. Intelligent prediction of mobile vehicle trajectory based on space-time information. In: 2019 IEEE 89th Vehicular Technology Conference (VTC2019-Spring). Kuala Lumpur, Malaysia: IEEE, 2019: 1–5.
    [19]
    Gao H, Zhu J, Zhang T, et al. Situational assessment for intelligent vehicles based on stochastic model and Gaussian distributions in typical traffic scenarios. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2022, 52 (3): 1426–1436. doi: 10.1109/TSMC.2020.3019512
    [20]
    Jeong Y, Yi K. Bidirectional long shot-term memory-based interactive motion prediction of cut-in vehicles in urban environments. IEEE Access, 2020, 8: 106183–106197. doi: 10.1109/ACCESS.2020.2994929
    [21]
    Lenz D, Diehl F, Le M T, et al. Deep neural networks for Markovian interactive scene prediction in highway scenarios. In: 2017 IEEE Intelligent Vehicles Symposium (IV). Los Angeles, USA: IEEE, 2017: 685–692.
    [22]
    Deo N, Trivedi M M. Multi-modal trajectory prediction of surrounding vehicles with maneuver based LSTMs. In: 2018 IEEE Intelligent Vehicles Symposium (IV). Changshu, China: IEEE, 2018: 1179–1184.
    [23]
    Shi X, Chen Z, Wang H, et al. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In: Proceedings of the 28th International Conference on Neural Information Processing Systems-Volume 1. Montreal, Canada: MIT Press, 2015: 802–810.
    [24]
    Shi X, Gao Z, Lausen L, et al. Deep learning for precipitation nowcasting: A benchmark and a new model. [2022-06-15]. https://arxiv.org/abs/1706.03458.
    • Supplements(0)
    • Related Articles
    • Track Citations
  • 加载中

Catalog

    Get Citation
    PDF
    XML

    Figure  1.  Att-BiLSTM overview.

    Figure  2.  Recurrent neural network.

    Figure  3.  The cyclic unit structure of LSTM network.

    Figure  4.  Attention mechanism-based BiLSTM.

    Figure  5.  Loss change curve of training and test sets.

    Figure  6.  The prediction results with ID 30000 car.

    Figure  7.  The prediction results with ID 20000 car.

    Figure  8.  Comparison of different forecasting time series.

    Figure  9.  The mean square error between the forecast result and the real result of each model.

    Figure  10.  The increase of prediction time step on the interference degree of each model.

    [1]
    Li D, Gao H. A hardware platform framework for an intelligent vehicle based on a driving brain. Engineering, 2018, 4 (4): 464–470. doi: 10.1016/j.eng.2018.07.015
    [2]
    Lefèvre S, Vasquez D, Laugier C. A survey on motion prediction and risk assessment for intelligent vehicles. Robomech Journal, 2014, 1 (1): 1–14. doi: 10.1186/s40648-014-0001-z
    [3]
    Brännström M, Coelingh E, Sjöberg J. Model-based threat assessment for avoiding arbitrary vehicle collisions. IEEE Transactions on Intelligent Transportation Systems, 2010, 11 (3): 658–669. doi: 10.1109/TITS.2010.2048314
    [4]
    Ortiz M G, Fritsch J, Kummert F, et al. Behavior prediction at multiple time-scales in inner-city scenarios. In: 2011 IEEE Intelligent Vehicles Symposium (IV). Baden-Baden, Germany: IEEE, 2011: 1068−1073.
    [5]
    Xie G, Gao H, Qian L, et al. Vehicle trajectory prediction by integrating physics-and maneuver-based approaches using interactive multiple models. IEEE Transactions on Industrial Electronics, 2018, 65 (7): 5999–6008. doi: 10.1109/TIE.2017.2782236
    [6]
    Zheng X, Zhang D, Gao H, et al. A novel framework for road traffic risk assessment with HMM-based prediction model. Sensors, 2018, 18 (12): 4313. doi: 10.3390/s18124313
    [7]
    Gao H, Cheng B, Wnag K, et al. Object classification using CNN-based fusion of vision and LIDAR in autonomous vehicle environment. IEEE Transactions on Industrial Informatics, 2018, 14 (9): 4224–4231. doi: 10.1109/TII.2018.2822828
    [8]
    Gao H, Su H, Cai Y, et al. Trajectory prediction of cyclist based on dynamic Bayesian network and long short-term memory model at unsignalized intersections. Science China Information Sciences, 2021, 64: 172207. doi: 10.1007/s11432-020-3071-8
    [9]
    Pu J, Zhou W, Hu H, et al. Boosting continuous sign language recognition via cross modality augmentation. In: Proceedings of the 28th ACM International Conference on Multimedia. Seattle, USA: Association for Computing Machinery, 2020: 1497–1505.
    [10]
    Ondruska P, Posner I. Deep tracking: Seeing beyond seeing using recurrent neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, 2016, 30 (1): 3361–3367. doi: 10.1609/aaai.v30i1.10413
    [11]
    Khosroshahi A, Ohn-Bar E, Trivedi M M. Surround vehicles trajectory analysis with recurrent neural networks. In: 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC). Piscataway, USA: IEEE, 2016: 2267–2272.
    [12]
    Jeong Y, Kim S, Yi K. Surround vehicle motion prediction using LSTM-RNN for motion planning of autonomous vehicles at multi-lane turn intersections. IEEE Open Journal of Intelligent Transportation Systems, 2020, 1: 2–14. doi: 10.1109/OJITS.2020.2965969
    [13]
    Park S H, Kim B D, Kang C M, et al. Sequence-to-sequence prediction of vehicle trajectory via LSTM encoder-decoder architecture. In: 2018 IEEE Intelligent Vehicles Symposium (IV). Jiangsu, China: IEEE, 2018: 1672–1678.
    [14]
    Benterki A, Judalet V, Maaoui C, et al. Multi-model and learning-based framework for real-time trajectory prediction. In: 2020 28th Mediterranean Conference on Control and Automation. Akko, Israel: IEEE, 2020: 776–781.
    [15]
    Gao H, Guo F, Zhu J, et al. Human motion segmentation based on structure constraint matrix factorization. Science China Information Sciences, 2022, 65: 119103. doi: 10.1007/s11432-020-2967-3
    [16]
    Dai S, Li L, Li Z. Modeling vehicle interactions via modified LSTM models for trajectory prediction. IEEE Access, 2019, 7: 38287–38296. doi: 10.1109/ACCESS.2019.2907000
    [17]
    Xin L, Wang P, Chan C Y, et al. Intention-aware long horizon trajectory prediction of surrounding vehicles using dual LSTM networks. In: 2018 21st International Conference on Intelligent Transportation Systems (ITSC). Hawaii, USA: IEEE, 2018: 1441–1446.
    [18]
    Guan D, Zhao H, Zhao L, et al. Intelligent prediction of mobile vehicle trajectory based on space-time information. In: 2019 IEEE 89th Vehicular Technology Conference (VTC2019-Spring). Kuala Lumpur, Malaysia: IEEE, 2019: 1–5.
    [19]
    Gao H, Zhu J, Zhang T, et al. Situational assessment for intelligent vehicles based on stochastic model and Gaussian distributions in typical traffic scenarios. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2022, 52 (3): 1426–1436. doi: 10.1109/TSMC.2020.3019512
    [20]
    Jeong Y, Yi K. Bidirectional long shot-term memory-based interactive motion prediction of cut-in vehicles in urban environments. IEEE Access, 2020, 8: 106183–106197. doi: 10.1109/ACCESS.2020.2994929
    [21]
    Lenz D, Diehl F, Le M T, et al. Deep neural networks for Markovian interactive scene prediction in highway scenarios. In: 2017 IEEE Intelligent Vehicles Symposium (IV). Los Angeles, USA: IEEE, 2017: 685–692.
    [22]
    Deo N, Trivedi M M. Multi-modal trajectory prediction of surrounding vehicles with maneuver based LSTMs. In: 2018 IEEE Intelligent Vehicles Symposium (IV). Changshu, China: IEEE, 2018: 1179–1184.
    [23]
    Shi X, Chen Z, Wang H, et al. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In: Proceedings of the 28th International Conference on Neural Information Processing Systems-Volume 1. Montreal, Canada: MIT Press, 2015: 802–810.
    [24]
    Shi X, Gao Z, Lausen L, et al. Deep learning for precipitation nowcasting: A benchmark and a new model. [2022-06-15]. https://arxiv.org/abs/1706.03458.
    Recommended articles
    TrendMD
    Volume 52 Issue 9 page: 6
    Cover

    Article Metrics

    Article views (598) PDF downloads(2160)
    Proportional views

    Copyright © Editorial Office of JUSTC, All Rights Reserved. 皖ICP备05002528号

    Supported by: Beijing Renhe Information Technology Co. Ltd    

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return