Cross-Modal Attention for Accurate Pedestrian Trajectory Prediction

Mayssa ZAIER (IMT NORD EUROPE),* Hazem Wannous (University of Lille), Hassen Drira (University of Strasbourg), Jacques boonaert (imt lille douai)
The 34th British Machine Vision Conference


Accurately predicting human behavior is essential for a variety of applications, including self-driving cars, surveillance systems, and social robots. However, predicting human movement is challenging due to the complexity of physical environments and social interactions. Most studies focus on static environmental information, while ignoring the dynamic visual information available in the scene. To address this issue, we propose a novel approach called Cross-Modal Attention Trajectory Prediction (CMATP) able to predict human paths based on observed trajectory and dynamic scene context. Our approach uses a bimodal transformer network to capture complex spatio-temporal interactions and incorporates both pedestrian trajectory data and contextual information. Our approach achieves state-of-the-art performance on three real-world pedestrian prediction datasets, making it a promising solution for improving the safety and reliability of pedestrian detection and tracking systems.



author    = {Mayssa ZAIER and Hazem Wannous and Hassen Drira and Jacques boonaert},
title     = {Cross-Modal Attention for Accurate Pedestrian Trajectory Prediction},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year      = {2023},
url       = {}

Copyright © 2023 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection