G2N2: Lightweight Event Stream Classification with GRU Graph Neural Networks
Thomas Mesquida (CEA LIST),* Manon Dampfhoffer (SPINTEC University Grenoble Alpes), Thomas Dalgaty (CEA List), Pascal Vivet (CEA-LIST), Amos Sironi (PROPHESEE), Christoph Posch (PROPHESEE)
The 34th British Machine Vision Conference
Abstract
Event camera pixels efficiently encode visual information through triggered events, offering advantages in temporal detail, dynamic range, and data reduction. However, the optimal machine learning method for leveraging these characteristics remains unclear. Existing approaches often convert events into 2D frames, losing crucial time-domain information. A promising alternative is event-graph neural networks, but they suffer from computational intensity and limited temporal dependencies. As a solution, we propose to combine the recently proposed lightweight event-graph neural network HUGNet with gated recurrent units to model temporal dependencies between the features extracted by HUGNet. We benchmark our model against other event-graph and convolutional neural network based approaches on the challenging DVS-Lip dataset (spoken word classification). We find that not only does our method outperform state of the art approaches for similar model sizes, but that, relative to the convolutional models, the number of calculation operations per second was reduced by 81%. Furthermore, we introduce a new event-data augmentation technique that boosts by up to 7.4% the performance of both event-graph and convolutional neural networks on this task.Video
Citation
@inproceedings{Mesquida_2023_BMVC,
author = {Thomas Mesquida and Manon Dampfhoffer and Thomas Dalgaty and Pascal Vivet and Amos Sironi and Christoph Posch},
title = {G2N2: Lightweight Event Stream Classification with GRU Graph Neural Networks},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0660.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by