EyeGuide - From Gaze Data to Instance Segmentation
Jacqueline Kockwelp (University of Münster), Joerg Gromoll (CeRA), Joachim Wistuba (Centre of Reproductive Medicine and Andrology), Benjamin Risse (University of Münster)*
The 34th British Machine Vision Conference
Abstract
Obtaining precise instance-level segmentations is a challenging task in machine learning. Especially for objects with complex and non-convex geometries or with partial occlusions scribble, bounding boxes or user clicks are often provided to guide the segmentation. In this paper, we explore the usage of a remote eye tracking system to generate gaze data as an additional input for object segmentation models (called EyeGuide). The gaze data is recorded during routine image inspections (i.e. without giving a particular task) and is used as an additional input to train neural networks. Our results indicate that the acquisition of gaze data is faster and more convenient than providing explicit user input, less annotations are necessary to generate equal or better segmentation results and also overall better generalisation capabilities on unseen classes compared to state-of-the-art techniques. In summary, EyeGuide is a simple yet powerful guidance strategy that can directly be integrated in image inspection routines and neural network architectures.Video
Citation
@inproceedings{Kockwelp_2023_BMVC,
author = {Jacqueline Kockwelp and Joerg Gromoll and Joachim Wistuba and Benjamin Risse},
title = {EyeGuide - From Gaze Data to Instance Segmentation},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0719.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by