Supervised Contrastive Learning with Identity-Label Embeddings for Facial Action Unit Recognition
Tangzheng Lian (Nottingham Trent Univeristy), David A Adama (Nottingham Trent University), Pedro Machado (Nottingham Trent University), Doratha E Vinkemeier (NTU)*
The 34th British Machine Vision Conference
Abstract
Facial expression analysis is a crucial area of research for understanding human emotions. One important approach to this is the automatic detection of facial action units (AUs), which are small, visible changes in facial appearance. Despite extensive research, automatic AU detection remains a challenging computer vision problem. This paper addresses two central difficulties: the first is the inherent differences in facial behaviour and appearance across individuals, which leads current AU recognition models to overfit subjects in the training set and generalize poorly to unseen subjects; the second is representing the complex interactions among different AUs. In this paper, we propose a novel two-stage training framework, called CL-ILE, to address these long-standing challenges. In the first stage of CL-ILE, we introduce identity-label embeddings (ILEs) to train an ID feature encoder capable of generating person-specific feature embeddings for input face images. In the second stage, we present a data-driven method that implicitly models the relationships among AUs using contrastive loss in a supervised setting while eliminating the person-specific features generated by the first stage to enhance generalizability. By removing the ID feature encoder and ILEs from the first stage after training, CL-ILE becomes more lightweight and readily applicable to real-world applications than models using graph-based structures. We evaluate our approach on two widely-used AU recognition datasets, BP4D and DISFA, demonstrating that CL-ILE can achieve state-of-the-art performance on the F1 score.Video
Citation
@inproceedings{Lian_2023_BMVC,
author = {Tangzheng Lian and David A Adama and Pedro Machado and Doratha E Vinkemeier},
title = {Supervised Contrastive Learning with Identity-Label Embeddings for Facial Action Unit Recognition},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0335.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by