Integrating Transient and Long-term Physical States for Depression Intelligent Diagnosis
Ke Wu (Beihang University), Han Jiang (State Key Laboratory of Virtual Reality Technology and Systems，Beihang University),* Li Kuang (Beihang University), Yixuan Wang (Beihang University), Huaiqian Ye (Beihang University), Yuanbo He (State Key Laboratory of Virtual Reality Technology and Systems, Beihang University)
The 34th British Machine Vision Conference
Abstract
As social competition intensifies, the number of depression patients has rapidly increased. Many researchers have proposed diagnostic models for depression based on various physiological signals and behavioral information, such as Electroencephalogram (EEG) and facial expressions. However, it should be noted that these signals tend to reflect transient information, which may make them insufficient for accurately diagnosing depression characterized by persistent low mood over a prolonged period of time. Meanwhile, traditional Chinese medicine(TCM) believes that different parts of the tongue correspond to different Zang-fu and can indicate the long-term physical condition of the human body. Therefore, we use EEG and tongue images to reflect the subject's instantaneous state and long-term physical condition respectively, and establish a multimodal model MMTV to assist doctors in diagnosing depression. Specifically, MMTV innovatively introduces the dual-stream input mechanism and self-attention mechanism to EEGNet to better extract the spatio-temporal features of EEG. Meanwhile, to obtain higher quality tongue surface images, MMTV introduces a segmentation step before inputting tongue images into the ViT model. Meta-learning techniques are applied to gain better pretrained weights for ViT. Furthermore, we analyze the correlation between tongue images and EEG and subsequently fuse the output features of the brain-tongue branches in MMTV. The ability of MMTV to recognize patients with depression has been validated on multiple datasets, with the highest recognition accuracy reaching 98.18%. Our code is available at https://github.com/Clearlangw/Depression-diagnosis-based-on-EEG-and-Tongue.Video
Citation
@inproceedings{Wu_2023_BMVC,
author = {Ke Wu and Han Jiang and Li Kuang and Yixuan Wang and Huaiqian Ye and Yuanbo He},
title = {Integrating Transient and Long-term Physical States for Depression Intelligent Diagnosis},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0119.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by