Semantic Adversarial Attacks via Diffusion Models
Chenan Wang (Drexel University),* Jinhao Duan (Drexel University), Chaowei Xiao (ASU), Edward Kim (Drexel University), Matthew c Stamm (Drexel University), Kaidi Xu (Drexel University)
The 34th British Machine Vision Conference
Abstract
Traditional adversarial attacks concentrate on manipulating clean examples in the pixel space by adding adversarial perturbations. By contrast, semantic adversarial attacks focus on changing semantic attributes of clean examples, such as color, context, and features, which are more feasible in the real world. In this paper, we propose a framework to quickly generate a semantic adversarial attack by leveraging recent diffusion models since semantic information is included in the latent space of well-trained diffusion models. Then there are two variants of this framework: 1) the \textbf{S}emantic \textbf{T}ransformation (ST) approach fine-tunes the latent space of the generated image and/or the diffusion model itself; 2) the \textbf{L}atent \textbf{M}asking (LM) approach masks the latent space with another target image and local backpropagation-based interpretation methods. Additionally, the ST approach can be either white-box or black-box. Extensive experiments are conducted on the CelebA-HQ and AFHQ datasets, and our framework demonstrates great fidelity, generalizability, and transferability compared to other baselines. Our ST white-box approach achieves 100\% attack success rate and 36.61 FID score, our ST black-box approach achieves up to 100\% attack success rate and 96.88 FID score, and our LM approach achieves up to 99.8\% attack success rate and 64.38 FID score on CelebA-HQ dataset.Video
Citation
@inproceedings{Wang_2023_BMVC,
author = {Chenan Wang and Jinhao Duan and Chaowei Xiao and Edward Kim and Matthew c Stamm and Kaidi Xu},
title = {Semantic Adversarial Attacks via Diffusion Models},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0271.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by