Divide & Bind Your Attention for Improved Generative Semantic Nursing
Yumeng Li (Bosch Center for Artificial Intelligence),* Margret Keuper (University of Siegen, Max Planck Institute for Informatics), Dan Zhang (Bosch Center for Artificial Intelligence), Anna Khoreva (Bosch Center for Artificial Intelligence)
The 34th British Machine Vision Conference
Abstract
Emerging large-scale text-to-image generative models, e.g., Stable Diffusion (SD), have exhibited overwhelming results with high fidelity. Despite the magnificent progress, current state-of-the-art models still struggle to generate images fully adhering to the input prompt. Prior work, Attend & Excite, has introduced the concept of Generative Semantic Nursing (GSN), aiming to optimize cross-attention during inference time to better incorporate the semantics. It demonstrates promising results in generating simple prompts, e.g., “a cat and a dog”. However, its efficacy declines when dealing with more complex prompts, and it does not explicitly address the problem of improper attribute binding. To address the challenges posed by complex prompts or scenarios involving multiple entities and to achieve improved attribute binding, we propose Divide & Bind. We introduce two novel loss objectives for GSN: a novel attendance loss and a binding loss. Our approach stands out in its ability to faithfully synthesize desired objects with improved attribute alignment from complex prompts and exhibits superior performance across multiple evaluation benchmarks.Video
Citation
@inproceedings{Li_2023_BMVC,
author = {Yumeng Li and Margret Keuper and Dan Zhang and Anna Khoreva},
title = {Divide & Bind Your Attention for Improved Generative Semantic Nursing},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0366.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by