TD-GEM: Text-Driven Garment Editing Mapper
Reza Dadfar (KTH), Sanaz Sabzevari (KTH University),* Marten Bjorkman (KTH), Danica Kragic (KTH Royal Institute of Technology)
The 34th British Machine Vision Conference
Abstract
Language-based fashion image editing allows users to try out variations of desired garments through provided text prompts. Inspired by research on manipulating latent representations in StyleCLIP and HairCLIP, we focus on these latent spaces for editing fashion items of full-body human datasets. Currently, there is a gap in handling fashion image editing due to the complexity of garment shapes and textures and the diversity of human poses. In this paper, we propose an editing optimizer scheme method called Text-Driven Garment Editing Mapper (TD-GEM), aiming to edit fashion items in a disentangled way. To this end, we initially obtain a latent representation of an image through generative adversarial network inversions such as Encoder for Editing (e4e) or Pivotal Tuning Inversion (PTI) for more accurate results. An optimization-based Contrastive Language-Image Pre-training (CLIP) is then utilized to guide the latent representation of a fashion image in the direction of a target attribute expressed in terms of a text prompt. Our TD-GEM manipulates the image accurately according to the target attribute, while other parts of the image are kept untouched. In the experiments, we evaluate TD-GEM on two different attributes (i.e., "color" and "sleeve length"), which effectively generates realistic images compared to the recent manipulation schemes.Video
Citation
@inproceedings{Dadfar_2023_BMVC,
author = {Reza Dadfar and Sanaz Sabzevari and Marten Bjorkman and Danica Kragic},
title = {TD-GEM: Text-Driven Garment Editing Mapper},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0787.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by