Abstract
Rapid advancements in generative models, including generative adversarial networks (GANs) and diffusion models, have made possible of automated efficient image editing through the use of text descriptions, semantic segmentation, and/or reference style images. Nevertheless, in the fashion industry, the task of image editing requires more flexible, and typically iterative, modifications to the image content, while existing methods struggle to achieve. This paper proposes a new model called Content Decoupled and Enhanced GAN (CoDE-GAN), which is formulated and trained for the task of image reconstruction, more specifically, image inpainting, with sketch-guidance. Through this proxy task, the trained model can be used for flexible image editing, generating new images with consistent colours and required textures based on sketch inputs. In this new model, a content decoupling block is introduced including specially designed dual encoders, which pre-process inputs and further transformed into separated structure and texture representations. Moreover, a content enhancing module is designed and applied to the decoder, improving the colour consistency and refining the texture of the generated images. The proposed CoDE-GAN can achieve coarse-to-fine results in one single stage. Extensive experiment on three datasets, covering human, garment-only and scene images, show that CoDE-GAN outperforms other state-of-the-art methods in terms of both generated image quality and editing flexibility.
