Building AI to inform people's fashion choice

What the research is:

An AI system that proposes easy changes to a person’s outfit to make it more fashionable. Our Fashion++ system uses a deep image-generation neural network to recognize garments and offer suggestions on what to remove, add, or swap. It can also recommend ways to adjust a piece of clothing, such as tucking in a shirt or rolling up the sleeves. Whereas previous work in this area has explored ways to recommend an entirely new outfit or to identify garments that are similar to one another, Fashion++ instead aims to suggest subtle alterations to an existing outfit that will make it more stylish.

How it works:

Fashion++ focuses specifically on minimal edits, suggesting adjustments that are more realistic and practical than buying an entirely new outfit. The system uses a discriminative fashionability classifier that is trained on thousands of publicly available images of outfits that have been judged to be stylish. These serve as ground truth examples of fashionable outfits, and unfashionable examples are then bootstrapped by swapping garments on the fashionable examples with their least similar counterparts.

Once the classifier is trained, our system gradually updates the outfit in order to make it more fashionable. An image-generation neural network renders the newly adjusted look, using a variational auto-encoder to generate the silhouette and a conditional generative adversarial network (cGAN) to generate the color and pattern. The latent encodings learned by this generator are further used to identify which garments in its inventory will best achieve the style.

Experiments show that the system’s recommendations bring images closer to ground truth examples and that human evaluators find the Fashion++ suggestions not only fashionable but also easy to implement.

Why it matters:

This work demonstrates the potential for more useful AI assistive technologies that suggest small, practical changes that have a meaningful impact. Our method of bootstrapping unfashionable examples shows how AI systems can learn even without resource-intensive human annotation.

By focusing on easy-to-make changes, Fashion++ could lead to applications that help consumers easily tweak an existing outfit. Fashion++ is an example of how AI can be useful in a domain such as fashion, which some might think would be too creative or subjective for these systems. Rather than dictating or redefining what is fashionable, Fashion++ learns from examples in order to offer practical fashion advice. Research like this might one day enable new ways for people to create and share the styles they most like, or even help fashion designers create new looks. By sharing our work, we hope to help others in the research community use AI to address aesthetic, creative challenges.

Read the full paper:

Fashion++: Minimal edits for outfit improvement