A Computer Model From Facebook AI Research Group Can Offer You Styling Tips

If fans of Netflix’s Queer Eye have learned anything from Tan France, the flamboyant fashion consultant of the series, it’s that a simple modification can take a look from fine to fabulous. Tricks like the French tuck or cuffing the sleeves of a T-shirt can create the illusion of a slimmer waist or a more robust bicep, all without changing the basic components of the look. It’s about working with what you’ve got, then making it better.

Imagine, then, having your own personal Tan France to adjust your outfit every day. This kind of “minimal editing” makes up new research from a group of computer scientists affiliated with Facebook AI Research, who have created a machine learning system called Fashion++ to make outfits more stylish with small changes. A suggestion might involve tucking in a shirt, adding a necklace, or cuffing a sleeve rather than changing into an entirely different outfit. The research will be presented later this month at the International Conference on Computer Vision.

At this point in the story of AI, researchers have a good grasp on classic problems like object recognition or labeling the components of an image. In the fashion space, this has led to programs that can separate the individual components of an outfit (shirt, pants, shoes) and match the items in a photo to ones that are available to buy online. Pinterest, a leader in computer vision research with fashion applications, offers a tool that can zero in on one item in a picture—a black tulle skirt, for example—and find similar items across its database of pins. Amazon has an analogous tool called StyleSnap, which uses machine learning to match an item in a photo to a similar garment for sale on Amazon.

Modeling creativity in fashion is a little more complex. “Think about a person trying to explain to another person their creative process, versus how to recognize a cat,” Kristen Grauman, a computer scientist at UT Austin who works with Facebook AI Research. “These are very different ways of thinking.”

For Grauman, who contributed to the new research, this kind of work extends the effort to model creative problems with artificial intelligence. “Some of the challenges are around how you model things that are so small and subtle,” she says. “How do you train a system and teach it these differences between ‘good’ and ‘slightly better’ outfits? How do you capture style in a computational way?”

While Fashion++ is pure research for now, you can easily picture it becoming a consumer-ready feature in one of Facebook’s connected gadgets, like the Portal. Amazon already sells the Echo Look, a camera-enabled gadget that uses AI to choose the better of two outfits. “You could imagine this future AI assistant that would have intelligence about what styles exist, what personal style is, what someone owns, and make intelligent suggestions,” says Grauman. If tech companies’ interest in fashion is any indication, that future won’t be far away.

Wear This, Not That

To build the data set for Fashion++, the researchers used thousands of publicly available images from the social fashion sharing site Chictopia, which features photos of real people wearing current trends. The definition of a “stylish” outfit is constantly evolving, so the group opted for a set of photos that reflect what’s currently in style. The researchers then manipulated some of these photos to create a “worse” version by swapping one part of the outfit with a garment from a different photo. These mismatches helped to train the model on how to improve the overall fashionability in an individual outfit.

The research also focuses on representing the various components of an outfit—cataloging not just individual items (tops versus bottoms versus shoes), but also textures and shapes. “For texture, things like materials or colors or things that have to do with the digital appearance,” says Grauman. Denim might create a more casual look; an all-black outfit might come across as more sophisticated. Different shapes, like a turtleneck versus a V-neck top, create different looks depending on how they are combined. “The model learns which is more influential, which needs to be edited to be closer to the fashionable space,” says Grauman.

About The Author