Facebook recently began testing a new camera feature that makes photos and live video look like famous works of art.At first glance, the effects look cool, though not hugely different from those you would get on a number of other image editing apps. You'd be forgiven for dismissing the feature as yet another attempt by the social network to compete with Snapchat.But, behind the scenes, the feature represents one of the company's most significant AI advancements and one that will could help power more artificial intelligence applications.
The feature, which Facebook is currently testing in Ireland but plans to roll out more broadly "soon," is officially called Style Transfer. Part of a new camera feature in the main Facebook app, it allows you to add stylized filters designed to look like famous works of art to photos and live video.
What sets Style Transfer apart from other photo and video apps, though, is how these effects are able to make their way into the camera in real-time. It uses a type of machine learning called a neural network, which is trained to recreate the styles of famous artists.
Neural networks are how a lot of AI advancements are made but training a system like this typically requires more power than what a smartphone can deliver. And training a neural network to work locally on a phone in real-time is even more difficult. Competing photo app Prisma also uses local neural networks to enable its photo and video editing but its app takes several seconds to process and apply each filter.
"This is a pretty big leap in terms of power of what you can do," Facebook CTO Mike Schroepfer said during a press event in San Francisco.
And while photo and video filters in themselves may not be that interesting, regardless of how they are created, Schroepfer notes that Style Transfer represents just the "tip of the iceberg" of what's possible. Eventually, the core technology behind the feature, a new AI framework called Cafe2 Go, will help enable more applications powered by AI and computer vision.
"We can create gesture-based controls, where the computer can see where you're pointing and activate different styles or commands," Schroepfer writes in a blog post of the tool's potential. "We can recognize facial expressions and perform related actions, like putting a 'yay' filter over your selfie when you smile."
That's right, filters that can automatically add themselves to your selfies. While at this point it's still a theoretical update rather than an actual feature, it offers an interesting look at how Facebook could use AI to transform sharing.