Facebook bans deepfakes to curb online misinformation
Facebook said it will remove deepfakes, which means videos created with artificial intelligence that look real but distort reality, in a bid to fight online manipulation.
The social media giant said it will step up efforts to remove videos edited or synthesized that can mislead someone into thinking a subject of the video said words they did not actually say, and those "merges, replaces or superimposes content on to a video, making it appear to be authentic".
Created by artificial intelligence or machine learning, deepfakes combine or replace content to create images that can be almost impossible to tell are not authentic.
"While these videos are still rare on the internet, they present a significant challenge for our industry and society as their use increases," Facebook's vice president of global policy management, Monika Bickert, said in a blog post.
But she said the new rules won't include parody or satire, or clips edited just to change the order of words.
Experts have raised questions about its implementation.
Gregory, whose group was among those that gave feedback to Facebook for the policy, said that while the new rules look strong on paper, there are questions around how effective the company will be at uncovering synthetic videos.
Facebook has built deepfake-detecting algorithms and can also look at an account's behavior to get an idea of whether its intention is to spread disinformation. That will give the company an edge over users or journalists in sniffing them out, Gregory said.
But those algorithms haven't been used widely for deepfakes in the wild. "So it is an open question how effective detection will be," he said. "This is an algorithmic kind of game of cat and mouse, where the forgeries will get better alongside the detection."