Twitter to target fake videos and photos

The move, while short of an outright ban, was announced one day after YouTube also said it planned to remove misleading election-related content on its site. Twitter’s new policy highlights a balancing act — between allowing parody and removing disinformation — that social media companies face as they try to more aggressively police the content posted to their platforms.

To determine whether a tweet should be removed or labelled, Twitter said in a blog post, it will apply several tests: Is the media included with a tweet significantly altered or fabricated to mislead? Is it shared in a deceptive manner? In those cases, the tweet will probably get a label.

But if a tweet is “likely to impact public safety or cause serious harm,” it will be taken down. Twitter said it might also show a warning to people before they engaged with a tweet carrying manipulated content, or limit that tweet’s reach.

“Our approach does not focus on the specific technologies used to manipulate or fabricate media,” said Yoel Roth, Twitter’s head of site integrity. “Whether you’re using advanced machine learning tools or just slowing down a video using a 99-cent app on your phone, our focus under this policy is to look at the outcome, not how it was achieved.”

The company developed its rules after surveying more than 6,500 users, civil groups and academics, said Del Harvey, Twitter’s vice president for trust and safety.

They found that about 70% of surveyed Twitter users believed it was unacceptable for the company to take no action against manipulated content. More than 90% said such content should be removed or placed behind a warning label saying the video or image had been altered. “Things that distort or distract from what’s happening threaten the integrity of information on Twitter,” Harvey said. Last year, an altered video of Speaker Nancy Pelosi that made it appear that she was slurring her words spread across the internet. Another heavily edited clip of former Vice President Joe Biden falsely made it seem as though he had made racist remarks. Roth said the manipulated videos of Pelosi and Biden would get a label under Twitter’s new policy. Depending on what a tweet sharing the video said and if it caused harm, Roth said, Twitter could take the tweet down.

Last year, Twitter said it would add warning labels to hide messages from major political figures who broke the company’s rules for harassment or abuse. Normally, those tweets would be taken down, but the company argued that they would be newsworthy enough to remain on the platform. Twitter had not yet used the labels.

In January, Facebook banned “deepfake” videos from its platform. But the company said the videos of Pelosi and Biden would not be removed under the policy because they had been edited withvideo editing software, notartificial intelligence.