ai nudes

apps-like-grok-are-explicitly-banned-under-google’s-rules—why-is-it-still-in-the-play-store?

Apps like Grok are explicitly banned under Google’s rules—why is it still in the Play Store?

AI, ai nudes, Google, google play, grok, nudify apps, Tech, xAI / /

Elon Musk’s xAI recently weakened content guard rails for image generation in the Grok AI bot. This led to a new spate of non-consensual sexual imagery on X, much of it aimed at silencing women on the platform. This, along with the creation of sexualized images of children in the more compliant Grok, has led regulators to begin investigating xAI. In the meantime, Google has rules in place for exactly this eventuality—it’s just not enforcing them.

It really could not be more clear from Google’s publicly available policies that Grok should have been banned yesterday. And yet, it remains in the Play Store. Not only that—it enjoys a T for Teen rating, one notch below the M-rated X app. Apple also still offers the Grok app on its platform, but its rules actually leave more wiggle room.

App content restrictions at Apple and Google have evolved in very different ways. From the start, Apple has been prone to removing apps on a whim, so developers have come to expect that Apple’s guidelines may not mention every possible eventuality. As Google has shifted from a laissez-faire attitude to more hard-nosed control of the Play Store, it has progressively piled on clarifications in the content policy. As a result, Google’s rules are spelled out in no uncertain terms, and Grok runs afoul of them.

Google has a dedicated support page that explains how to interpret its “Inappropriate Content” policy for the Play Store. Like Apple, the rules begin with a ban on apps that contain or promote sexual content including, but not limited to, pornography. That’s where Apple stops, but Google goes on to list more types of content and experiences that it considers against the rules.

“We don’t allow apps that contain or promote content associated with sexually predatory behavior, or distribute non-consensual sexual content,” the Play Store policy reads (emphasis ours). So the policy is taking aim at apps like Grok, but this line on its own could be read as focused on apps featuring “real” sexual content. However, Google is very thorough and has helpfully explained that this rule covers AI.

Play Store policy

Recent additions to Google’s Play Store policy explicitly ban apps like Grok.

Credit: Google

Recent additions to Google’s Play Store policy explicitly ban apps like Grok. Credit: Google

The detailed policy includes examples of content that violate this rule, which include much of what you’d expect—nothing lewd or profane, no escort services, and no illegal sexual themes. After a spate of rudimentary “nudify” apps in 2020 and 2021, Google added language to this page clarifying that “apps that claim to undress people” are not allowed in Google Play. In 2023, as the AI boom got underway, Google added another line to note that it also would remove apps that contained “non-consensual sexual content created via deepfake or similar technology.”

Apps like Grok are explicitly banned under Google’s rules—why is it still in the Play Store? Read More »

largest-deepfake-porn-site-shuts-down-forever

Largest deepfake porn site shuts down forever

ai nudes, Artificial Intelligence, deepfake porn, generative ai, ncii, non-consensual intimate imagery, Policy, take it down act / /

The shuttering of Mr. Deepfakes won’t solve the problem of deepfakes, though. In 2022, the number of deepfakes skyrocketed as AI technology made the synthetic NCII appear more realistic than ever, prompting an FBI warning in 2023 to alert the public that the fake content was being increasingly used in sextortion schemes. But the immediate solutions society used to stop the spread had little impact. For example, in response to pressure to make the fake NCII harder to find, Google started downranking explicit deepfakes in search results but refused to demote platforms like Mr. Deepfakes unless Google received an unspecified “high volume of removals for fake explicit imagery.”

According to researchers, Mr. Deepfakes—a real person who remains anonymous but reportedly is a 36-year-old hospital worker in Toronto—created the engine driving this spike. His DeepFaceLab quickly became “the leading deepfake software, estimated to be the software behind 95 percent of all deepfake videos and has been replicated over 8,000 times on GitHub,” researchers found. For casual users, his platform hosted videos that could be purchased, usually priced above $50 if it was deemed realistic, while more motivated users relied on forums to make requests or enhance their own deepfake skills to become creators.

Mr. Deepfakes’ illegal trade began on Reddit but migrated to its own platform after a ban in 2018. There, thousands of deepfake creators shared technical knowledge, with the Mr. Deepfakes site forums eventually becoming “the only viable source of technical support for creating sexual deepfakes,” researchers noted last year.

Having migrated once before, it seems unlikely that this community won’t find a new platform to continue generating the illicit content, possibly rearing up under a new name since Mr. Deepfakes seemingly wants out of the spotlight. Back in 2023, researchers estimated that the platform had more than 250,000 members, many of whom may quickly seek a replacement or even try to build a replacement.

Further increasing the likelihood that Mr. Deepfakes’ reign of terror isn’t over, the DeepFaceLab GitHub repository—which was archived in November and can no longer be edited—remains available for anyone to copy and use.

404 Media reported that many Mr. Deepfakes members have already connected on Telegram, where synthetic NCII is also reportedly frequently traded. Hany Farid, a professor at UC Berkeley who is a leading expert on digitally manipulated images, told 404 Media that “while this takedown is a good start, there are many more just like this one, so let’s not stop here.”

Largest deepfake porn site shuts down forever Read More »