Deepfake pornography

In June 2019, a downloadable Windows and Linux application called DeepNude was released which used GAN to remove clothing from images of women. The app had both a paid and unpaid version, the paid version costing $50.[3] On June 27, the creators removed the application and refunded consumers, although various copies of the app, both free and for charge, continue to exist.[4] On GitHub, the open-source version of this program called "open-deepnude" was deleted.[5] The open-source version had the advantage of allowing to be trained on a larger dataset of nude images to increase the resulting nude image's accuracy level.[6]

In July 2019 a deepfake bot service was launched on messaging app Telegram that uses AI technology to create nude images of women. The service is free and has a user-friendly interface, enabling users to submit photos and receive manipulated nude images within minutes. The service is connected to seven Telegram channels, including the main channel that hosts the bot, technical support, and image sharing channels. While the total number of users is unknown, the main channel has over 45,000 members. As of July 2020, it is estimated that approximately 24,000 manipulated images have been shared across the image sharing channels.[7]

In 2018, Rana Ayyub, Indian investigative journalist was the target of online hate campaign stemming from her condemnation of the Indian government, specifically her speaking out against the rape of an eight-year-old Kashmiri girl. Ayyub was bombarded with rape and death threats, and had doctored porngraphic video of her circulated online.[10] In a Huffington Post article, Ayyub discusses the long-lasting psychological and social effects this experience has had on her. She explains that she continues to struggle with her mental health and how the images and videos continue to resurface whenever she takes a high-profile case.[11]

In 2023, Twitch streamer Atrioc stirred controversy when he accidentally revealed deepfake pornographic material featuring female Twitch streamers while on live. The influencer has since admitted to paying for AI generated porn, and apologized to the women and his fans.[12][13]

Deepfake technology has made the creation of child sexual abuse material (CSAM), also often referenced to as child pornography, faster, safer and easier than it has ever been. Deepfakes can be used to produce new CSAM from already existing material or creating CSAM from children who have not been subjected to sexual abuse. Deepfake CSAM can, however, have real and direct implications on children including defamation, grooming, extortion, and bullying.[14]

Deepfake detection has become an increasingly important area of research in recent years as the spread of fake videos and images has become more prevalent. One promising approach to detecting deepfakes is through the use of Convolutional Neural Networks (CNNs), which have shown high accuracy in distinguishing between real and fake images. One CNN-based algorithm that has been developed specifically for deepfake detection is DeepRhythm, which has demonstrated an impressive accuracy score of 0.98. This algorithm utilizes a pre-trained CNN to extract features from facial regions of interest and then applies a novel attention mechanism to identify discrepancies between the original and manipulated images. While the development of more sophisticated deepfake technology presents ongoing challenges to detection efforts, the high accuracy of algorithms like DeepRhythm offers a promising tool for identifying and mitigating the spread of harmful deepfakes.[15]

Aside from detection models, there are also video authenticating tools available to the public. In 2019, Deepware launched the first publicly available detection tool which allowed users to easily scan and detect deepfake videos. Similarly, in 2020 Microsoft released a free and user-friendly video authenticator. Users upload a suspected video or input a link, and receive a confidence score to assess the level of manipulation in a deepfake.

In 2023, there is a lack of legislation that specifically addresses deepfake pornography. Instead, the harm caused by its creation and distribution is being addressed by the courts through existing criminal and civil laws.

The most common legal recourse for victims of deepfake pornography is pursuing a claim of “revenge porn” because the images are non-consensual and intimate in nature. The legal consequences for revenge porn vary from country to country.[16] For instance, in Canada, the penalty for publishing non-consensual intimate images is up to 5 years in prison.[17] Whereas in Malta, it is a fine of up to €5,000.[18]

The “Deepfake Accountability Act” was introduced to the United States Congress in 2019. It aimed to make the production and distribution of digitally altered visual media that was not disclosed to be such, a criminal offense. The title specifies that making any sexual, non-consensual altered media with the intent of humiliating or otherwise harming the participants, may be fined, imprisoned for up to 5 years or both. However the act has yet to be passed into law.[19]

Several major online platforms have taken steps to ban deepfake pornography. As of 2018, gfycat, reddit, Twitter, Discord, and Pornhub have all prohibited the uploading and sharing of deepfake pornographic content on their platforms.[20][21] In September of that same year, Google also added "involuntary synthetic pornographic imagery" to its ban list, allowing individuals to request the removal of such content from search results.[22] It's worth noting, however, that while Pornhub has taken a stance against non-consensual content, searching for "deepfake '' on their website still yields results and they continue to run ads for deepfake websites and content.[23]