[Unofficial translation provided by the Business & Human Rights Resource Centre]

“Where is AI safety?”: University campuses exposed to deepfake sexual crimes”, 25 August 2024

Need to strengthen punishment criteria and supplier responsibilities; establish communication channels for preventing harm

…Recently, it was confirmed that sexually explicit material, which involved superimposing female students' faces onto nude photos, was shared in a group Telegram chat room. This cases of deepfake crime, where fake videos and images are created using AI, has resurfaced. The Telegram chat room, which has been operating since 2020, is known to have 1,200 participants. Additionally, deepvoice technology, which generates fake audio of specific voices using AI, was also employed.

…Telegram, a major distribution channel, operates its servers overseas, making it difficult for investigative agencies to track.

…Attorney Yang…advised that it is necessary to strengthen the responsibilities of AI suppliers. “Generative AI suppliers should establish their own standards for producing potentially criminal videos and audio materials and enhance oversight of their members,” he said.

…Attorney Oh Jeong-ik shared a similar view. “It is essential to strengthen the responsibilities of platforms providing these services and to build systems for tracking deepfake video creators.”

On 27 August Reuters reported that it had asked Telegram for a statement regarding deepfake sex crimes on the platform, but Telegram did not immediately respond.