gettyimagesbank
An increasing number of Koreans, including minors, have fallen victim to "deepfake" sex crimes, where images and videos combining photos of unidentified individuals with explicit content are circulated in Telegram group chats, according to police and IT industry experts, Monday.
Law enforcement authorities have pledged to mobilize additional resources to apprehend the perpetrators, while experts are calling on the government to initiate a nationwide campaign to raise public awareness about the crime and advocate for stricter penalties for the offenders.
An online post uploaded Sunday on X, formerly Twitter, disclosed a list of hundreds of middle schools, high schools, and universities where students have become victims of deepfake sex crimes.
Whether these schools have actually been affected and the extent of any such impact remains unconfirmed.
However, the post received over 57,000 retweets and 26,000 likes as of 2 p.m., Monday, with the numbers continuing to rise as the account accumulates reports from online users about similar cases.
This X post uploaded on Sunday shows a list of schools in Korea whose students have allgedly fallen victim to deepfake sex crimes. The post has over 570,000 retweets and 260,000 likes as of Monday. Captured from X
In addition, student councils of several schools in Seoul and Gyeonggi Province, including Hongik Design High School, have issued warnings on their social media accounts, alerting students to the potential risks of deepfake sex crimes.
The emergency notification read, “Currently, deepfake images using photos of Hongik Design High School students and their personal information are being shared on Telegram.” It urged students to remove or avoid posting photos of themselves online to prevent becoming victims of deepfake crimes.
According to the Seoul Metropolitan Police Agency, Monday, 10 teenagers aged 14 or older were arrested for deepfake crimes in Seoul alone between January and July this year.
An overwhelming 75.8 percent of suspects involved in deepfake image and video crimes last year were teenagers, according to data from the National Police Agency. Additionally, approximately 20 percent were in their twenties, resulting in a total of 95.8 percent of suspects being aged between their teens and 20s.
The vast majority of these cases involve male students targeting female students, but deepfake crimes are increasingly emerging as a form of school violence or harassment that affects all genders.
“It is deeply concerning that not only students but also teachers have fallen victim to deepfake crimes, which are proliferating among teenagers who are well-versed in IT devices,” a police official said during a regular press briefing.
Deepfake sex crimes targeting female soldiers have also come to light.
An online chat room with over 900 participants was discovered, where deepfake videos and images of female soldiers, referred to by users as "munitions," were distributed and exchanged alongside degrading and abusive comments.
Furthermore, a Telegram chatroom in which paid and free bots immediately turn images into deepfake nude photos was reported by local media last week.
The chat room offers the first two deepfake images free of charge, then switches to a paid system that charges $0.49 per photo in cryptocurrency. As of Aug. 21, the chat room has been joined by nearly 227,000 online users.
“There are approximately 260,000 taxis in Korea. This means the likelihood of encountering a sex criminal in Korea is almost as high as finding a cab on the streets,” an online user wrote on X, in response.
According to data submitted by the National Police Agency on Wednesday to the office of Rep. Cho Eun-hee of the Democratic Party of Korea (DPK), the number of deepfake-related crimes increased from 156 in 2021 to 160 in 2022 and 180 in 2023.
While technology advances rapidly, laws and systems struggle to keep pace in preventing emerging criminal activities.
Although an AI Act was proposed during the 21st National Assembly, it failed to pass and was discarded before reaching the threshold for approval.
While the 22nd National Assembly is anticipated to consider several AI-related legislations, the revised Sexual Violence Punishment Act of 2020 mandates that perpetrators of deepfake sex crimes, whether producing or distributing such content, face imprisonment for up to five years or fines of up to 50 million won.
However, only a few have actually faced punishment.
According to the Supreme Court's online ruling system, as of Monday, 35 cases — over half of the 71 rulings in the first and second trials related to deepfake crimes — were suspended between 2020 and last year.
Of the remaining 36 cases, 31 resulted in prison sentences for committing other sexual crimes in addition to deepfake offenses, while only 5 cases were sentenced to prison solely for deepfake crimes. Most first-time offenders received probation.
"The government should educate the public about the seriousness and unethical nature of deepfakes. If ethical approaches are insufficient, the law should be used to address the issue," said Kim Myung-joo, a professor of information protection at Seoul Women's University.
"The Sexual Violence Punishment Act is not effectively raising awareness because the penalties, which are often lenient for first-time offenders, do not serve as a strong deterrent," the professor said, urging for harsher penalties for deepfake-related crimes.
"We will strengthen prevention education to ensure that students understand the serious consequences of criminal behavior and how a criminal record can significantly impact their social and professional lives," said Seoul Metropolitan Police Agency Commissioner Kim Bong-sik.
The Korea Communications Commission plans to respond strongly to deepfake videos, although identifying distributors remains a challenge as Telegram has servers overseas.
Artificial intelligence experts and industry executives abroad, including pioneering technologist Yoshua Bengio, signed an open letter in February calling for stricter regulations on the creation of deepfakes.
Titled “Disrupting the Deepfake Supply Chain,” the letter recommends regulating deepfakes, including full criminalization of deepfake pornography and mentions criminal penalties for any individual who creates or facilitates the spread of harmful deepfakes.