China's new challenge as natural disasters strike - fake AI videos

As China grapples with the aftermath of Typhoon Noul, which made landfall over the weekend, authorities anticipate an additional hurdle: dangerously deceptive videos crafted using Artificial Intelligence (AI).

Over recent months, powerful storms and floods have led to a surge of fabricated videos on social media. These include images of deceased individuals in floodwaters, inaccurate portrayals of severe flooding in unaffected regions, and content with overtly false details about emergency responses. In some areas, videos falsely reporting power outages incited residents to hoard emergency supplies.

Other videos exploit genuine incidents. For instance, flooding in Hengzhou city, Guangxi, in early July caused hundreds of snakes to escape a breeding farm. Subsequently, fake images emerged depicting crocodiles being released into a river.

Amidst already tense and perilous situations involving mass evacuations and intricate rescue operations, authorities are keen to prevent panic. They now fear that AI technology—an area where China is investing billions—is furnishing simple, accessible tools that can be used to mislead the public.

Huang Zhihua, Deputy Chief of online public security for the Zhejiang Police Department, informed Chinese state media, "Many of these content creators use disaster news to try to gain followers, knowing that more sensational or exaggerated content will garner more likes and more views. They are deliberately fabricating information to grab people's attention."

In recent weeks, arrests and penalties, ranging from detention pending criminal prosecution to fines, have been imposed for creating bogus reports and video content. However, experts caution that an unprecedented misinformation crisis could be on the horizon.

Professor Chen Bin, in an article published by the Communist Party's Central Party School newspaper, stated, "AI dramatically lowers the technical barriers to producing false information. Individuals no longer require sophisticated technical skills. By simply prompting AI tools, they can rapidly generate convincing text, images, audio and video… which is highly realistic but entirely fabricated content, manufactured on a large scale."

Professor Gao Fuping, from the East China University of Political Science and Law, told the BBC that motivations for publishing such false content vary from those who do it "to create something amusing without thinking about the consequences," to those seeking to "generate traffic for personal gain," or those aiming to deliberately "cause panic or social unrest." He added, "Some users think the online world is anonymous and unregulated or that they cannot be traced."

There have been calls for all AI-generated content to be clearly marked, but users can easily re-edit the material to remove these markings. Professor Xu Xiaoke from Beijing Normal University told the BBC that greater efforts are needed to verify geographic locations with timestamps and to check image sources. He warned, "That said, I would also stress that detection technology still lags behind generation technology. We're in a constant cat-and-mouse game. So we should not assume that technology alone can identify AI-generated content with the click of a button."

The Chinese government is embracing AI as it strives to become the leading tech superpower. In 2017, it declared AI would be "the main driving force" of the country's progress. Significant capital is flowing into AI businesses, fueling domestic competition. The state-controlled wire service Xinhua reports that China has over 6,000 "AI companies."

In this environment, Professor Xu believes China needs to establish "trusted information infrastructure," potentially including official government early warning systems and disaster interfaces accessible to news organizations.

However, there have been instances where official sources, including government-controlled media, have deliberately downplayed disasters, leading to public mistrust. Earlier in July, local journalists were asked to "soften" their language, describing a major breach in the Liulan Dam near Naning as an "opening" or "gap." Officials seemingly attempted to make the breach sound less alarming, but the consequences were deadly, with water bursting through the broken dam leading to at least 26 deaths downstream. This is not an isolated incident; in 2021, local officials were arrested, accused of concealing or underreporting 139 deaths in Henan province.

Such incidents have created a trust vacuum, making it easier for malicious actors to disseminate fake content. Nevertheless, police are urging people to trust accredited sources during emergencies. Gao Kai from the Ningbo Public Security Bureau stated, "When encountering disaster-related information from an unknown source, approach it with scepticism and rely on official announcements." He also reassured the public that creators of false information would be apprehended, asserting that "the internet is not a lawless zone. Public security authorities will continue to intensify efforts to crack down on illegal online activity." He added, "We've created a specialised task force to monitor social media closely, especially short video platforms. They will investigate and handle these cases quickly, cracking down in a very strict manner." Punishment could range from a fine to up to seven years in prison if the misinformation's impact is severe enough.

On July 23, China's Cyberspace Administration announced a nationwide crackdown on fake internet content related to floods and other disasters. It instructed its local branches to focus on identifying fabricated data, malicious editing, staged scenes, misattributed historical events, and those impersonating officials. Additionally, it called on social media platforms to enhance their content review processes.

Professor Gao believes "the answer is public education combined with law enforcement," emphasizing the "need to publicize real cases where people have been punished." He added that "the harm caused by AI-generated fake content is ultimately the result of how people choose to use the technology - not the technology itself," and that existing laws were sufficient to address it.

However, others are less confident. Professor Chen noted that "AI technology continues to evolve much faster than legal frameworks. As new legal questions emerge, AI misinformation remains a considerable challenge."

Peter Pang, a lawyer based in Shanghai and Washington, observes an explosion in cases dealing with these issues globally. He believes that companies creating AI apps and platforms used to publish fake videos and other material, along with the individuals who create misinformation, must be held legally accountable. He told the BBC, "The first line would be the platforms, because they're the ones that are the enablers. They're the ones that allow this to go out." He offered a hypothetical example of a person who runs over a pedestrian due to panic induced by a fake social media post. "The video creator might say 'you ran over somebody. That's your problem. I didn't do it.' But let's think it through. You created the foundation from which people reacted. If you created that panic, you need to have responsibility for consequences that occur."

Professor Xu warns that "AI-generated misinformation is likely to increase not only in volume but in the speed with which it spreads." He concluded, "Looking ahead, political elections, public health emergencies, armed conflicts, natural disasters and issues affecting people's livelihoods are all likely to become high-risk areas for AI-generated misinformation. Instead of waiting for false information to go viral before trying to contain it, governments and platforms should now be preparing in advance."