Malicious Deepfakes: A New Era of AI Fraud

Recently actress  Scarlett Johansson denounced a deepfake video that went viral and used her voice and likeness to express opposition to Kanye West’s antisemitic comments. Before we lose a grip on reality , she urged lawmakers to regulate AI technology.

These incidents show that malicious deepfakes are real today and are being used to deceive, defame, and manipulate people. As a result, governments and public figures have called for immediate regulatory action.

Once laughable and entertaining, artificial intelligence generated deepfake media has evolved to go beyond entertainment or artistic learning and are now capable of being very advanced, malicious instruments of deception. Used maliciously, these artificial videos, audio, and images can be used to misinform the public, con businesses, ruin reputations, and influence the majority. Deepfakes in its initial context, are becoming an existential security and trust issue in the digital world.

What Is a Malicious Deepfake?

Although a simple photoshopped image or audio is easy to identify as inauthentic, deepfakes are difficult for the average individual to detect as fake. The goal of malicious deepfakes is to mislead the target audience into accepting something that did not happen, such as a fictitious financial transaction, a political forgery, or blackmail at an individual level. 

 In 2019, the CEO of a UK energy company was impersonated by a voice clone built by an AI in a phone call. The deepfake fooled an employee into transferring €220,000 to a supplier that was instantly withdrawn. 

Imposter Scams Are Becoming Perfect Con Artist

Malicious deepfakes that are meant to hurt people have made a common type of scam. AI-created audio or video, is another weapon in the scammers tool kit when it comes to impersonation scams. Scammers can convincingly impersonate people by using social engineering and AI audio/video together.  

Imposter scams primarily target businesses. Some scams like imposter scams target people with a lot of money. While others go after regular people. Deepfake impersonation scams are very dangerous because they get around natural trust in familiar sounds and faces.

Deepfake imposter scams are especially dangerous, as they bypass the natural human trust normally reserved for familiar sounds and faces. 

In Hong Kong, hackers produced a deepfake video of the CFO at a company and also of some of their colleagues when they had a video call. The employee that worked for finance, who was manipulated, transferred over $25 million in a transaction that didn’t look out of the ordinary as it was authorized by familiar faces and voices.

Disinformation and Political Propaganda

Deepfakes have forged a weapon for disinformation and political influence. In the current political climate, one convincing deepfake video could induce public panic, invite violence, or erode public trust in institutions.

Cybercriminals can produce false speeches, confessions, and endorsements made by politicians, activists, and journalists. 

As the 2024 European elections were near, a  number of  candidates posted on social media using deepfake videos in which they practically called for votes for extremist parties or said racist things. The posts generated some voter confusion, annoyance, and political polarisation even after they were confirmed to be phoney deepfakes.

Corporate Espionage and Financial Fraud

Within the business community, malicious deepfakes are being used for fraud on a never before seen scale. This is a part of an increasing number of whaling attacks driven by deepfakes that use executive identities to commit financial fraud. Deepfakes can create realistic audio or video messages from CEOs, CFOs, and other corporate leaders directing employees to authorize payments, share confidential information, or change access procedures.

A global bank implemented AI-enabled deepfake detection technology in its internal video conferencing platform after a failed deepfake attempt by cybercriminals when they attempted to extort a merger negotiation that included a deepfake video of the CFO. The fraud was discovered in time and prevented financial and reputational loss.

Blackmail and Reputation Destruction

At the individual level, malicious deepfakes are being used to stalk, blackmail, and harm one’s reputation. Criminals create fake videos or images  sometimes in a sexually explicit or compromising nature and threaten to put them online if a ransom is not paid.

For the victim, the fact that the deepfake exists no matter if it’s real or not can lead to professional safety, emotional distress and/or relationship safety  that can have adverse impacts. 93% of victims of non-consensual image abuse report experiencing severe mental distress, and more than 50% say it has affected their relationships or career, according to research by the Cyber Civil Rights Initiatives (CCRI).

Several journalists from India and Eastern Europe had deepfake videos made of them in compromising positions posted online. The deepfake videos went viral and created immediate public outcry leading to dismissals, resignations and disgrace despite some of the deepfake videos being discredited. 

How to Fight Back?

We need a multi-layered attack against malicious deepfakes through technology, policy, and public awareness. 

Deepfake Detection Technologies 

Companies like Facia provide Al-powered detection solutions which can detect deepfakes based on pixel anomalies. Certain technologies can be applied to communications platforms, financial systems, and social networks to be alerted to suspicious activity as it arises.

The U.S. Department of Defense’s Media Forensics (MediFor) program researches stochastic image analysis and detection methods to create safeguards to protect government and military communications from intrusion from deepfakes, etc.

Stronger Policies 

Governments are beginning to introduce legislation that regulates the production and distribution of malicious deepfakes. Take It Down Act permits minors to request that explicit AI-generated content be taken down on a federal level. International coordination is crucial, though, because there are currently no universal legal standards.

Public Awareness and Digital Literacy 

People need to be educated on how to critically interact with digital content, be doubtful of the veracity of media they consume and be able to report suspicious content. Educational institutions, workplaces, and social media all need to play a role in educating natural people of the dangers of deepfakes.

Facia Alert in the Age of Synthetic Lies

Deepfake that are meant to trick people are a harsh reminder of how technology can be used and even turned into dangerous tools of lies.

As deepfakes become more difficult to detect with the naked eye, there is a growing need for detection and verification, both advanced and commonplace. Facia’s biometric technology enables deepfake detection and identity verification in almost real-time, allowing individuals and businesses to protect themselves from synthetic threats. Facia AI technology provides a crucial layer of protection by detecting facial cues and differences in video footage that humans are unable to notice and delivering  100% accuracy on deepfake benchmarks, Stopping this new era of synthetic deception will require group action; technologists, lawmakers, businesses and every day online users alike. The next time you see or hear something that ‘shocks’ you online users, pause, question, and verify. And when it comes to protecting your digital identity online, Facia can give you peace of mind and keep you one step ahead.