Using AI to clone itself to scam banks

WSJ reporter Joanna Stern tried using AI to create a fake version of herself that fooled both her family and the banking system.

Using artificial intelligence to insert faces into videos or fake voices, commonly known as deepfakes, is becoming more common and more dangerous. This is a real danger on the Internet.

In addition to sensitive videos, deepfakes can also cause serious harm if used for political purposes. The video of former US President Barack Obama's face being morphed, which went viral in 2018, showed that leading political figures can also become victims.

Deepfake porn is becoming a global problem. Photo: Wired.

To test the effectiveness of AI, WSJ reporter Joanna Stern used Synthesia, a deepfake tool, to create a simulated version of herself. The result was that the clone fooled both her family and the banking system.

Perfect Copy

Recounting her journey to clone herself, Stern said she spent months experimenting with Synthesia and training the AI from her original videos and audio files.

Using this tool is also very simple when users just need to enter any text and will receive a new video with their voice talking about that content.

“Because of the nature of my work, which requires a lot of working with dialogue and on screen, I thought AI could help me be more efficient and take some of the hard work out of it,” Stern said.

Joanna Stern records her voice to train Synthesia's AI. Photo: WSJ.

To train the tool, a WSJ reporter went to a recording studio and spent about 30 minutes filming herself and two hours recording her voice. Within a few weeks, Stern’s virtual version, which she dubbed Joanna AI, was ready to do her work.

Stern used his vacation day to put Joanna AI to work. For his first assignment, the WSJ reporter used ChatGPT to write a script for a TikTok video about iOS tips.

Stern then pasted the text into Synthesia. After initialization, her voice was heard over the video. "It was like looking at my reflection in a mirror, although the hand gestures and facial expressions were slightly different. It was very convincing," the reporter commented.

Stern said Joanna AI has impressive mimicry abilities for short sentences, but its weaknesses become apparent as the sentences get longer.

That’s also why short-form video platforms are the most suitable environments for AI, Stern said. According to her, users often pay less attention to details in TikTok videos, making computer-like features easy to miss.

For phone calls, Stern uses another AI app called ElevenLabs instead of Synthesia. This AI voice software has the advantage of users simply uploading their audio files to the tool without having to go to a recording studio.

Stern said Joanna AI has impressive mimicry abilities for short sentences. Photo: WSJ.

The WSJ reporter was surprised when ElevenLabs was able to clone her voice in just two minutes. Notably, the fee to use this application is only about $5 /month.

Compared to Synthesia, Stern commented that his cloned voice on ElevenLabs sounded more human, with good intonation and pronunciation.

To test its credibility, the reporter tried using the voice to talk to a relative. Stern’s sister said the AI voice sounded a lot like her, except it didn’t pause to catch its breath.

Meanwhile, when Stern used the AI voice to call his father to ask for his social security number, he immediately realized something was wrong because it sounded like a recorded audio file.

Fool the bank

Most notably, the AI was so effective that it fooled the voice biometric system on Stern's Chase credit card.

The WSJ reporter first gave Joanna AI a few questions that Chase would ask. She then dialed customer service and let the AI answer.

AI was so effective that it fooled the voice biometric system on Stern's Chase credit card. Photo: WSJ.

At the biometric step, when the automated system asked for her name and address, Joanna AI responded officially. As soon as she heard the AI's voice, Chase's system immediately recognized it as Stern and connected her to a bank representative.

After the successful test, Stern contacted Chase directly to say he had no dealings with the bank.

A Chase spokesperson later said the bank uses voice biometrics, along with other tools, to verify callers.

Chase added that the feature is intended to help customers identify themselves quickly and securely, but that customers will need to provide additional information to complete transactions and other financial requests.

What’s most worrying, though, is that apps like ElevenLabs have made a very good copy without much trouble. Users simply agree to a liability policy and can upload audio files and make their own copies.

Following the WSJ ’s testing, ElevenLabs said it would block accounts that appear to be creating fraudulent or illegal content. The startup is also working on an AI labeling tool for videos created on its platform.

Synthesia, meanwhile, requires users to verbally consent to audio and video files, which is why Stern had to go into the studio to film and record live with the company.

Using Synthesia is simple: when you enter any text, the user will receive a new video with their voice talking about that content. Photo: WSJ.

According to Siobhan Johnson, a spokeswoman for the Federal Bureau of Investigation (FBI), on average, families in the US have lost about $11,000 for each scam call.

In 2022, data from the US Federal Trade Commission (FTC) showed that Americans lost a total of $2.6 billion due to scam calls.

The development of cheap, accessible AI programs has allowed scammers to freely clone voices and create conversations that sound exactly like the original.

"This threat is not just hypothetical. We are seeing fraudsters weaponize these tools. They can create a relatively good voice clone with less than a minute of audio. For some people, even a few seconds is enough," said Hany Farid, a professor of computer science at the University of California, Berkeley.

Source link