Generative AI Fraud: How to Protect Your Firm Against Cyber-Enabled Crime

Generative AI fraud is a financial crime powered by artificial intelligence tools that can create realistic fake content on demand.

This includes:

• Deepfake videos
• Cloned voices
• Synthetic identities
• AI-written phishing emails
• Fabricated documents

Criminals use these tools to impersonate real people, trick employees into transferring money, bypass identity checks, and run scams at a scale that was not possible a few years ago.
Here is the short version:

If you can imagine a convincing lie, generative AI can now produce it in seconds. And the numbers back that up. According to Feedzai's 2025 AI Trends in Fraud and Financial Crime Prevention report, more than 50% of fraud now involves artificial intelligence, and 92% of financial institutions say fraudsters are using generative AI against them.

So what does this mean for banks, law firms, and the professionals who work inside them? It means the old rules of "trust but verify" need a serious upgrade.

And yes, it’s growing fast. Faster than most compliance teams can keep up.

Let's break down how this new era of fraud works, what the biggest risks look like, and what regulators are doing about it.

Why Generative AI Changed the Game: The New Face of Fraud

For decades, fraud had tells. Badly spelt phishing emails. Phone scammers with awkward scripts. Fake documents with slightly wrong logos. If you were paying attention, you could usually spot something off.

Generative AI removed those tells almost overnight.

Think about it this way: a scammer used to need time, skill, and sometimes a team to pull off a sophisticated con. Now they need a laptop and a subscription. Tools can translate languages perfectly, mimic a CEO's voice from a 30-second clip, and generate a video of a senior executive saying whatever the criminal wants them to say.

The result?

Fraud is faster, cheaper to produce, and painfully convincing. Here are the main ways criminals are using generative AI today:

• Deepfake video and audio: Realistic videos or voice clones of real people, often executives, used to authorise fake transactions.
• AI-powered phishing and smishing: Personalised, grammatically perfect emails and texts that reference real details scraped from social media.
• Synthetic identities: Fake people built from a mix of real and fabricated data, used to open accounts and apply for credit.
• Fabricated documents: Forged court orders, invoices, contracts, and government IDs that look genuine.
• Fake websites and trading platforms: Entire scam operations dressed up to look like legitimate financial firms.
• Chatbot-Based Social Engineering: AI bots engaging in real-time conversations to manipulate victims.

The Feedzai report found that 60% of financial professionals flag voice cloning as a major concern, while 59% point to AI-powered SMS and phishing scams, and 44% identify deepfakes as a serious threat.

And if you think the scale is manageable, take a breath: the UK government estimates that 8 million deepfakes were shared in 2025 alone, up from just 500,000 in 2023. That is a 16-fold jump in two years.

A Note on the Bigger Picture: Why This Is Bigger Than Deepfakes

This article focuses on fraud powered by impersonation tools such as deepfakes, voice clones, and synthetic identities. But that is only one slice of how generative AI is reshaping cybercrime.

In April 2026, Anthropic launched Project Glasswing alongside its most powerful frontier model, Claude Mythos Preview. According to Anthropic, Mythos's strong agentic coding and reasoning skills allow it to find software vulnerabilities autonomously, and in some cases, create working exploits for them. Within weeks, partners including AWS, Apple, Google, JPMorgan Chase, Microsoft, and the Linux Foundation had used it to uncover more than 10,000 high- or critical-severity flaws across the world's most important software systems.

Mythos itself is not on general release. Anthropic has held it back due to the cybersecurity risks of putting that capability in the wrong hands. We are not going to examine Mythos in detail here, because it does not perpetrate fraud in the same way that deepfakes and cloned voices do. But its existence makes the broader point hard to ignore.

Large language models are turbocharging both sides of the cyber fight. Attackers can now scale their efforts in ways that used to take whole teams of specialists. Defenders, in turn, are racing to use the same technology to plug the gaps before someone less scrupulous gets there first. The gap between attack capability and defence capability is shrinking.

The firms that win in this environment will be the ones that take that race seriously.

The Numbers Behind the Threat

Let's get concrete. Generative AI fraud is not a future problem. It is costing real money right now.

In the first half of 2025 alone, UK criminals stole £629.3 million from consumers, a 3% increase year-on-year. Every minute, £2,300 was lost to confirmed fraud, and eight people were victimised.

Social engineering, turbocharged by generative AI, sits at the heart of most of this. Investment scams, which often begin with AI-polished messages on social media, surged by 55% in H1 2025 to £97.7 million. Romance fraud losses jumped 35%. These are not random trends.

Criminals are leaning hard into emotional manipulation because AI makes it easy to personalise at scale.

Case Study 1: The £20 Million Arup Deepfake Scam

If you only remember one story from this article, make it this one.

In January 2024, a finance employee at Arup's Hong Kong office received an email from someone claiming to be the firm's UK-based Chief Financial Officer. The email requested a confidential transaction to be arranged. The employee, to their credit, was suspicious. It felt like phishing.

So what did the fraudsters do? They invited the employee to a video call.

On the call, the employee saw and heard the CFO, along with several colleagues he recognised. They looked right. They sounded right. They behaved like themselves. The employee's doubts melted away.

The employee made 15 separate transfers totalling around $25 million (HK$200 million, roughly £20 million) to five Hong Kong bank accounts. It was only after following up with Arup's UK headquarters that the employee realised every single person on that video call, except themselves, had been an AI-generated deepfake.

Arup, a 78-year-old British engineering firm behind landmarks like the Sydney Opera House and the Bird's Nest stadium, had just become the textbook example of how far deepfake fraud has come.
Rob Greig, Arup's Chief Information Officer, made a telling point afterwards.

This was not a traditional cyberattack. No systems were compromised. No data was stolen.

The attackers used what he called "technology-enhanced social engineering." In other words, they hacked the humans, not the computers.

What can we learn from Arup?

1. Suspicion is not enough. The employee actually spotted the initial red flag. The deepfake call overrode their judgment.
2. Multiple "familiar" faces amplify trust. Seeing several recognisable colleagues on the same call made the request feel legitimate.
3. Urgency and authority are weaponised. Impersonating the CFO created pressure that the employee felt unable to push back against.
4. Process beats instinct. Proper out-of-band verification, like calling the CFO on a known number, would have stopped this cold.

Case Study 2: The Florida Voice-Clone Extortion

Not every deepfake scam targets global corporations. Many are very personal.

In 2025, a woman in Florida received a phone call from what sounded exactly like her daughter. The voice was panicked, claiming there had been an emergency and she needed money urgently. The mother transferred $15,000. The scammers then pressured her for a further $30,000 using fake legal threats.

Her family intervened before the second transfer went through. But the first $15,000 was gone.

This attack required very little technical skill. A short audio clip from social media, a voice-cloning tool, and a believable script were all it took. The same playbook has been used against elderly parents, business owners, and even law firms. In one case reported by the American Bar Association, a New York law firm received invoice emails so perfectly mimicking a longstanding vendor, down to the writing style and reference numbers, that the firm nearly paid out.

The lesson?

Generative AI fraud scales both up and down. It can steal $25 million from a multinational or $15,000 from a family. Either way, the underlying trick is the same: make the victim believe they are talking to someone they trust.

The Tools Are Already in Criminal Hands

One uncomfortable point ties both Arup and Florida together: the tools that powered these attacks are not exotic, locked down, or hard to find.

Many sit one search away on the dark web, and an increasing number have moved into plain sight on the open internet.

Voice-cloning services, deepfake video generators, phishing kits, off-the-shelf AI chatbot personas, and even step-by-step playbooks. All of it is for sale, often for the price of a takeaway, and a good chunk is free.

You do not need a computer science degree to use any of it.

You do not need a team.

You do not need to write a line of code.

Point, click, paste in a few details, and out comes a convincing scam. That is the part most firms are still not fully reckoning with. The bar to commit serious AI-enabled fraud is now lower than the bar to set up a moderately complicated spreadsheet.

That is the uncomfortable truth driving everything that follows.

Why Finance and Legal Professionals Are Prime Targets

You might be wondering: why are banks and law firms such juicy targets?

A few reasons, and none of them are flattering.

First, you control the money. Or at least, you control the instructions that move it. A single authorised transfer can be worth more than months of smaller consumer scams.

Second, your work involves confidential, time-sensitive requests. Deals close quickly. Clients need things done yesterday. Fraudsters know this and exploit the urgency.

Third, your data is everywhere. LinkedIn, company websites, press releases, podcast appearances, and conference videos give criminals everything they need to build a convincing impersonation. The more senior you are, the more training data exists for AI tools to copy.

Fourth, the regulatory stakes are enormous. A successful fraud does not just cost money. It can trigger SMCR accountability questions, FCA scrutiny, GDPR breach assessments, and reputational damage that lingers for years.

And fifth, smaller firms are seen as low-hanging fruit. The average law firm, financial adviser, or even sizable City firm simply does not have the cyber defences of an HSBC, PwC, or a major global insurer. Systems are often older. Legacy tech and obsolete patches sit deep in the stack. Security training is patchier. Budgets are tighter. Specialist cyber and fraud staff are thinner on the ground, if they exist at all.

Fraudsters know all of this, and they actively prefer targets with weaker controls.

If you are a mid-sized firm telling yourself, "we are too small for criminals to bother with," that is exactly the assumption a fraudster is counting on. The economics of AI-enabled fraud mean that even modestly sized firms now sit comfortably inside the threat zone.

The Regulatory Response: Risk, Regulation, and Resilience

Regulators are moving, although they are often playing catch-up. Here is the current picture in the UK and beyond.

The UK Approach

The UK has chosen a sector-led, technology-neutral approach. Rather than a single AI law, existing regulators adapt their rules to cover AI risks.

• The Financial Conduct Authority (FCA) published an AI Update in 2024 setting out how its existing Consumer Duty, SMCR, and operational resilience rules apply to AI risks, including fraud.
• The Online Safety Act 2023 created duties for platforms to tackle illegal content, including fraud-related material, with fines of up to 10% of global turnover.
• The Data (Use and Access) Act 2025 introduced new offences around AI-generated intimate imagery and is part of a broader push to criminalise the creation, not just the sharing, of harmful synthetic media.
• The Payment Systems Regulator (PSR) introduced mandatory APP fraud reimbursement rules in October 2024. In the first three months, 86% of in-scope APP fraud losses were reimbursed to victims.
• In early 2026, the UK government launched a new deepfake detection testing framework, working with the City of London Police to test detection tools against real-world fraud and impersonation scenarios.

Don't Forget UK GDPR and the Data Protection Act 2018

Sitting underneath all of the above is the regime most boards already lose sleep over: the UK GDPR and the Data Protection Act 2018.

Almost every generative AI fraud scheme has personal data at its centre. Sort codes. Account numbers. Payment card details. Passport scans. National Insurance numbers. The richer the data a fraudster can scrape, buy, or extract, the more convincing their attack and the bigger their eventual payout.

That is where the data protection regulators come in. If your firm's personal data is compromised through inadequate technical or organisational controls, the Information Commissioner's Office (ICO) can fine you up to 4% of your global annual turnover or £17.5 million, whichever is higher. Where EU personal data is involved, EU data protection authorities can do the same under the EU GDPR, and several have shown an appetite for the upper end of that range.

For many boards, this is the point at which AI-enabled fraud stops being an "IT issue" and becomes a senior leadership issue. A failed transfer is painful. A failed transfer, plus a multi-million-pound ICO penalty, plus the inevitable client letters, is an existential one.

If you are trying to get budget and attention for AI fraud controls inside your firm, the data protection lens is often the one that gets decision-makers to lean in.

The EU Approach

The EU Artificial Intelligence Act, adopted in 2024, takes a more prescriptive stance. It prohibits certain manipulative AI practices outright and imposes strict transparency obligations on high-risk AI systems, including those used in financial services.

Penalties can reach up to €35 million or 7% of global annual turnover, whichever is higher.

What This Means for Firms

Regulators are no longer willing to accept "we didn't know" as a defence. The expectation is now that financial and legal firms will:

• Conduct AI-specific risk assessments
• Update internal controls to cover deepfake and synthetic media risks
• Train staff on AI-enabled social engineering
• Document governance decisions around AI tools, both those used internally and those targeting the firm
• Have incident response plans that cover deepfake scenarios specifically

Building Resilience: A Practical Playbook

So what can you actually do? Here is a quick framework that works whether you’re a junior analyst or managing partner.

1. Assume Verification, Not Trust

Every high-value instruction, especially those involving payments, account changes, or confidential transactions, should require out-of-band verification. That means calling back on a known number, not a number provided in the suspicious message.

No exceptions, even if "the CFO is on the line right now."

2. Build Friction Into High-Risk Moments

The Arup fraud worked partly because the employee felt social pressure to act quickly. Good controls deliberately slow things down for large or unusual transfers.

A two-person approval rule, a 24-hour cooling-off period, or a mandatory callback protocol can all save millions.

3. Train for the New Threats

Old-school phishing training is not enough anymore. Your team needs to know what a voice clone sounds like, what a deepfake video looks like, and what questions to ask when something feels off.

(Pro tip: ask someone on a video call to turn their head sideways or pick up a pencil. Deepfakes often struggle with profile views and unusual movements.)

4. Use AI to Fight AI

This is not ironic; it is essential. 90% of financial institutions are now using AI to detect fraud, and two-thirds have integrated AI in the past two years.

Tools that flag synthetic identities, detect unusual transaction patterns, and spot AI-generated content are no longer optional.

5. Audit Your Public Footprint

How much video of your senior executives is publicly available? How many voice recordings?

If the answer is "a lot," assume fraudsters have that data too. Executive protection now includes digital footprint reviews.

6. Create a Reporting Culture

The Arup employee did the right thing by reporting the fraud internally. But the damage was already done. A culture where staff feel safe reporting suspicions, not just confirmed incidents, catches threats earlier.

Fear of embarrassment is a fraudster's best friend.

Ready to Stay One Step Ahead?

Generative AI has given criminals a set of tools that used to require a film studio and a forger's workshop. Today, those tools fit on a laptop and cost less than a monthly gym membership.

But here is the good news: the same technology is available to defenders.

Banks, law firms, and training providers are investing in AI-powered detection, better controls, and smarter staff training. Regulators are moving, even if the pace feels slow. And the more we talk openly about cases like Arup, the harder it becomes for criminals to rely on silence and embarrassment.

The question is no longer whether your organisation will face a generative AI fraud attempt. The question is whether your people and processes will be ready when it happens.
Generative AI fraud is evolving faster than most compliance programmes can keep up. The firms that thrive in this environment will be the ones whose people understand the risks deeply, spot the red flags early, and know exactly what to do when something does not feel right.

Redcliffe Training's AI and Cyber-Enabled Fraud: Risk, Regulation and Resilience course gives you the practical knowledge to protect your firm, your clients, and your career. You will learn how AI-enabled fraud really works, how regulators expect you to respond, and the frameworks that turn theory into action.

Whether you are in compliance, risk, audit, legal, or senior management, this course equips you to lead your organisation's defence against the new face of financial crime and gives you the confidence to recognise a threat before it becomes a headline.

Secure your place on the AI and Cyber-Enabled Fraud course today and make sure you are the person in the room who spots the deepfake before the transfer goes through.

FAQ

How can businesses detect generative AI-driven fraud early?

Organisations should combine technology and process controls. AI-based anomaly detection can flag unusual transactions or communication patterns, while employee training helps identify deepfakes and phishing attempts. Strong verification procedures—such as multi-factor authentication and callback protocols for sensitive requests—are essential. Regular audits of systems and data access also reduce exposure. Early detection relies on layering these controls rather than relying on a single solution.